client wants "explainable" rag citations, spent week testing different methods
Client asked for rag system where every answer includes citation to source document, but they also want "explanation" of why that document was retrieved. Like "I'm citing document X because it contains the most relevant information about topic Y". Tested three approaches: 1. Prompt model to explain retrieval after generating answer - works but adds 200ms latency 2. Use embedding similarity score as explanation - fast but clients say "cosine distance 0.83" is not human readable 3. Fine-tune small model to generate explanations from retrieval metadata - hallucinated explanations that sounded good but were wrong Anyone solved this problem? Specifically looking for method that is both fast and actually accurate. Not sure if this is even solvable requirement or if I need to push back on client)
we ended up just returning the retrieval score and calling it confidence lol. clients never push back
what's the actual test setup? are you measuring coherence degradation on synthetic data or real client requests? specifically interested in whether schema validation order affects context retention or just parse errors
what did you end up using?
curious what you ended up using too. we've been trying rag with sentence-level embeddings and returning the top-k sentences as citations but clients keep asking why citation X was chosen over Y
just return retrieval score and call it confidence. clients never push back on that