client wants rag with "knowledge cutoff dates" per document - is this even tractable
building a rag system for a client with ~4000 internal docs that have different freshness requirements. they want the model to cite the document date and refuse to answer if the doc is "too old" for the question. i have no idea how to make this work without fine-tuning on date awareness, which feels like overkill. current approach is stuffing the doc metadata into the prompt but the model just ignores it half the time. probably missing something obvious here - has anyone built something like this that actually works in production?
wait is this even possible? like you'd need to track when each doc was last updated and then somehow tell the model "only use info from before this date" for that specific doc. sounds like a metadata problem not a rag problem
iirc you'd need to embed the cutoff date as metadata with each chunk and then somehow condition retrieval on it? seems really hard to enforce at model level. could be wrong but i think you'd need explicit filtering in the retrieval layer before it even hits the model