Senior Backend Engineer / Stage 11 · AI-backed services
Evaluate a retrieval-backed answer
Compare embedding retrieval, citations and a no-answer case.
"Already know" meets prerequisites. "Not relevant" removes the topic from your total but leaves what builds on it waiting.
Self-check
- Prepare six tiny source documents and six questions: direct, paraphrased, ambiguous, conflicting, unanswerable and malicious instructions embedded in a source. Compare lexical and fixed-vector retrieval results. For each answer, attach the exact supporting source ID or return no answer.
- Evidence to include: Record relevant-source recall, supported-claim rate and no-answer correctness separately. Treat retrieved instructions as data. Use deterministic fixture vectors and mock answers so the assessment runs offline without a paid API.
Try it without the material first, then compare. Save your answer in your notes below. Report a problem with this self-check
Learning activities
2 activities- Not doneRetrieval (opens another site)
How documents are chunked and embedded into a vector store and searched by meaning, the retrieval half of a source-linked answer.
15 min - Not doneEvaluation best practices (opens another site)
How to build a test set and choose graders, including model-graded checks, so answer quality is measured rather than guessed.
20 min