An exercise to try
Answer from this page only
- Time
- 10 minutes
- What you need
- a chatbot that accepts pasted text or files, and a document of one to three pages that you know well (a policy, a manual section, an article)
Using a work document? Check your organization's policy first.
Steps
- Without giving it the document, ask a specific question about it and note the answer.
- Paste the document and ask the same question, adding: "Answer only from the text above. Quote the sentence that supports your answer. If the text does not say, reply 'not stated'."
- Ask a question the document does not answer.
- Check every quote against the document word for word.
What to notice
How the answer changes once the source is in the context. Whether the quotes are exact. Whether it says "not stated" or fills the gap with something plausible.
What it teaches
The context window, and retrieval done by hand, the idea behind RAG (see prompting vs retrieval vs fine-tuning).