Turning two scattered procurement documents into grounded, page-cited answers — retrieved, reasoned, and never invented.
What if an assistant could answer a question using two different company documents at once — while still showing exactly where the answer came from?
Both files below are converted into a single searchable vector knowledge base. When a question arrives, the system retrieves the most relevant context and hands it to a local model — producing an answer grounded in the documents, not the model's general knowledge.
Covers document indexing, cross-document retrieval, policy reasoning, cited sources, and a deliberate out-of-scope question.
A typical question-answering system looks at a single document. This one reasons across both — matching a live figure from the review against the rule that applies to it.
The two PDFs are loaded and split into meaningful chunks. Overlap preserves context between neighbouring sections so a sentence split across a boundary isn't lost.
Each chunk is converted into a vector representation using nomic-embed-text.
Embeddings are stored in a single ChromaDB collection — enabling search across both documents together, not separately.
Relevant chunks are retrieved per question, passed to llama3.2 running locally via Ollama, and the final response includes the source file and page reference.
Covering retrieval, calculation, policy interpretation, cross-document reasoning, table interpretation, and hallucination resistance — deliberately including questions where a plausible-but-wrong answer would be easy to produce.
Built on llama3.2, a lightweight local model — chosen after the assignment's provided API key returned an authentication error, per HCL's own guidance to use alternatives like Ollama.
Table interpretation — for the approval-authority question, the correct table was retrieved, but the model selected the wrong row.
Unsupported numerical reasoning — for the Kaveri Metals question, correct clauses were identified, but the model generated a debit-note figure that couldn't be derived from the available data.
Retrieval was not the bottleneck — raising top_k from 5 to 6 didn't fix either case, since the correct information was already present. The remaining errors point to model reasoning and table interpretation, not missing context.
Run again whenever the source PDFs change.
Better handling of policy and approval tables.
Combining semantic search with keyword-based retrieval.
Improving the ordering of retrieved context.
Checking generated numbers against retrieved evidence.
Reducing errors in multi-step policy interpretation.
Automatically verifying cited pages support the answer.