Answers that trace back to a source
Why I grounded a fintech's LLM agents in a Neo4j knowledge graph instead of relying on vectors alone.
In 2025 I worked as an AI engineer consultant with a fintech's AI/Data team. Internal teams and external clients wanted GenAI answers about company data. The problem was the usual one: an LLM on its own hallucinates, and it doesn't know the company's facts.
Vectors find similar text. They don't find facts.
Vector search is good at retrieving passages that look like the question. It is weaker when the answer depends on how things relate: which entity owns what, which rule applies to which case. So I integrated a Neo4j knowledge graph alongside the vector databases, and grounded LangGraph agents in both.
The graph takes modelling effort. In return, every answer can point back to the nodes and documents it came from. That traceability is what let the system sustain 90% answer accuracy, and it is what made people trust it.
Deliver where people already work
Instead of building a new interface, outputs went out as automated reports, newsletters and alerts wired into HubSpot. Less control than a custom UI, but no adoption barrier: manual reporting time fell by 60%.
Not every task needs the biggest model
I advised on the architecture to balance accuracy against cost, routing tasks to the model they actually needed. LLM and inference spend fell by 30%, and five of the use cases I proposed went into production.
The takeaway
- If users must trust an answer, make it traceable first and fluent second.
- Graphs and vectors are complementary: relationships from one, recall from the other.
- Adoption is a design decision: ship into the tools people already open.