I mean, it's pretty damn reliable now. Has been for simple high contrast Q&A for a while now.
I've just been doing research and experiments for work related stuff.
Typically we've used plain embeddings for a lot of high contrast documents aka discrete facts.
However I've been working with a >1000 page document of complex procedures with incredibly low contrast where embedding falls flat.
There's top down/graph searching, bottom up/embedded; alts like colbert, reranking, reasoning, search agents and now (though seemingly quite new) specific search agent models.
Ultimately I found that a reasoning enabled search agent doing a hybrid of bottom up (with reranking) followed by top down, gave the absolute best results. Paired with Luna for cheaper and faster tokens it benchmarks pretty well even for vague references to procedures.
I would imagine that search specific models just coming out are even better and I'll have to evaluate using these but for now the above works well for us.
Having an agent get vector search results to use as anchors and then being able to explore the sections and subsections above that, then eventually digesting as much as is relevant (big context, cheap tokens) is amazing.
I've just been doing research and experiments for work related stuff.
Typically we've used plain embeddings for a lot of high contrast documents aka discrete facts.
However I've been working with a >1000 page document of complex procedures with incredibly low contrast where embedding falls flat.
There's top down/graph searching, bottom up/embedded; alts like colbert, reranking, reasoning, search agents and now (though seemingly quite new) specific search agent models.
Ultimately I found that a reasoning enabled search agent doing a hybrid of bottom up (with reranking) followed by top down, gave the absolute best results. Paired with Luna for cheaper and faster tokens it benchmarks pretty well even for vague references to procedures.
I would imagine that search specific models just coming out are even better and I'll have to evaluate using these but for now the above works well for us.
Having an agent get vector search results to use as anchors and then being able to explore the sections and subsections above that, then eventually digesting as much as is relevant (big context, cheap tokens) is amazing.