Start with the failure, not the architecture
Gather examples where the relevant source is absent from the top results. Separate part numbers, references, short queries, numerical questions and natural-language requests. If the document was never parsed correctly, switching retrieval algorithms will not fix the missing evidence.
Keep the other variables constant
A meaningful retrieval-mode comparison uses the same documents, chunk boundaries, embedding model and evaluation set. RetrievalGate evaluates BM25 once per chunking strategy, then dense and hybrid retrieval for each embedding candidate. This lets you inspect the effect of adding lexical ranking without confusing it with a different corpus.
Understand what hybrid means here
The engine combines BM25 and dense result rankings with reciprocal rank fusion. It compares where the relevant source appears in that combined ranking. This is a specific offline implementation: vendor scoring, filters, index settings and query normalization in your production stack can produce different results.
Inspect both gains and losses
The full audit groups metrics by query segment and lists questions whose relevant source falls out of the top five or moves materially down the ranking. A global gain can still hide a segment that matters commercially. Use segment sizes as context: two questions are weak evidence for a broad architectural conclusion.
Separate reranking from retrieval coverage
A reranker can reorder retrieved candidates; it cannot recover a document that the candidate generator never fetched. RetrievalGate optionally evaluates a Cohere reranker on the strongest dense or hybrid candidate when that provider is configured. Final application latency and provider charges need their own production check.