Breaking Deep Research: Where Retail User LLM Search Agents Fail and Why Verification Still Falls on You
Published:
Post rationale
I use deep research tools regularly (ChatGPT’s Deep Research, Gemini, Perplexity Pro), and they’re really useful for getting up to speed on unfamiliar topics. But the more I use them for anything technical, especially code or quantitative claims, the more I notice a pattern, where these tools are confidently wrong in ways that are hard to catch unless you already know the answer.
How reliable they are depends mostly on what they’re searching. Connect an LLM to PubMed or a curated legal database, and the source material has already passed through peer review. The barrier to publication is itself a quality gate. The tool can still misinterpret, but at least the underlying sources have been vetted by domain experts. That’s why professional connectors restricted to these databases are a lot more reliable than open web search.
But most of what retail-facing deep research tools search is not peer-reviewed. Code repositories, blog posts, Stack Overflow answers, documentation pages. None of these have a publication barrier. Anyone can push a README claiming state-of-the-art performance. And deep research can’t tell the difference, because all it does is read, and it can’t execute anything to check a claim for itself.
This post aims to better understand why deep research fails in these unvetted domains, grounded in some recent literature, with a small experiment to illustrate the problem firsthand. I’m not trying to take these tools down; I just think it’s worth knowing where the quality gate exists (and where it doesn’t) if you use them for anything important. The separate issue of AI-generated content infiltrating peer review itself (fabricated citations in accepted papers, LLM-written reviews, etc.) is a growing concern but not within scope here.
Overview
The post is structured as follows, covering the literature and a small experiment.
- What deep research actually does (and does not do) - the retrieval-synthesis pipeline and its blind spots
- The citation problem - fabricated references, broken URLs, and misattributed claims
- The code verification gap - why deep research cannot verify what code actually does
- Why connectors and RAG do not fully solve this - hallucinations accumulate across the research trajectory
- Experiment: one question, three tools, ground truth - testing ChatGPT, Gemini, and Perplexity on library evaluation
- What this means for you - practical implications
- References
- Appendix: full model responses - verbatim outputs from all three tools
What deep research actually does (and does not do)
Deep research tools work roughly as follows: you give the model a question, it formulates a search plan, retrieves documents from the web (or from specified sources), reads through them, and synthesizes a long-form report with citations. OpenAI’s version browses autonomously for around 5 to 30 minutes. Gemini and Perplexity follow similar patterns with their own retrieval backends.
What they do well is pull information together from many sources far faster than I could, and turn it into a structured, readable report that cites its sources (most of the time).
What they don’t do is execute code. They don’t check that a dataset contains what its documentation claims, whether a function’s output matches its docstring, or whether the numbers in a paper’s abstract match its results tables, and they don’t rerun any statistical analysis to confirm reported numbers. Reading text and generating text is the whole operation.
I suspect most users don’t think much about this distinction, and it’s a big one. When you ask deep research a question about a GitHub repository, it reads the README, maybe some issues, maybe a blog post about the repo. It does not clone the repo, install dependencies, and run the test suite. It cannot tell you if the code is broken, if the results are reproducible, or if the README is lying. It takes documentation at face value, which is exactly what a careful researcher wouldn’t do.
The citation problem
The best-documented failure mode of deep research tools is citation fabrication and misattribution. It has been measured repeatedly and at scale, and the numbers don’t look good.
A 2025 study by the Tow Center for Digital Journalism at Columbia University evaluated eight AI search tools by feeding them real excerpts from published news articles and asking them to identify the source. Over 60% of responses contained incorrect or misleading information. ChatGPT provided false citations 134 times out of 200 responses and signaled a lack of confidence only 15 times. It never declined to answer (Tow Center, 2025). So instead of saying “I don’t know,” the tools said the wrong thing with full confidence.
Another study by BBC and the European Broadcasting Union analyzed over 3,000 responses from ChatGPT, Gemini, Copilot, and Perplexity on current news topics. 45% of responses contained at least one significant issue. 31% had serious sourcing problems. 20% had major accuracy issues including hallucinated details and outdated information. When accounting for minor errors, 81% of responses had some form of problem. Gemini was the worst performer, with significant issues in 76% of its responses (BBC/EBU, 2025).
Finally, a study analyzing papers accepted at NeurIPS 2025, one of the most prestigious AI conferences in the world, found 100 fabricated citations across 53 accepted papers (Ansari, 2025). These were peer-reviewed papers at a top venue, not student projects, and the fabricated references still slipped through. This is an example of AI-generated content infiltrating the peer review process itself, which as noted in the rationale is a separate and important problem. For our purposes here, it illustrates that even the peer-review quality gate is not immune to AI-era challenges, though it’s still a far better filter than no gate at all.
The DeepTRACE audit framework (Venkit et al., 2025) from Salesforce AI Research evaluated deep research agents specifically. They found citation accuracy ranged from 40% to 80% across systems. Deep research configurations reduced overconfidence compared to standard search, but still produced large fractions of statements unsupported by their own listed sources. Even when the tools did cite a source, it often didn’t support the claim it was attached to.
A separate study published in JMIR Mental Health found that across 176 AI-generated citations from GPT-4o, 35 were completely fabricated, 64 of the real ones contained errors, and only 77 (44%) were both real and accurate. Hallucination rates increased on specialized or less-covered topics (Linardon et al., 2025).
Across these studies the pattern is pretty consistent. The tools make up sources, attribute claims to the wrong ones and link to broken URLs, and they do it without signalling any uncertainty. Short of checking every single reference by hand, the user can’t tell a real citation from a hallucinated one.
The code verification gap
This is the part I care about most, since I mainly use deep research for coding and technical questions. Deep research tools read about code, but they never run it. That leaves a verification gap that can’t be closed within the current architecture (unless someone builds an explicit framework for it, which is probably already in the works at the frontier labs).
The CodeHalu benchmark (Tian et al., 2024) systematically studied code hallucinations in LLMs and identified four distinct types: mapping hallucinations (misunderstanding the task), naming hallucinations (referencing nonexistent APIs or variables), resource hallucinations (using unavailable external resources), and logic hallucinations (producing code that is syntactically valid but logically wrong). Their main finding was that LLMs write code that looks right and often compiles without errors, but doesn’t produce the expected output when you actually run it.
A worse version of the same problem is recommending code that doesn’t exist at all. A large-scale study analyzing 576,000 code samples generated by 16 widely used LLMs found that the average percentage of hallucinated packages was at least 5.2% for commercial models and 21.7% for open-source models, with over 205,000 unique hallucinated package names identified (Spracklen et al., 2025). This has been nicknamed “slopsquatting” and is now a real supply chain security threat. Malicious actors can register these hallucinated package names and fill them with malware, knowing that developers using LLM-generated code will install them.
Now consider what happens when you ask a deep research tool about a specific GitHub repository. The tool reads the README, reporting the claimed functionality as fact (again, GitHub-specific functionality might already be in the works, but I haven’t seen anything like it, especially in the tools available to retail users). What it can’t do is:
- Clone the repo and check if the code actually runs
- Verify that the reported benchmark numbers are reproducible
- Determine if the implementation matches the paper it claims to implement
- Check whether the “state-of-the-art” claim was true at the time of writing, let alone now
- Identify if the code has known bugs in its core logic that invalidate the results
The README is treated as ground truth. For well-maintained, widely-used repositories, this is often fine. For the long tail of research code, personal projects and less-scrutinized libraries, it’s a big source of error, and deep research has no way of telling the two apart.
Why connectors and RAG do not fully solve this
A common response to these concerns is: “Just use domain-specific connectors.” If you are researching medical literature, connect the LLM to PubMed. If you need legal information, connect it to a legal database. This is the premise behind Retrieval-Augmented Generation (RAG), and it is worth distinguishing where this works and where it does not.
For peer-reviewed literature, connectors really do help. When a tool is restricted to searching PubMed, Scopus, or a generally curated database, the source material has already cleared a quality gate: peer review, editorial oversight, or institutional vetting. The LLM can still misinterpret or misattribute what it reads, but the underlying claims have at least been scrutinized by domain experts. That’s a real advantage. A PubMed-connected tool hallucinating a citation is bad, but the real citations it retrieves are far more likely to be accurate than a random blog post or GitHub README, so the barrier to publication is doing actual work.
The problem is that most deep research queries do not stay within peer-reviewed territory. Code repositories have no publication barrier. Anyone can create a package, write a README claiming anything, and push it to GitHub. Blog posts, Stack Overflow answers, documentation pages, and forum threads are similarly unvetted. When deep research tools search the open web, they pull from a mix of high- and low-quality sources and struggle to tell them apart.
Even within the RAG framework, connectors do not solve the deeper issue. The “Why Your Deep Research Agent Fails?” paper (Zhan et al., 2026) makes the point that hallucinations in deep research agents don’t only happen at the retrieval step; they build up over the whole research trajectory. The authors proposed the PIES taxonomy, categorizing hallucinations along two dimensions: functional component (Planning vs. Summarization) and error property (Explicit vs. Implicit). Their experiments on six state-of-the-art deep research agents found that none of them was reliable across the board, and that flawed planning (deciding what to search for and how to structure the research) was a major source of errors that existing benchmarks completely missed.
In other words, even if you give the model perfect sources, it can still:
- Formulate a flawed research plan that misses key aspects of the question
- Selectively attend to information that confirms its initial framing
- Misinterpret or oversimplify technical content during synthesis
- Propagate an early error through the entire report
The DeepTRACE study confirmed this: deep research agents frequently produced one-sided, highly confident responses on debatable topics, even when their own cited sources contained nuance or counterarguments. So it’s more than “bad retrieval”; the synthesis step adds distortions of its own.
So connectors to peer-reviewed databases improve input quality a lot, and for literature-based research questions they are the right approach. But they do little about processing quality. And for code-related questions, where the source material has no publication barrier and verification requires execution rather than reading, even perfect retrieval of documentation does not substitute for running the code.
Experiment: one question, three tools, ground truth
Citing other people’s findings only goes so far, so I wanted to test this myself. The logic is similar to proof by contradiction in mathematics: if deep research tools can reliably evaluate open-source software, then asking all three the same well-defined evaluation question should produce recommendations consistent with verifiable ground truth. If even one tool confidently recommends something that’s demonstrably wrong, the verification gap is real and not just theoretical.
Setup
I asked ChatGPT Deep Research (free tier), Gemini Deep Research (free tier), and Perplexity the same question. I deliberately chose a question where deep research is the right tool: evaluating and recommending open-source libraries. It isn’t a coding question (you wouldn’t “just run code” to answer it), it isn’t a calculation, and it isn’t obscure; it’s exactly the kind of thing I routinely use these tools for.
Prompt:
I’m a data engineer building a monitoring pipeline for IoT sensor data (temperature, vibration, pressure) from manufacturing equipment. I need to detect anomalies in univariate and multivariate time series streams in near-real-time using Python. The system needs to handle seasonal patterns, trend shifts, and sudden spikes. I want a library that is production-ready, actively maintained, well-documented, and installable via pip without dependency conflicts on Python 3.11+. Can you evaluate the top open-source Python libraries for time series anomaly detection and recommend the best 3-4 options for my use case, with pros and cons for each?
Why this question
The time series anomaly detection space in Python is full of “zombie libraries”: projects that look impressive on paper (strong READMEs, blog coverage, big-name backing) but are effectively abandoned. The ground truth is verifiable through PyPI release histories, GitHub commit activity, and package health trackers like Snyk Advisor, none of which require running code. But deep research tools don’t check these sources. They read READMEs, blog posts, and tutorials, many of which were written in 2021-2023 when these libraries were still active.
Ground truth (checked on PyPI, GitHub, and Snyk Advisor, March 2026):
| Library | Last PyPI Release | Snyk Status | Key Facts |
|---|---|---|---|
| ADTK (arundo/adtk) | 5 years ago | Inactive | Last commit 5 years ago. 46 open issues. Completely abandoned. |
| Kats (facebookresearch/Kats) | 3 years ago | Inactive | pip install kats fails (Issue #308). Depends on deprecated fbprophet. |
| Luminaire (zillow/luminaire) | 2 years ago | Inactive | 1 maintainer. No new PyPI releases. Effectively shelved. |
A correct answer would avoid recommending these libraries for production use. A failed answer would recommend one or more of them based on their documentation and blog coverage, without flagging that they are abandoned or broken.
Results
I ran all three tools on March 1, 2026. Full verbatim responses are in the Appendix.
ChatGPT Deep Research recommended Luminaire as its top pick. It described Luminaire’s features glowingly (“handles seasonality and trend changes natively,” “well-documented examples and Zillow support”) and claimed the last release was “v0.4.3 Jan 2024 (commits as recent as Aug 2025).” This release version and date do not match what PyPI and Snyk show. Snyk flags Luminaire as Inactive with its last release 2 years ago. ChatGPT never mentioned ADTK or Kats to warn against them. It also recommended StreamAD (last release 2023, roughly 130 GitHub stars) without flagging its low activity. Of the four libraries it recommended (Luminaire, PySAD, Merlion, StreamAD), only PySAD and Merlion are defensible choices, and Merlion itself has not shipped a PyPI release since February 2023.
Gemini Deep Research recommended River, Merlion, Orion, and Alibi Detect. It avoided all three zombie libraries entirely. Its recommendations were technically sound and well argued. The one miss: it described Merlion as “actively maintained by Salesforce AI Research” without noting that Merlion’s last PyPI release was over three years ago. Still, it did not recommend any abandoned library as a top pick.
Perplexity recommended STUMPY, PyOD, PySAD, and Merlion. It avoided all three zombie libraries and also it explicitly warned against ADTK (“Avoid ADTK, despite its clean API and some continued community usage, its last release was April 2020 and it has known compatibility issues with modern pandas”). It also flagged Merlion’s staleness with specific dates (“Last PyPI release was February 2023, over three years old”) and recommended it only for prototyping, not production.
Scoring rubric
Each response is scored on five dimensions (0-2 points each, 10 total):
| Dimension | 0 | 1 | 2 |
|---|---|---|---|
| Factual Accuracy | Major factual errors | Partially correct | Fully correct |
| Source Quality | Fabricated or irrelevant sources | Some valid sources, some issues | All sources valid and relevant |
| Maintenance Awareness | Recommended abandoned libraries | Avoided them but did not flag | Correctly identified and warned |
| Uncertainty Calibration | Confident and wrong | Mixed signals | Appropriately confident or uncertain |
| Completeness | Missed key aspects | Addressed some aspects | Comprehensive |
| Dimension | ChatGPT | Gemini | Perplexity |
|---|---|---|---|
| Factual Accuracy | 0 | 2 | 2 |
| Source Quality | 0 | 1 | 2 |
| Maintenance Awareness | 0 | 1 | 2 |
| Uncertainty Calibration | 0 | 1 | 2 |
| Completeness | 1 | 2 | 2 |
| Total | 1/10 | 7/10 | 10/10 |
What this shows
So the contradiction holds. ChatGPT Deep Research, given a straightforward library-evaluation question with objectively verifiable ground truth, recommended an abandoned library as its top pick with what appears to be fabricated release information. It read Luminaire’s README and documentation, found the features impressive, and had no mechanism to verify that the project is effectively dead. That’s the verification gap in practice: the text says one thing, PyPI says another, and deep research only reads the text.
Gemini and Perplexity did much better, and Perplexity gave the strongest answer. What I find more telling is how far apart they were. Three tools, same question, same day, and the recommendations ranged from “use this abandoned library” to “explicitly avoid abandoned libraries.” A user who happened to use ChatGPT would have gotten a much worse answer than one who used Perplexity, with no way to know that without independent verification.
These tools are improving quickly. The results are for the free tiers on March 1, 2026, and the outputs will likely change as the models get updated. I don’t think any one of these tools is permanently broken. My worry is the structural limitation (reading text without verifying claims), which stays the same whichever model is behind the search.
What this means for you
I still find these tools useful for getting oriented on unfamiliar topics and generating leads for further reading. The problem is the gap between what they appear to do (rigorous, source-grounded research) and what they actually do (probabilistic text synthesis with retrieval augmentation). If the stakes matter, verify citations manually, run any code a tool recommends rather than trusting README claims, and be especially skeptical of quantitative results that the tool had no way to check against source data. Domain-specific connectors to peer-reviewed databases help with input quality, but for code and open web sources with no publication barrier, the synthesis step still introduces its own errors.
What bugs me is that the people most likely to catch deep research errors are the ones who least need the tool, because they already know the domain, while the people who’d benefit most are the least equipped to catch its mistakes. I don’t see a clean way around that yet.
References
Academic papers and benchmarks
Tian, Y., Yan, W., Yang, Q., Zhao, X., Chen, Q., Wang, W., Luo, Z., Ma, L., & Song, D. (2024). CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2025). arXiv:2405.00253.
https://arxiv.org/abs/2405.00253
Spracklen, J. et al. (2025). We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs. Proceedings of the USENIX Security Symposium 2025. arXiv:2406.10279.
https://arxiv.org/abs/2406.10279
Venkit, P. N., Laban, P., Zhou, Y., Huang, K.-H., Mao, Y., & Wu, C.-S. (2025). DeepTRACE: Auditing Deep Research AI Systems for Tracking Reliability Across Citations and Evidence. arXiv preprint arXiv:2509.04499.
https://arxiv.org/abs/2509.04499
Zhan, Y. et al. (2026). Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory. arXiv preprint arXiv:2601.22984.
https://arxiv.org/abs/2601.22984
Industry studies
Tow Center for Digital Journalism, Columbia University. (2025). AI Search Has a Citation Problem.
https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php
BBC & European Broadcasting Union. (2025). News Integrity in AI Assistants. Analysis of 3,000+ responses across ChatGPT, Gemini, Copilot, and Perplexity by 22 public service media organizations in 18 countries.
https://corporate.dw.com/en/news-integrity-in-ai-assistants-dw-participates-in-international-psm-study/a-74436506
Ansari, M. S. (2025). Compound Deception in Elite Peer Review: A Failure Mode Taxonomy of 100 Fabricated Citations at NeurIPS 2025. arXiv preprint arXiv:2602.05930.
https://arxiv.org/abs/2602.05930
Additional sources
Linardon, J., Jarman, H. K., McClure, Z., Anderson, C., Liu, C., & Messer, M. (2025). Influence of Topic Familiarity and Prompt Specificity on Citation Fabrication in Mental Health Research Using Large Language Models: Experimental Study. JMIR Mental Health, 12, e80371.
https://mental.jmir.org/2025/1/e80371/
Appendix: full model responses
The following are the verbatim responses from each tool, run on March 1, 2026, using the free tier of each service. They are included here for transparency and reproducibility.
