AI Research Agent

Most research tools sit at one of two extremes. Chatbots write fluent summaries but invent citations. Search engines return links and leave the synthesis to you. Neither one verifies anything.

I wanted a system where every paragraph traces back to a real source, and where disagreement between sources is surfaced rather than quietly averaged away. That constraint — traceability first — drove almost every architectural decision below.

The result is deliberately not a chatbot. It is an agentic pipeline with discrete, inspectable stages.

1. Thirteen sources behind one interface

   Wikipedia, SerpAPI, Tavily, arXiv, Semantic Scholar, CrossRef, GitHub, HuggingFace, Kaggle, GNews, NewsAPI, PubMed and OpenAlex each live behind a single SourceAdapter contract. Adding a fourteenth is one new file plus one line in the registry — no changes to the agents that consume them.

2. Provider failover instead of a single LLM

   Gemini → OpenRouter → Groq → Ollama, with automatic failover down the chain. Swapping the model that powers any stage is pure configuration, so a rate limit or an outage degrades the run instead of ending it.

3. Hybrid retrieval, not vectors alone

   Pure vector search misses exact terms; pure keyword search misses paraphrase. The retriever runs both — ChromaDB with sentence-transformer embeddings alongside BM25 — and merges them with reciprocal rank fusion. Chunking is sentence-aware, so citations never land mid-sentence.

4. Fact checking as its own pipeline stage

   Verification is a stage, not a prompt suffix. Claims are cross-checked across sources, conflicts are detected and resolved to a “likely correct” reading, and both per-claim and overall confidence scores travel with the finished report.

5. Charts built from data, never from the model

   Publication trends, GitHub stars and model-download figures are computed deterministically from the raw API responses. The LLM is trusted only with the knowledge graph and the Mermaid mind map — the places where being generative is actually the point.

6. Production hygiene from the start

   Structured logging that records latency and token usage per stage, Redis caching that degrades gracefully when Redis is absent, exponential-backoff retries, JWT auth, Celery workers and Docker Compose. The test suite mocks every external API, so pytest runs with no network and no keys.

1. Configure

   Copy the example env file and add keys. Wikipedia, arXiv, CrossRef, OpenAlex, PubMed, Semantic Scholar, GitHub and HuggingFace all work with no key at all — a single LLM key is enough for a working system.

   cp .env.example .env

2. Run the backend

   SQLite is the default database, so there is no setup step. Redis is optional in development; without it the cache simply no-ops, and jobs run in-process unless USE_CELERY=true.

   cd backend
   python -m venv .venv && .venv\Scripts\activate
   pip install -r requirements.txt
   uvicorn app.main:app --reload      # http://localhost:8000/docs

3. Run the frontend

   cd frontend
   npm install
   npm run dev                        # http://localhost:3000

4. Or skip both with Docker

   docker compose up --build

5. Start a research job

   The API returns a job_id immediately. Polling that job shows progress moving through each agent stage, so a long run stays observable instead of becoming a spinner.

   POST /api/v1/research
   {"topic": "Quantum Computing in Healthcare",
    "depth": "standard",
    "language": "en"}

6. Collect the output

   Download the cited report as Markdown, HTML, PDF or DOCX — or stream the generated podcast.

   GET /api/v1/research/{job_id}
   GET /api/v1/research/{job_id}/report?format=pdf