SkepTiX

overview.md

SkepTiX is built around a question PDF chatbots never ask: "Why should I believe this paper?" A LangGraph StateGraph passes a shared WorkingMemory object through five agents — Claim, Evidence, Memory, Skeptic, Report — each mutating specific fields. The final output is a structured Markdown report with claim-evidence tables, limitation analysis, reviewer questions, and an executive summary.

pipeline.drawio

1. PDF Upload → font-size-aware Section Parser (PyMuPDF) identifies headings across IEEE, ACM and other formats by size ratio, not hardcoded aliases.
2. ClaimAgent — parallel ThreadPoolExecutor (capped at 5) extracts first-person claims per section; deduplicates by lowercased text.
3. EvidenceAgent — chunks sections into ChromaDB (knowledge_base), retrieves top-1 chunk per claim via vector search, calls LLM to score support strength (Strong/Medium/Weak).
4. MemoryAgent (read) — queries analysis_memory ChromaDB collection for semantically similar previously-analysed papers before the Skeptic runs.
5. SkepticAgent — single structured LLM call produces Limitations and ReviewerQuestions; explicitly instructed to surface concerns, not declare verdicts.
6. ReportAgent — parallelises two expensive LLM calls (Executive Summary + Overall Assessment); remaining sections (claim table, evidence mapping, limitations, questions) generated deterministically.
7. MemoryAgent (write) — persists AnalysisRecord to analysis_memory for future retrievals; report cached by content hash so re-uploads return instantly.

engineering.log

• 3-tier LLM routing: Gemini 2.5 Flash primary (2 retries) → Qwen3 14B via Ollama fallback (think=False drops latency from ~40s to ~1.5s).
• Report cache by content hash eliminates re-analysis of the same paper entirely.
• ChatbotAgent is a standalone agent outside LangGraph — grounds answers in retrieved chunks + the stored Overall Assessment; never re-runs the pipeline.
• CI via GitHub Actions: ruff lint + pytest on every push.

status.log

Shipped July 6, 2026 after a 5-day sprint. All five agents implemented and wired; Next.js frontend replaced the initial Streamlit prototype on day 5. No empirical benchmarks published yet — planned evaluation: claim extraction F1 and a 5-dimension human rubric (usefulness, explainability, limitation quality, reviewer question insightfulness, overall report quality).