Literature Review Pipeline

Stage 1

Collect Corpus

Fetch metadatapaper_scanner.py fetch pulls OpenReview and ACL Anthology records.

Persist raw dataJSON and XLSX are saved before any filtering.

Human decisionChoose venues, years, accepted/rejected status, and whether Findings are included.

PitfallOpenReview tab names vary; ACL/EMNLP are not OpenReview-first.

Stage 2

Local Prefilter

Bucket taggingMatch human, agent, data_science, and graph.

Tier assignmentExport high-recall candidates by overlap rules.

Human decisionTune keywords and decide whether recall or precision matters more.

PitfallBroad terms like agent, eda, and decision-making create useful noise.

Stage 3

LLM Triage

Structured triagepaper_scanner.py triage scores uncertain candidates using the anchor file.

Auto keepStrong prefilter tiers are retained without LLM calls.

Human decisionMaintain problem_space_anchor.md and decide what counts as keep/borderline/drop.

PitfallLLM triage is not final truth; it mirrors the anchor quality.

Stage 4

Manual Screening

Read efficientlytriage_browser.html supports filtering, search, and abstract review.

Explore clustersintersection_browser.html shows bucket combinations and interpretations.

Human decisionKeep notes for core, benchmark, method, background, contrast, and drop.

PitfallXLSX is useful for checking, but painful for long abstract reading.

Stage 5

Freeze Seed Set

Finalize layersSeparate core papers from benchmark/reference/method/background papers.

Reverse analysisUse manual notes to infer the real relevance boundary.

Human decisionDecide whether the current venue set is enough before expanding scope.

PitfallExpanding venues too early can restart the entire reading burden.

Human Intervention Points

Before fetch

Set venue/year/status scope. Example: ML/NLP top venues first; KDD/WSDM/CIKM only if citation chains demand it.

Before prefilter

Edit keyword buckets or pass CLI overrides. The first pass intentionally favors recall.

Before triage

Update problem_space_anchor.md with positive examples, boundary notes, and known false positives.

During screening

Record short notes. Useful labels include core, benchmark, method, background, contrast, and drop.

After screening

Audit what was kept and dropped to refine the problem boundary and justify the review scope.

Optional Side-Topic Rescue

Trigger

Use when a narrow line is likely missed by the main funnel, such as fully autonomous graph-problem benchmarks.

Seed signal

Provide seed papers or notes. Preprints can guide the search even when they are absent from the accepted corpus.

Local scan

seed_rescue_scan.py or a focused preset scans the cached corpus and writes a separate output directory.

LLM pass

Optional. Use it when the rescue pool is still noisy, as with graph problem benchmark summaries and relevance scores.

Boundary

Side-topic results do not merge into the main human-agent review unless explicitly promoted.