Atlas

Atlas maps stated research debt in academic papers through a deterministic, rule-based, human-reviewed workflow.

It extracts explicit limitations, future-work claims, missing evaluations, open problems, threats to validity, reproducibility concerns, and related statements from all detected sections of English .txt and .pdf papers. Every retained item traces to a source file, section heading, and character span.

Atlas does not use LLMs, embeddings, APIs, semantic search, or summarization. PDF text is extracted locally in reading order with Poppler's pdftotext; Atlas does not invent research gaps, claim novelty, automate literature review, or replace reading papers.

Install

Atlas requires Python 3.11 or newer and has no runtime dependencies outside the standard library. PDF input additionally requires Poppler's pdftotext executable. Plain-text input does not.

PDF conversion defaults to --pdf-mode raw; layout preserves approximate geometry, while auto compares both with a deterministic structural metric. Use --suppress-appendices, --workers N, and --cache-sections for optional appendix filtering, per-paper parallelism, and content-addressed section reuse.

Projects can set extract.rules_file = "rules_local.toml". The overlay accepts regex arrays under [patterns] and non-negative weights under [rule_scores] and [section_scores]; Atlas validates regex syntax when loading the project.

python3 -m venv .venv
.venv/bin/pip install -e .

Quickstart

atlas init my-atlas-project
cp examples/sample_papers_txt/*.txt my-atlas-project/papers_txt/
atlas run --project my-atlas-project
atlas review set --reviews my-atlas-project/out/reviews.jsonl --candidate-id CANDIDATE_ID --label good --note "clear stated limitation" --output my-atlas-project/out/reviews.jsonl --force
atlas promote --candidates my-atlas-project/out/candidates.jsonl --reviews my-atlas-project/out/reviews.jsonl --output my-atlas-project/out/debt_records.jsonl
atlas map build --records my-atlas-project/out/debt_records.jsonl --output my-atlas-project/out/map_nodes.jsonl
atlas validate --output my-atlas-project/out
atlas stats --output my-atlas-project/out

See Quickstart and Workflow for a complete walkthrough.

Workflow

source .txt or .pdf papers
  -> papers.jsonl + sections.jsonl + candidates.jsonl
  -> reviews.jsonl (manual judgment)
  -> debt_records.jsonl (reviewed records)
  -> map_nodes.jsonl (deterministic evidence groups)
  -> static HTML and Markdown exports

Commands

atlas extract                  atlas review init|summary|set
atlas promote                  atlas debt summary|tag|tag-file|status|report
atlas map build|summary|report|status
atlas init                     atlas run
atlas manifest                 atlas validate
atlas stats                    atlas failure-log init
atlas export debt-markdown|map-markdown|index
atlas workflow                 atlas version
atlas doctor                   atlas release-check

Use atlas COMMAND --help for options. atlas extract --debug-rules records exact stable pattern IDs and score components; --target-sections-only restores the narrower legacy scan. Existing review files are preserved by atlas run. Commands that can replace user-edited reviews, debt records, or map nodes refuse to overwrite unless --force is explicit. Artifact writes use atomic file replacement.

atlas doctor reports whether pdftotext is available and prints its resolved path and version. Unreadable, encrypted, malformed, or unconvertible inputs are skipped with the converter's failure reason on stderr.

Artifacts

Artifact	Purpose
papers.jsonl	Input paper identity and size
converted_text/*.txt	Exact text used for PDF section and candidate offsets
sections.jsonl	Detected headed sections and boundaries
candidates.jsonl	Exact extracted spans, matched rules, and scores
reviews.jsonl	Manual labels and notes
debt_records.jsonl	Reviewed, structured research-debt records
map_nodes.jsonl	Deterministic groups of traceable debt evidence
manifest.json	Corpus filenames, sizes, and content hashes
report.html, debt_report.html, map_report.html	Static HTML inspection reports
exports/*.md	Deterministic human-readable exports

Field definitions are documented in Schemas.

Real corpus testing

Create a corpus manifest, run the project workflow, manually review candidates, then run atlas validate and atlas stats. Keep a deterministic failure-log template with atlas failure-log init --output notes/failure_log.md and record missed sections, weak rules, boundaries, spans, and grouping issues. Atlas output is triage evidence, not novelty evidence; verify all claims by reading the source papers and relevant literature.

Limitations

• PDF ingestion requires the local pdftotext executable. Candidate offsets for PDFs refer to its deterministic UTF-8 text output, not PDF byte offsets or page coordinates.
• Regex patterns target English academic prose.
• Rule-based extraction can miss nuanced statements and retain false positives.
• Human review is required before promotion and interpretation.
• Map nodes are grouped evidence, not generated ideas or novelty claims.

See Limitations for details.

Status

Atlas 1.0.1 is the current stable release of the deterministic research-debt mapping workflow.

It is suitable for small-to-medium plain-text academic corpora, manual review workflows, and traceable research triage.

It does not generate research gaps, verify novelty, or replace reading papers.

Development

.venv/bin/python -m pytest tests/ -v
atlas release-check

Atlas is licensed under the MIT License. See LICENSE.