Building the Research Documentation Agent
parsidion · Claude Code Build Session · 2026-03-12
user
~/Repos/parsidion
$
claude
✦ Welcome to Claude Code!
claude-sonnet-4-6 · parsidion
The problem
Every Claude Code session starts cold. Web research done in one session — API docs, framework patterns, debugging solutions — vanishes when the session ends. The next session re-investigates the same ground. The vault captures project decisions, but nothing drives external knowledge into it automatically.
Sonnet-powered
vault-first
proactive trigger
311 lines
Design Goals
What success looks like
- Vault-first — never re-research what's already known; check the vault before going to the web
- Proactive — triggers automatically from user intent, not just explicit requests
- Structured output — YAML frontmatter, wikilinks, confidence levels on every note
- Multi-source — Brave Search + Web Fetch + optional NotebookLM deep synthesis
- Project-aware — saves to vault AND to
docs/research/ if the project has one
- Style-guide compliant — reads the project's style guide before writing a word
Model choice: Sonnet over Haiku. This agent synthesizes across multiple sources,
judges source quality, and writes well-structured prose — it needs reasoning
capability, not just recall speed.
Frontmatter: Getting the Trigger Right
The description field is the agent's only auto-invocation surface. It must fire broadly enough to catch proactive research moments, but never so broadly that it fires on unrelated tasks. Four concrete scenarios were chosen.
The four trigger scenarios
1User exploring new tech — "I'm thinking about using WebGL for 3D rendering"
2User mentions unfamiliar APIs — "I need to implement signed distance functions"
3User debugging complex issue — "I'm getting shader compilation errors with GLSL"
4Beginning a new project — "Let's start building a fractal renderer"
model: sonnet # synthesis + judgment, not just lookup
color: pink # visually distinct from vault-explorer (purple)
Step 1: Vault-First Search
The agent checks the vault before touching the web. It dispatches vault-explorer, which now uses a three-tier fast-path: semantic search (vault_search.py, MRR 0.889) → metadata search (vault-search filter mode, note_index table) → grep-based fallback. DB-absent is handled gracefully at every tier.
vault-explorer 7-step procedure (dispatched)
# Step 1: Semantic search — intent-aware
vault_search.py "QUERY" --json # ≥3 results (score ≥ 0.35) → done
# Step 2: Metadata search — indexed sub-ms DB query
vault-search --tag TAG --folder Patterns # ≥3 results → done
# Steps 3–5: CLAUDE.md orient → extract signals → grep folders
# Steps 6–7: Rank+read top 5 → synthesize ## Answer + ## Sources
update_index.py populates note_index on every rebuild — vault notes written by this agent are available for metadata search immediately after the index run.
Step 2: NotebookLM Deep Synthesis (Optional)
NotebookLM can run deep web research that takes minutes. The key design constraint: non-blocking. The agent fires it off, spawns a background subagent to wait for results, then continues immediately to Brave Search — the two pipelines run in parallel.
Non-blocking pattern
# Check availability first — skip silently if absent
notebooklm status
# Create notebook + start deep research (non-blocking)
notebooklm create "Research: [topic]"
notebooklm source add-research "[query]" --mode deep --no-wait
# Background subagent waits; agent continues to step 3
# notebooklm research wait -n [id] --import-all --timeout 1800
optional — graceful skip
parallel with web research
background subagent
Steps 3–4: Web Research + Content Extraction
Brave Search strategy
- Get current date first:
date -Iseconds
- Broad → narrow: landscape first, then specific
- Query patterns: "official documentation", "best practices", "after:2023"
- Prioritize: official docs > academic > reputable blogs
- Cross-reference 2+ sources before committing to any claim
Content extraction
agentchrome page html → html-to-md pipeline
- Strips nav, banners, cookie notices — preserves code fences
- Fallback:
curl -sL URL | html-to-md -
- Flag conflicts — document both perspectives
- Preserve attribution and source URLs
Source hierarchy enforced at write time: confidence: high only when multiple
authoritative sources agree. confidence: low when only one source found or
source is a blog/forum rather than official documentation.
YAML Frontmatter Standard
Every vault note must have frontmatter that the indexer, vault-explorer, and summarizer can all parse. Six required fields — no optional fields in the core set so that notes are always machine-readable without conditional handling.
Canonical form
---
date: 2026-03-12 # ISO 8601, set at write time
type: research|language|framework|tool|pattern|debugging
tags: [topic-tag, subtopic-tag] # short, single-word preferred
confidence: high|medium|low # multi-source → high
sources:
- https://example.com/docs
related:
- "[[related-note-name]]" # at least one — no orphan notes
---
✓ No orphan notes: related must contain at least one [[wikilink]]
✓ type drives vault subfolder routing + graph color assignment
Documentation Structure Template
A fixed 9-section template ensures every note is navigable and complete. Developers who need a quick answer hit Overview. Developers who need to implement something hit Examples. Developers who hit an error hit Common Pitfalls.
Required sections
1Overview — what and why
2Key Concepts — terminology
3Technical Details — APIs, algorithms
4Best Practices
5Common Pitfalls
Optional but expected
6Performance Considerations
7Examples — complete, runnable
8Further Reading — sourced URLs
9Notes — version caveats
Filenames: kebab-case, no date suffix (date lives in frontmatter).
Examples: webgl-ray-marching.md · fastapi-middleware-auth.md · rust-lifetimes.md
File Organization Rules
Vault routing by content type
Language-specific → ~/ClaudeVault/Languages/
Framework / library → ~/ClaudeVault/Frameworks/
Tool / CLI / package → ~/ClaudeVault/Tools/
General research → ~/ClaudeVault/Research/
Design patterns → ~/ClaudeVault/Patterns/
Error patterns / fixes → ~/ClaudeVault/Debugging/
Three invariants
✓Search before create — update existing notes rather than creating duplicates
✓Also save to docs/research/ in the current project if that directory exists
✓Subfolder rule — when 3+ notes share a common subject prefix, move them into a named subfolder (one level only)
✓ After every vault write: uv run update_index.py (keeps CLAUDE.md fresh)
Self-Validation Checklist
Agents drift on output quality without explicit quality gates. The checklist forces the agent to verify completeness before returning the summary report — it cannot mark a task done while items are unchecked.
✓Sources are authoritative and current
✓Cross-referenced from multiple sources
✓Code examples complete and formatted
✓Markdown syntax correct
✓Style guide requirements met
✓Saved to correct vault folder with frontmatter
✓Vault index rebuilt via update_index.py
✓Source attribution complete
✓Documentation comprehensive yet concise
✓Technical accuracy validated
✓Practical examples included
✓(If NotebookLM) notebook ID in summary
Output Requirements + Summary Report
Three required outputs
1Vault notes — appropriate subfolder, YAML frontmatter, wikilinks. Also docs/research/ if it exists.
2Index rebuild — uv run update_index.py after all notes are saved.
3Summary report — topics researched, files created/updated, key findings, gaps, recommendations for further research.
NotebookLM output (when available)
# After web research — add all discovered URLs to notebook
notebooklm source add "https://source-1"
notebooklm source add "https://source-2"
# Summary report includes notebook ID + source count
# User can return to generate podcasts, quizzes, flashcards...
The summary report surfaces gaps explicitly — "X could not be found" is
more useful than silently omitting it.
Sync to ~/.claude/ and Verify
claude
→
uv run install.py --force --yes
Parsidion Installer
+ Install skill: parsidion → ~/.claude/skills/parsidion
+ Patch vault_common.py: VAULT_ROOT=/Users/probello/ClaudeVault
+ Install agent: research-agent.md → ~/.claude/agents/
+ Install scripts: scripts → ~/.claude/scripts/
✓ All hooks already registered
+ Install CLAUDE-VAULT.md → ~/.claude/CLAUDE-VAULT.md
+ Rebuild vault index (update_index.py)
✓ Vault index rebuilt
✓ Installation complete!
$ ls ~/.claude/agents/
research-agent.md ✓
$ grep -c "research-agent" ~/.claude/CLAUDE-VAULT.md
2 ✓ (debugging + implementation save paths)
First Research Task: WebGL Ray Marching
user
$
Let's start building a fractal renderer with ray marching
Trigger matched — proactive launch. Dispatching research-agent for WebGL ray marching techniques and fractal rendering algorithms.
Agent execution trace
✓Vault check — no existing ray marching notes found
✓NotebookLM: created "Research: WebGL ray marching", deep search started (background)
✓Brave Search: 8 sources — iquilezles.org, MDN WebGL2, Shadertoy docs, Khronos specs
✓Saved: ~/ClaudeVault/Research/webgl-ray-marching.md (confidence: high)
✓Saved: ~/ClaudeVault/Research/sdf-primitives.md (confidence: high)
✓Index rebuilt — 2 new notes indexed
proactive trigger fired
2 vault notes
8 sources
What We Built
New files
- agents/research-agent.md
- skills/parsidion/scripts/html-to-md.py (PEP 723)
- ~/.claude/agents/research-agent.md
- ~/.claude/skills/parsidion/scripts/html-to-md.py
Key design decisions
- Sonnet model — synthesis over speed
- 4-scenario proactive trigger
- 24h staleness check on vault index
- Non-blocking NotebookLM pattern
Standards defined
- 6-field YAML frontmatter (no orphan notes)
- 9-section documentation template
- 6-folder vault routing by content type
- 3-invariant file organization
- 12-item self-validation checklist
Impact
Research findings now persist across
every session — the vault becomes
richer with each research task.
⚠ SubagentStop hook exclusion: this agent is listed in
excluded_agents — its transcripts are not harvested
by the SubagentStop hook, preventing recursive vault-write loops.
311 lines
vault-first
Sonnet model
agentchrome + html-to-md
live in ~/.claude/