画板 1

    AI Agent for a Local Markdown Knowledge Base: A Practical Implementation Guide

    5 hr ago0 views

    This guide shows how to use a command-line AI agent to maintain a local Markdown knowledge base, with the files themselves as the single source of truth. It covers the vault directory layout, YAML front matter conventions, a rules file that bounds what the agent can read and write, a lightweight index plus a context budget for retrieve-then-read workflows, an automated organizing pipeline with a processing manifest, a health-check script, risk controls, troubleshooting notes, and a go-live checklist. You also get reusable Python and shell snippets, including a PyYAML-based indexer and a checker that validates front matter fields, wikilinks, orphan notes, and stale notes.

    1. Why Let an AI Agent Maintain Your Knowledge Base

    If you already have hundreds of local Markdown notes and want to cut the manual cost of keeping them organized, this approach is for you. The prerequisite is simple: the notes are stored as plain-text Markdown in a directory tree.

    The more notes accumulate, the harder they are to maintain by hand:

    • Scattered entry points. Web clippings, meeting notes, and reading excerpts each live in a different place, with no unified structure.
    • Written but never used. Keyword search only matches literally. You note “retrieval-augmented generation” but search for “RAG in practice” and usually find nothing.
    • Organizing costs more than writing. Adding tags, creating links, and archiving on a schedule are all repetitive labor.

    A command-line AI agent (referred to below as “the agent”) can read, search, and rewrite files on the filesystem, so it is well suited to that repetitive work. It can distill scattered inbox content into structured notes, create backlinks, update topic indexes, and check for broken links. People handle the rest: asking good questions, setting the rules, and reviewing the results. Only when the agent is part of a defined workflow does it change from a hazard into an assistant.

    2. Overall Design: Files as the Single Source of Truth

    The design comes down to “one source of truth, four layers”:

    • Single source of truth. All content lives in local Markdown files. Nothing depends on a plugin’s private database.
    • Structure layer. Conventions for directories, naming, and front matter metadata.
    • Rules layer. A rules file in the project root that defines the boundaries of the agent’s responsibilities.
    • Tooling layer. Scripts handle indexing, retrieval, and validation. The agent calls deterministic tools instead of relying on memory.

    2.1 Plan the Directory Structure

    The directory tree is organized by lifecycle stage, not by subject area:

    1. kb-vault/
    2. ├── _meta/            # Rule files, templates, scripts, and the index (excluded from content scans)
    3. ├── 00-inbox/         # Inbox: all new content lands here first
    4. ├── 10-notes/         # Permanent notes: one concept or question per note
    5. ├── 20-maps/          # Content maps: topic indexes and navigation pages
    6. ├── 30-daily/         # Daily and weekly notes
    7. ├── 90-assets/        # Attachments
    8. └── 99-archive/       # Archive

    All new content goes to 00-inbox first. The organizing pipeline then distills it into permanent notes.

    2.2 Consistent Naming and Metadata

    Every note starts with YAML front matter, the main basis for machine processing:

    1. ---
    2. title: "Retrieval-Augmented Generation in a Personal Knowledge Base"
    3. aliases: ["RAG", "retrieval-augmented generation"]
    4. tags: [ai/rag, note/concept]
    5. type: concept          # concept / howto / source / daily
    6. status: seedling       # seedling / growing / evergreen
    7. created: 2025-03-11
    8. updated: 2025-03-12
    9. related: ["[[Vector Retrieval Basics]]", "[[Personal Knowledge Base Structure]]"]
    10. ---

    Note that aliases, tags, and related are YAML lists. The indexing script must parse them as lists. Otherwise they degrade into plain strings, which breaks filtering and alias matching.

    3. Constraining the Agent with a Rules File

    Codify long-term rules in an AGENT_RULES.md file at the project root, loaded automatically on every start. The key requirement: the directory constraints must be identical in the rules file, the scripts, and the prompts.

    1. # Knowledge Base Agent Rules
    3. ## Role
    4. You are the organizing assistant for this knowledge base. You only work with Markdown files inside this directory.
    6. ## Allowed operations
    7. - Read: all files under 00-inbox, 10-notes, 20-maps, and 30-daily
    8. - Write (new files only): new notes under 10-notes/, plus index pages under 20-maps/
    9. - Write (modify): you may modify front matter fields (including related) of draft notes under 10-notes/
    10.   whose status is seedling, in order to record the links established during organizing
    11. - Generated content must cite specific source file paths. Never answer from memory.
    13. ## Forbidden operations
    14. - Deleting or renaming any existing file; moving files between directories
    15. - Modifying mature notes whose status is growing or evergreen
    16. - Modifying notes the user wrote personally or marked "Do not edit"
    17. - Changing image links, attachment paths, or embedded blocks
    18. - Creating or writing files under 90-assets, 99-archive, _meta, or 30-daily
    20. ## Output rules
    21. - New notes must be generated from _meta/templates/note.md, with front matter filled in
    22. - Mark unverified information as "to verify". Never fabricate it.
    23. - Use [[wikilinks]] for all links. Every target must actually exist.

    This resolves a common conflict. The rules originally allowed only new notes, yet they also required updating the related field. Changing this to “may modify the front matter of draft notes” makes write permissions consistent with the linking workflow. Updating related on mature notes is handled by scripts or by people instead.

    Enforce the constraints as executable checks (use an allowlist and deny by default):

    1. # Only .md files under 10-notes and 20-maps may be written; block everything else
    2. deny_write() {
    3.   case "$1" in
    4.     10-notes/*.md|20-maps/*.md) return 0 ;;
    5.     *) return 1 ;;
    6.   esac
    7. }
    8. # deny_write "99-archive/old.md" && echo allow || echo block   # output: block

    4. Search First, Read Later: Controlling the Context

    Handing the entire knowledge base to the model at once is neither realistic nor necessary. The approach has two steps. Maintain a lightweight index that holds only paths, titles, aliases, tags, and summaries. The agent first picks candidate notes from the index, then reads the full text of only a few of them.

    4.1 Build a Lightweight Index (Parse List Fields Correctly)

    1. import json
    2. import re
    3. import yaml
    4. from pathlib import Path
    6. VAULT = Path("kb-vault")
    7. FM_PATTERN = re.compile(r"\A---\s*\n(.*?)\n---\s*\n", re.S)
    10. def parse_front_matter(text):
    11.     """Parse front matter with PyYAML, restoring list fields such as aliases correctly"""
    12.     m = FM_PATTERN.match(text)
    13.     if not m:
    14.         return {}
    15.     try:
    16.         data = yaml.safe_load(m.group(1))
    17.     except yaml.YAMLError:
    18.         return {}          # fall back safely when parsing fails
    19.     return data if isinstance(data, dict) else {}
    22. def as_text(value):
    23.     """Normalize a string or a list into text that is easy to search"""
    24.     if isinstance(value, list):
    25.         return " ".join(str(v) for v in value)
    26.     return str(value or "")
    29. def build_index():
    30.     index = []
    31.     for path in (VAULT / "10-notes").rglob("*.md"):
    32.         text = path.read_text(encoding="utf-8")
    33.         meta = parse_front_matter(text)
    34.         body = FM_PATTERN.sub("", text, count=1)
    35.         index.append({
    36.             "path": str(path.relative_to(VAULT)),
    37.             "title": meta.get("title", path.stem),
    38.             "aliases": as_text(meta.get("aliases")),   # list -> text
    39.             "tags": as_text(meta.get("tags")),
    40.             "summary": re.sub(r"\s+", " ", body)[:120],
    41.         })
    42.     out = VAULT / "_meta" / "index.json"
    43.     out.write_text(json.dumps(index, ensure_ascii=False, indent=2),
    44.                    encoding="utf-8")
    45.     print(f"Indexed {len(index)} notes -> {out}")
    48. if __name__ == "__main__":
    49.     build_index()

    The dependency is PyYAML (pip install pyyaml).

    When you ask a question, have the agent scan the index and list candidates first. It should read the full text only after you confirm:

    1. Read _meta/index.json and find the 3 notes most relevant to
    2. "adding automated tests to a legacy project".
    3. Output only the paths and the reason for each match, then wait for my
    4. confirmation before reading the full notes.

    4.2 Allocate a Context Budget

    Content Suggested share How to handle it
    Rules file ~5% Always injected
    Index data ~20% Paths, titles, and summaries only; split into batches when large
    Full text of candidate notes ~35% Selected from the index by the model and read on demand
    Material to be written ~20% New content from the inbox
    Output reserve ~20% Avoids getting cut off halfway through the output

    These percentages are rules of thumb. Measure and adjust them for the model’s context window, the size of the index, and the type of task. Do not treat them as fixed parameters.

    5. The Automated Organizing Pipeline

    Organizing happens in four steps:

    1. Distill – read 00-inbox and generate drafts.
    2. Link – add wikilinks and update related on drafts.
    3. Index – update the 20-maps navigation pages.
    4. Validate – a script checks the result, then a person reviews it.

    Use a processing manifest to avoid re-organizing (no file moves). The original idea of moving files after processing conflicted with the rule against moving existing files. Instead, maintain a manifest that records the path and content hash of each processed file. The pipeline processes only files that are not on the manifest:

    1. # _meta/processed.py: read the inbox manifest and return new files that have not been processed yet
    2. import hashlib
    3. import json
    4. from pathlib import Path
    6. def file_hash(path: Path) -> str:
    7.     return hashlib.sha256(path.read_bytes()).hexdigest()
    9. def pending_files(vault: Path):
    10.     manifest = vault / "_meta" / "processed.json"
    11.     done = json.loads(manifest.read_text()) if manifest.exists() else {}
    12.     inbox = vault / "00-inbox"
    13.     result = []
    14.     for path in inbox.rglob("*.md"):
    15.         h = file_hash(path)
    16.         if done.get(str(path)) != h:      # new files or changed content count as pending
    17.             result.append(path)
    18.     return result

    Organizing script:

    1. #!/usr/bin/env bash
    2. # kb_organize.sh: daily scheduled organizing run
    3. set -euo pipefail
    4. VAULT="${1:-$PWD/kb-vault}"
    5. LOG_DIR="$VAULT/_meta/logs"; mkdir -p "$LOG_DIR"
    7. python3 "$VAULT/_meta/build_index.py"                       # 1. refresh the index
    8. run_agent "Read pending files in 00-inbox, distill them into drafts per the rules, and update the topic index"   # 2. organize
    9. python3 "$VAULT/_meta/check_kb.py" --vault "$VAULT" \
    10.          --report "$LOG_DIR/latest.md"                      # 3. validate; abort on failure

    For non-interactive organizing tasks, use the batch mode of the command-line tool. Write changes to a working copy first, and merge them only after validation passes.

    6. Quality Checks: Keep the Knowledge Base Healthy

    The validation script’s scan scope must be limited to the content directories (00-inbox, 10-notes, 20-maps, 30-daily), excluding _meta, 99-archive, and 90-assets. Otherwise report files such as _meta/logs/latest.md get mistakenly checked as if they were notes:

    1. import argparse
    2. import re
    3. from collections import defaultdict
    4. from datetime import date
    5. from pathlib import Path
    6. import yaml
    8. WIKILINK = re.compile(r"\[\[([^\]|#]+)(?:[|#][^\]]*)?\]\]")
    9. REQUIRED_FIELDS = ("title", "tags", "type", "created", "updated")
    10. FM_PATTERN = re.compile(r"\A---\s*\n(.*?)\n---\s*\n", re.S)
    11. SCAN_DIRS = ("00-inbox", "10-notes", "20-maps", "30-daily")   # content directories only
    14. def parse_front_matter(text):
    15.     m = FM_PATTERN.match(text)
    16.     if not m:
    17.         return {}
    18.     try:
    19.         data = yaml.safe_load(m.group(1))
    20.     except yaml.YAMLError:
    21.         return {}
    22.     return data if isinstance(data, dict) else {}
    25. def iter_notes(vault: Path):
    26.     for d in SCAN_DIRS:
    27.         base = vault / d
    28.         if not base.exists():
    29.             continue
    30.         for path in base.rglob("*.md"):
    31.             yield path
    34. def check(vault: Path, report: Path):
    35.     notes = {}
    36.     for path in iter_notes(vault):
    37.         text = path.read_text(encoding="utf-8")
    38.         notes[path.stem] = (path, text, parse_front_matter(text))
    40.     problems = defaultdict(list)
    42.     for stem, (path, _, meta) in notes.items():              # 1. required fields
    43.         for field in REQUIRED_FIELDS:
    44.             value = meta.get(field)
    45.             if not value:
    46.                 problems["Missing metadata fields"].append(
    47.                     f"{path.relative_to(vault)} is missing {field}")
    49.     for stem, (path, text, _) in notes.items():              # 2. broken links
    50.         for target in WIKILINK.findall(text):
    51.             if target.strip() not in notes:
    52.                 problems["Broken links"].append(
    53.                     f"{path.relative_to(vault)} references nonexistent [[{target.strip()}]]")
    55.     referenced = set()                                       # 3. orphan notes
    56.     for _, (_, text, _) in notes.items():
    57.         referenced.update(t.strip() for t in WIKILINK.findall(text))
    58.     for stem, (path, _, _) in notes.items():
    59.         if stem not in referenced:
    60.             problems["Orphan notes"].append(str(path.relative_to(vault)))
    62.     today = date.today()                                     # 4. not updated for a long time
    63.     for stem, (path, _, meta) in notes.items():
    64.         updated = str(meta.get("updated", ""))
    65.         if re.fullmatch(r"\d{4}-\d{2}-\d{2}", updated):
    66.             y, mo, d = map(int, updated.split("-"))
    67.             if (today - date(y, mo, d)).days > 365:
    68.                 problems["Not updated for a long time"].append(
    69.                     f"{path.relative_to(vault)} has not been updated for over a year")
    71.     lines = [f"# Knowledge Base Health Report ({today.isoformat()})", ""]
    72.     if not problems:
    73.         lines.append("No issues found.")
    74.     for category, items in problems.items():
    75.         lines.append(f"## {category} ({len(items)} items)")
    76.         lines.extend(f"- {item}" for item in items)
    77.         lines.append("")
    78.     report.write_text("\n".join(lines), encoding="utf-8")
    79.     print("\n".join(lines))
    80.     return sum(len(v) for v in problems.values())
    83. if __name__ == "__main__":
    84.     parser = argparse.ArgumentParser()
    85.     parser.add_argument("--vault", type=Path, default=Path("kb-vault"))
    86.     parser.add_argument("--report", type=Path, default=Path("report.md"))
    87.     args = parser.parse_args()
    88.     raise SystemExit(1 if check(args.vault, args.report) else 0)

    7. Risk Control: Make the Agent Work in a Controlled Way

    1. Least privilege. Grant access per directory. Scripts, secrets, and the attachment directory are not writable by default.
    2. Confirm dangerous operations. For bulk renames, replacements, and cross-directory moves, show a preview before executing.
    3. Leave a trail. Put the knowledge base under version control and tag it regularly, so any automated change can be rolled back.
    4. Treat external content as untrusted input. Clippings and emails may contain text like “ignore previous instructions.” Pin down the agent’s goal, and label the source of external content.
    5. Protect privacy and secrets. For sensitive information, prefer a local model. When you do need a hosted LLM service, choose a platform that offers data redaction, access auditing, and compliance features. Submit only what is necessary, following the principle of minimal data, and redact personal information before submission.

    You can start by running the agent as a “read-only advisor” that outputs only suggestions and diffs. Once it proves stable, open write access gradually.

    8. Common Problems and How to Handle Them

    Common problem Main cause How to handle it
    References to nonexistent notes The agent invents wikilinks from memory Run validation after every write; rules require references to come from the index
    Hand-written content gets rewritten Write scope too broad Allow only new files in protected directories; mature notes cannot be modified
    Tags become a mess No controlled tag vocabulary Maintain a controlled tag list and merge synonyms regularly
    Task gets cut off partway Too much injected at once Index first, read on demand, process in batches, and reserve output room
    More duplicate notes Missed during retrieval Run a similarity check for duplicates before creating a note, and merge similar content
    Validation script gives false positives or negatives Limitations of wikilink matching See below

    Why does the validation script produce false positives and negatives? Wikilinks are matched by file name (stem). If two directories contain files with the same name, the links get crossed. Links that include a path or the .md extension, cross-directory relative paths, and alias links may also be missed.

    The fixes:

    • Keep every note’s file name globally unique.
    • Always write links as [[file-name]].
    • Alternatively, extend the validation script with path resolution and alias mapping logic.

    9. From Individual to Team

    Teams can use a shared Markdown repository as the knowledge base. A code hosting platform’s branches and merge requests can then be used for content review.

    If you build on a mainstream cloud platform, you can layer general-purpose capabilities on top of the local architecture:

    • Object storage for snapshots and attachments, for off-site backup.
    • General-purpose embedding and text-generation services from an LLM service platform, to build semantic search and a question-answering entry point.
    • A general-purpose content moderation capability in the content-sharing step, to reduce compliance risk.

    All of these capabilities layer on top of “files as the source of truth” and do not change the structure of the knowledge base itself.

    10. Go-Live Checklist

    • Clear directory structure, with the inbox separated from permanent notes
    • Consistent front matter fields, with a reusable template
    • Rules file specifies the role, allowed/forbidden operations, and output rules, and matches the script allowlist
    • The indexing script parses list fields correctly with PyYAML and can rebuild the index with one command
    • The organizing pipeline runs on a schedule and uses a processing manifest to avoid duplicates
    • The validation script scans only content directories and covers four problem types: metadata, links, orphans, and not updated for a long time
    • Version backup and restore have been verified as working
    • If a cloud model is involved, data security, redaction, and compliance assessments are complete

    Once each item is in place, the agent changes from a chat tool that helps occasionally into a stable organizing assistant for the knowledge base. The knowledge base can then truly grow into a searchable, reusable, continuously evolving “second brain.”

    09040904

    Top articles