Daniel Benner

A Causal Graph as AI Memory

Sun, 05 Apr 2026 00:00:00 GMT

In a previous post, I described using a context.md file as the AI’s working memory — a flat list of open threads split into “In Flight,” “Blocked,” and “Needs Attention.” Each entry was a one-liner: what it is plus current status.

It worked for about two months. Then it stopped working.

Where Flat Context Breaks

The problem isn’t that a markdown list can’t hold enough items. It’s that it can’t express how things are connected.

Say you notice code quality problems across the team. That observation spawns several threads: a CI pipeline overhaul, a hiring push for a backend engineer, structured 1:1s, and a decision to restrict an AI auto-fix tool that was generating bad PRs. The CI pipeline enables trunk-based development. The hire leads to onboarding. The 1:1s surface a performance issue with one developer, and give another the space to build a bug triage skill. The AI auto-fix restriction evolves into a proper agent-ticket integration.

In context.md, these are separate bullet points:

## In Flight
- CI pipeline — live, main is default branch
- Backend engineer — hired, onboarding in progress
- 1:1s — running across team
- Dev A — performance improvement plan
- Bug triage skill — config-vs-code classifier
- Project management migration — agent integration next
...

Every session, the AI reads this list and knows what’s happening. But it doesn’t know why. It can’t see that the CI pipeline exists because of quality problems, that the bug triage skill emerged from the 1:1s with one developer, or that the project management migration is what unblocks the agent-ticket integration. The backstory gets re-explained in conversations. The file doesn’t carry it.

The second problem is decay. Flat lists don’t have a natural cleanup lifecycle. Resolved items sit there until someone removes them. “Someone” in practice means me, and I have better things to do than gardening a markdown file. So stale items linger, and the AI treats them as active.

The Graph

The replacement is a directed graph in DOT format. One file — compass/context/index.dot — maintained entirely by the AI. I never edit it.

Nodes are inflection points. Not routine progress, not daily updates — moments where something actually changed: a decision was made, something shipped, a thread got blocked, or a new workstream spawned from an existing one.

n_20260115_quality_problems [
    label="Code quality problems observed across team",
    timestamp="2026-01-15",
    status="resolved",
    detail="Sloppy code getting through, bugs in production.
            Triggered multiple improvement tracks."
]
 
n_20260129_ci_decision [
    label="Decided: trunk-based dev for new product,
            keep old branching model for legacy",
    timestamp="2026-01-29",
    status="resolved",
    detail="New product needs fast iteration.
            Legacy needs stability."
]
 
n_20260205_ci_live [
    label="CI pipeline live — main is default branch",
    timestamp="2026-02-05",
    status="resolved",
    detail="Develop branch deleted.
            Features must work when merged."
]

Edges encode causation:

n_20260115_quality_problems -> n_20260129_ci_decision [type="led_to"]
n_20260129_ci_decision -> n_20260205_ci_live [type="led_to"]
n_20260205_ci_live -> n_20260223_stacked_prs [type="enabled"]

Five edge types cover most relationships: led_to, spawned, enabled, blocked_by, invalidated. Subgraphs cluster related nodes into workstreams. Cross-thread edges connect workstreams when they interact — a hire in one thread enables a capability in another.

The convention that makes it work: leaves are the current state. A node with no outgoing edges to other active nodes is where things stand right now. Trace backwards from any leaf, and you get the causal chain of how it got there.

What Gets a Node

Not everything. This was the hardest convention to get right.

A node is an inflection point — a moment that changed the trajectory of a workstream. Specifically:

A decision was made
Something shipped or went live
A thread got blocked or unblocked
A new thread was spawned from an existing one
New information changed the picture

“Made progress on feature X” is not a node. “Feature X shipped” is. “Had a good conversation with Y” is not a node. “Y accepted the role change” is. The graph captures the shape of what happened, not the texture.

The AI makes this judgment call on its own, and it’s right about 90% of the time. The other 10% is usually over-persisting — creating a node for something that was really just routine progress. That’s the easier error to correct.

Pruning and Archiving

This is where the flat-list decay problem gets solved.

When all leaves of a subgraph are resolved or abandoned, and the subgraph is older than roughly three weeks, it gets collapsed: the full graph moves to an archive file, and a single summary node stays in the index.

n_archive_agent_pipeline [
    label="Agent pipeline — proven and working in production",
    timestamp="2026-02-08 to 2026-03-19",
    status="resolved",
    detail="Research, state machine, QA agent.
            Pipeline working hands-off.
            See compass/context/archive/agent-pipeline.dot"
]

The archive is still there if the AI needs to trace deep history. But the working graph stays focused on what’s active. Currently I have eight archived subgraphs compressed into summary nodes, and the index stays under 300 lines.

The cleanup is natural — it follows from the status conventions rather than requiring manual gardening.

Why DOT

People ask why not JSON or YAML. A few reasons.

DOT is scannable. Open the file and the structure is immediately visible — nodes, edges, subgraphs, labels. JSON would be a wall of nested objects. YAML would be fragile about indentation.

Claude reasons well about DOT. It can parse the graph structure, follow edges, identify leaves, and produce valid updates without a schema definition or parsing library. The format is simple enough that an LLM handles it natively.

It renders. If you ever want to visualize the graph, dot -Tpng index.dot -o graph.png works. I rarely do this, but it’s useful for debugging when subgraphs get tangled.

And it has just enough structure without being rigid. Nodes have attributes (label, timestamp, status, detail) but the format doesn’t enforce a schema. If I need a new attribute, I add it. No migrations.

What It Enables

The graph changes what the AI can do at session start. Instead of reading a flat list and knowing what’s happening, it reads a causal narrative and knows the story.

This matters most for advice and planning. When I ask about a workstream, the AI doesn’t just report status — it can explain the causal chain, point to decisions that constrained the current state, and notice when threads that should be connected aren’t.

It also makes session handoffs sharper. The previous post described context.md as “briefing notes for an incoming shift.” The graph is more like handing over a case file — not just the current situation, but the chain of events that produced it.

Two specific things it catches that flat files miss:

Stale threads. A node sitting at active status with no recent children is visibly stale — there’s been no inflection point in weeks. The graph makes this structural rather than requiring someone to notice a dusty bullet point.

Cross-thread dependencies. When one workstream blocks or enables another, that edge exists in the graph. The AI can surface it: “the project management migration you’ve been putting off is what unblocks the bug triage workflow.” In a flat list, those are just two unrelated bullet points.

Tradeoffs

It’s an AI-only file. I don’t read or edit the graph. This is by design — the AI maintains it as part of session wrap-up, and I review the effects (better session starts, better advice) rather than the artifact. But it means the memory system is opaque to me. If the AI misrepresents something, I might not notice until it gives bad advice.

Inflection point judgment is imperfect. The AI tends to over-persist rather than under-persist. A weekly conversation that was really just a status check sometimes gets a node. The fix is easy — delete or merge the node — but it requires noticing.

DOT gets big. Without the archiving lifecycle, the graph would grow indefinitely. Even with archiving, a busy quarter can produce a 400-line index. The subgraph structure keeps it navigable, but there’s a complexity ceiling.

It requires the session-end hook. The graph doesn’t maintain itself — it’s updated by a background agent that runs after each session. If that hook fails or isn’t configured, the graph goes stale. The system has a dependency on infrastructure that a markdown file doesn’t.

The Evolution

The progression was: no context → flat markdown list → causal graph. Each step solved a real problem with the previous approach.

The flat list solved cold starts — AI stopped asking “what are you working on?” every session. The graph solved narrative loss — AI stopped asking “why are you doing it this way?” and “what happened with X?”

Whether you need the graph depends on how many concurrent threads you’re managing and how long they live. If you’re tracking three things over two weeks, context.md is fine. If you’re tracking twenty things over three months with causal dependencies between them, the flat file will eventually fail you the same way it failed me.

The building blocks haven’t changed from the original system: plain text, simple conventions, full transparency. The graph is just a better data structure for the job.

📚 The Art of Action — Stephen Bungay

Sun, 29 Mar 2026 00:00:00 GMT

The Three Gaps

Execution breaks down in three distinct ways:

Knowledge gap — between plans and outcomes. What we know vs what we’d need to know.
Alignment gap — between plans and actions. What we intend vs what people do.
Effects gap — between actions and outcomes. What we do vs what results we get.

Organizations instinctively reach for the wrong lever: more information for the knowledge gap, more detailed instructions for alignment, tighter controls for effects. Each response tries to eliminate uncertainty rather than navigate it — and makes things worse.

The appropriate responses: accept uncertainty and build adaptive capacity (knowledge), clarify intent not instructions (alignment), tighter feedback loops not more reporting (effects).

Detail Is Not Clarity

More detail in instructions doesn’t make them clearer. It makes them more brittle. Every detail bakes in an assumption about the environment. When conditions change — and they will — some details become irrelevant, some impossible, some actively counterproductive.

The alternative: communicate intent. What are we trying to achieve? Why does it matter? What constraints must be respected? Then let people closest to the work figure out how. They have information the planner doesn’t, and they can adapt as conditions change.

Simplicity Precedes Clarity

To make something clear, you first have to make it simple. A complex message delivered with perfect articulation is still complex. People won’t misunderstand because you were unclear — they’ll misunderstand because there was too much to parse.

This reframes the leader’s job: it’s not a communication problem, it’s a thinking problem. The hard work is deciding what to leave out. The clarity follows.

Briefing and Backbriefing

The protocol that makes intent-based leadership actually work. A brief passes down three things: the context and what the level above is trying to achieve, your own intent (the what and why, not how), and the boundaries that must be respected.

The key move: share your superior’s intent and your own. People understand not just what you want but what the layer above wants — so they can make intelligent decisions when your specific intent becomes irrelevant.

Backbriefing closes the loop. The receiver explains back what they understood, how they plan to achieve it, and what risks they see. Misunderstandings surface before execution. If their plan differs from what you imagined but achieves the intent — you shut up and let them do it. You don’t correct the how if the what and why are right.

📚 The Choice — Eliyahu Goldratt

Tue, 10 Mar 2026 00:00:00 GMT

Inherent Simplicity

Goldratt’s foundational claim: complex systems are governed by very few root causes. The more interconnected the parts, the fewer degrees of freedom — everything constrains everything else, which paradoxically reduces the system to a small number of governing variables.

This isn’t optimism. It’s a structural argument about interdependent systems. The simplicity is hidden — you have to work through the complexity to find it. But it’s there.

Implications: if your fix requires orchestrating dozens of changes, you probably haven’t found the real cause. Complex solutions signal incomplete understanding.

Humble and Arrogant

Two stances, held simultaneously:

Humble toward reality — your current model is incomplete. Don’t defend your assumptions.
Arrogant toward your capability — reality is knowable. The complexity can be understood.

Most people invert this: arrogant about their understanding (defending their current view) and humble about their capability (believing the problem is too complex to solve).

This also reframes luck. Opportunities exist everywhere, but you only spot them when you perceive reality clearly rather than through the fog of assumptions. Luck = preparedness + opportunity.

Compromise as a Thinking Trap

Accepting that conflicts require compromise is itself a barrier to clear thinking. When you settle for a compromise, you stop looking for the false assumption that created the apparent conflict in the first place.

It feels mature — pragmatic, reasonable. But it’s intellectual surrender. You’re accepting a constraint without verifying it exists.

The alternative isn’t naive win-win optimism. It’s disciplined questioning: what assumption would have to be false for both sides to get what they need?

Every compromise narrows the solution space for future decisions. You optimize within boundaries that were never real.

From Complexity to Blame

The belief that reality is inherently complex doesn’t just produce worse solutions — it poisons relationships. The chain:

Assume complexity — no clean answer exists
Accept zero-sum framing — your gain is my loss
Settle for compromise — we each give up something
Blame when it fails — I sacrificed and it didn’t work, because of you

Each step feels reasonable in isolation. Together, they create a cycle of resentment. In a finite-cake world, compromise means loss — and when the outcome is bad, someone must be responsible for your loss.

Flip the first assumption — believe in inherent simplicity — and the whole chain unravels. You stop treating others as adversaries and start looking for root causes together.

The Complexity Assumption Trap

Assuming a problem is irreducibly complex becomes self-fulfilling. You stop looking too early, never find the root cause, and the persistent mess confirms your belief that it was complex all along.

What gets lost: root causes stay hidden, leverage points go unused, effort scatters across non-constraints, and repeated failure erodes confidence — reinforcing the belief the problem is unsolvable.

The fix is methodological: assume the few root causes exist and keep looking. This doesn’t guarantee you’ll find them — but assuming complexity guarantees you won’t.

📚 Turn the Ship Around! — L. David Marquet

Sun, 15 Feb 2026 00:00:00 GMT

Information Locality and Decision Rights

Decisions require information. You have two options:

Centralize information — move data up to where decisions are made
Decentralize decisions — move authority down to where information exists

Centralizing information is slow, lossy, and expensive. By the time context reaches the top, it’s stale. Nuance gets compressed. You build bureaucracy just to shuttle information around.

Pushing decisions to the edge is more efficient — no transmission delay, no compression loss. But it only works when people are competent enough to decide well, and aligned enough that distributed decisions don’t fragment.

Three Pillars of Decentralized Decisions

Decentralizing authority without supporting conditions creates chaos. Three pillars must hold together:

Control — actual authority to decide
Competence — capability to decide well
Clarity — shared intent so decisions align

Missing any one breaks the system. Control without competence produces mistakes. Control without clarity produces fragmentation. Competence and clarity without control is wasted potential — people who could decide but aren’t allowed to.

Inverting the Direction of Initiative

Traditional leadership pushes down: leader gives orders, subordinate executes. The alternative flips the active party.

Instead of leaders extracting information upward and pushing decisions down, subordinates surface decisions in ready-to-validate form: “I intend to…” The leader’s role shifts from directing to enabling — intervening only when needed.

This is more efficient because the person with the information packages it themselves. No lossy compression, no waiting for the next status meeting.

Skills Over Tools: Why CLI Wrappers Beat MCP Servers for AI Agents

Sat, 14 Feb 2026 00:00:00 GMT

The standard approach to giving AI agents capabilities is tool definitions — fixed function signatures with typed parameters. MCP servers formalize this further: a dedicated process that exposes a set of tools over a protocol.

Both work. Both are also unnecessarily rigid. There’s a simpler pattern that’s more flexible and requires less infrastructure: a skill file that teaches the agent how to use an existing CLI tool.

The Three Approaches

Fixed tools are function signatures baked into the agent’s configuration. A browser tool might expose navigate(url), click(selector), extract_text(selector). Each action is a separate tool call with a defined schema.

MCP servers package these tools into a standalone process. The agent discovers available tools via the protocol, calls them with structured JSON, gets structured JSON back. It’s tools-as-a-service.

Skills are instruction files — markdown documents that describe how to use a CLI tool via the agent’s existing shell access. No protocol, no server, no schema. Just documentation the agent reads and follows.

What a Skill Looks Like

I built bb, a browser automation CLI. Instead of wrapping it in an MCP server, I wrote a skill file — a markdown document that explains the commands:

# Browse
 
Browse the web using `bb`, a browser automation CLI
that drives a persistent headless Chrome instance.
 
## Quick reference
 
bb open https://example.com      # navigate and read
bb text ".article-body"          # extract specific content
bb click "button.load-more"      # interact
bb wait ".results"               # wait for dynamic content
bb screenshot page.png           # capture
bb stop                          # clean up

The file covers the most common commands and patterns — not all of them. It doesn’t have to. If the agent needs a command that isn’t in the skill file, it runs bb --help and figures it out. The skill provides the fast path; the CLI’s own documentation covers everything else.

Why This Works Better

No Capability Ceiling

Fixed tools define a finite set of operations. If the tool definition doesn’t include “wait for network idle then extract text from a specific selector,” the agent can’t do it. Someone has to anticipate every useful combination and define a tool for each.

With a CLI, the agent composes commands freely:

bb open https://example.com
bb wait ".results"
bb text ".results .item:first-child"

No one had to define a navigate_wait_and_extract_first_item tool. The agent read the available commands and composed them to fit the task.

Commands Chain Naturally

CLI tools compose with pipes and subshells. A single line can do what would take multiple round-trip tool calls:

bb text ".item" | grep "in stock" | wc -l

Count of in-stock items, one command. With fixed tools, this is three separate calls: extract text, filter results, count matches — each one a full round trip through the tool interface.

The agent already knows bash. Piping, redirection, grep, jq, awk — these aren’t features anyone needs to build. They’re already there, and they work with any CLI that outputs plain text.

No Server to Run

MCP servers are processes. They need to start, stay running, handle connections, manage state. That’s operational overhead for what amounts to “let the agent click buttons in a browser.”

A CLI is already running infrastructure. bb auto-starts Chrome on first use and persists state in ~/.bb/. The agent calls it like any other command. No daemon to manage, no protocol to speak.

The Agent Already Has a Shell

This is the part that gets overlooked. Agents that can execute bash commands already have the most flexible tool interface possible. A shell command can do anything a tool definition can do, plus everything else.

Adding an MCP server for browser automation means: install the server, configure the connection, handle the protocol overhead — all to give the agent capabilities it could have through bb open https://example.com.

Skill Files Are Self-Documenting

A tool definition is a schema. A skill file is documentation. The agent doesn’t just know what it can call — it knows when and how to use each command effectively.

From the bb skill:

## Tips
 
- Start with `bb open` — it navigates and extracts readable
  content in one step.
- For SPAs, use `bb open <url>` then `bb wait <selector>`
  before extracting.
- Use `bb ax-tree` when you need to understand page structure
  without reading raw HTML.

This kind of contextual guidance doesn’t fit in a tool schema. In an MCP server, it lives in description fields that are often too short to be genuinely useful. In a skill file, there’s no length limit. Write whatever the agent needs to know.

Updates Are Trivial

Adding a new command to bb means updating the skill file — a markdown edit. No schema changes, no server redeployment, no version negotiation. The agent reads the updated file next time it needs to browse.

When Tools and MCP Servers Make Sense

This isn’t universal. Fixed tools work well when:

The agent doesn’t have shell access. Web-based agents or sandboxed environments need structured tool interfaces.
You need strict input validation. Tool schemas enforce types and required fields at the protocol level.
Multiple agents share the same server. MCP’s discovery protocol helps when you have many consumers.

But for a coding agent that already runs in a terminal — which is most of them — skills over CLIs are simpler, more flexible, and require less maintenance.

The Pattern

Build the capability as a CLI with plain-text output.
Write a skill file that documents usage, patterns, and tips.
Let the agent read the skill and call the CLI through bash.

The agent gets full access to the tool’s capabilities without anyone having to predict which combinations of operations it might need. The CLI handles the complexity. The skill file provides the context. The shell connects them.

No servers. No schemas. No protocol overhead. Just commands and documentation.

From Audio to Action Items in Under a Minute

Tue, 10 Feb 2026 00:00:00 GMT

I stopped taking meeting notes. Not because I got lazy — because I automated the entire chain from raw audio to structured action items, decisions logged, and references updated.

The setup has two parts: a Go CLI tool called meetingcli that handles recording and transcription, and an AI agent skill that processes the transcript into my personal operating system. End to end, a one-hour meeting goes from audio to fully processed in about a minute after I hang up.

The Recording Problem

Meeting notes are lossy. You’re either paying attention or writing things down — rarely both. And even good notes miss nuance, exact phrasing, and the stuff you didn’t realize was important until later.

The alternative is recording, but most tools dump you with a giant audio file or a mediocre transcript locked inside some SaaS platform. I wanted something local, private, and plugged into my existing workflow.

meetingcli

meetingcli is a Go CLI that does three things: record, transcribe, and summarize. One command, no UI.

meeting start --name "product-sync"
# talk for an hour
# Ctrl+C to stop
# → transcription and summary happen automatically

Dual Audio Capture

The tricky part of meeting recording on macOS is capturing both sides of the conversation. meetingcli runs two audio streams in parallel:

System audio via Apple’s ScreenCaptureKit — this captures whatever’s coming out of your speakers or headphones. It taps directly into the OS audio mixer, so it works with Bluetooth headphones, external DACs, whatever. The implementation is Objective-C bridged into Go via cgo.
Microphone audio via ffmpeg from the default input device.

Both streams record to separate WAV files, then get merged into a single recording.wav using ffmpeg’s amix filter. The system audio capture does real-time downsampling from 48kHz stereo float32 to 16kHz mono int16 — keeps file sizes reasonable without losing speech clarity.

Each meeting lands in its own timestamped folder:

~/meetings/2026-02-06_14-00-00_product-sync/
├── recording.wav      # merged audio
├── system.wav         # system audio (raw)
├── mic.wav            # mic audio (raw)
├── transcript.md      # diarized transcript
└── summary.md         # AI-generated summary

Transcription

The merged audio gets sent to Mistral’s Voxtral API with speaker diarization enabled. The transcript comes back with speaker labels, so you get a readable conversation rather than a wall of text:

**Speaker 0:**
We need to decide on the auth model before shipping the API redesign.
 
**Speaker 1:**
I think JWT with refresh tokens is the way to go. OAuth adds complexity we don't need yet.

Voxtral handles multilingual audio well — useful when half your meetings are in German and the other half in English.

Summary

After transcription, the full text goes to Claude Haiku 4.5 for summarization. This produces a quick-reference summary with key topics, decisions, and action items. Useful for a glance, but the real value comes from what happens next.

Processing Transcripts Into the System

Recording and transcribing is table stakes. The interesting part is what happens to the transcript after.

I have an AI agent skill called process-meetings that reads transcripts and updates my entire operating system accordingly. When I run it, it spawns a parallel sub-agent for each meeting recorded that day. Each sub-agent reads the transcript and does six things:

1. Creates a Meeting Note

A concise summary goes into the relevant project folder — participants, key topics, decisions, action items with owners. Not a rehash of the transcript, but the stuff that matters a week from now.

2. Updates the Compass

The compass is my AI’s memory system. The agent updates decisions.md when significant decisions were made, updates the context graph when project status changed, and captures any preferences I expressed.

If I said “let’s drop the OAuth requirement and go with API keys” in a meeting, that decision gets logged with reasoning. Next time AI helps me with the auth system, it knows what was decided and why.

3. Updates References

Project briefs, people files, and other reference documents get updated when durable understanding changes. New team member mentioned? Their file gets created. Project scope shifted? The brief gets updated.

4. Flags Tasks

This one is deliberately conservative. The agent only creates tasks that are directly my responsibility — not things assigned to other team members, not things I just need to be aware of. It checks existing tasks first to avoid duplicates.

5. Updates Collections

If I mentioned a restaurant I liked or a hike worth doing, it gets added to my collections. Small thing, but it means casual mentions don’t get lost.

6. Zettelkasten Connections

Occasionally a meeting surfaces a genuinely interesting concept or pattern. The agent checks if it connects to existing notes in my knowledge base. Most meetings don’t produce anything here — that’s by design.

The Full Pipeline

Here’s what a typical day looks like:

Join a meeting. Run meeting start. Talk.
Hang up. Ctrl+C. Transcription and summary happen automatically.
At the end of the day (or whenever), run the process-meetings skill.
Sub-agents process each meeting in parallel. My compass, references, tasks, and notes all get updated.

Total manual effort: typing meeting start and pressing Ctrl+C.

Why This Works

Three design choices make the difference:

Local-first recording. The audio files live on my machine. No uploading hour-long recordings to some startup’s servers. The only thing that leaves my machine is the audio going to Mistral for transcription and the text going to Anthropic for summarization.

Separation of capture and processing. meetingcli does one job well: turn audio into text. The AI agent does the interpretation. This means I can improve either side independently. Better transcription model? Swap the API call. Better processing logic? Update the skill prompt.

Transparent output. Everything is markdown files in known locations. No database, no proprietary format. The meeting folders are just folders. The compass updates are just file edits. I can inspect, override, or correct anything.

Building meetingcli

The tool is open source and installable via Homebrew:

brew install devbydaniel/tap/meeting

The Go codebase follows hexagonal architecture — the recording, transcription, and summarization are separate use cases with clean interfaces. The macOS audio capture is the most interesting piece technically: an Objective-C implementation of ScreenCaptureKit bridged into Go, handling real-time audio format conversion in the capture callback.

If you’re curious about the implementation or want to adapt it for your own workflow, the source is on GitHub.

What’s Missing

Speaker identification. Voxtral gives you “Speaker 0” and “Speaker 1,” not names. I could map these manually or build a voice fingerprinting step, but honestly the context usually makes it obvious who said what.

The summary step inside meetingcli is redundant with the agent processing. I might remove it or make it optional — the agent produces better, more contextualized output anyway.

And the processing skill currently requires me to trigger it manually. It could watch the meetings directory and process automatically when a new transcript appears. Haven’t needed that yet — running it once at end of day works fine.

Building a Personal AI Agent with Raspberry Pi and Telegram

Sun, 08 Feb 2026 00:00:00 GMT

OpenClaw is everywhere right now. 145,000 GitHub stars, CNBC coverage, AI agents posting on their own social network. It manages your emails, browses the web, schedules your calendar, shops for you — “the AI that actually does things.”

It’s also a mass of dependencies you’ll never fully understand, running on your operating system with access to your private data. Palo Alto Networks called it a “lethal trifecta” of security risks. The creator himself says it’s not ready for non-technical users.

Here’s my counterproposal: build it yourself. Not because OpenClaw is bad — the ambition is right. But because personal AI infrastructure is personal. You want to understand every piece. And the architecture is simpler than you’d think.

The Setup

A Raspberry Pi. A Telegram bot. Git hooks. That’s the stack.

┌─────────────┐       ┌──────────────────┐
│  Telegram    │◄─────►│  Raspberry Pi     │
│  (you)       │       │                  │
└─────────────┘       │  ┌──────────────┐ │
                      │  │ Telegram Bot  │ │
                      │  │ ↕             │ │
                      │  │ AI Agent      │ │
                      │  │ ↕             │ │
                      │  │ Git Repo      │ │
                      │  │ (notes/tasks/ │ │
                      │  │  compass/)    │ │
                      │  └──────────────┘ │
                      └──────────────────┘

The Pi runs 24/7. The Telegram bot is your interface — from your phone, your laptop, anywhere. The git repo is the agent’s memory: your notes, context, tasks, and workflows, synced via git push/pull at session boundaries.

Why a Raspberry Pi

Three reasons.

It’s always on. Unlike your laptop, it doesn’t sleep. An agent that can check your calendar at 7am and send you a briefing needs a host that’s running at 7am.

It’s yours. Your data stays on hardware you own, on your network. No cloud provider reading your emails, no SaaS with a terms-of-service change waiting to happen.

It’s cheap. A Pi 5 costs €80. Running an LLM through an API costs a few dollars a month. Compare that to the compute overhead of OpenClaw’s full stack.

The Session Model

The interesting architectural decision is how the agent maintains state across conversations.

Session start:
  1. git pull (get latest notes, compass, context)
  2. Read compass/ files (rehydrate memory)
  3. Ready to work
 
Session end:
  1. Update compass/context with what happened
  2. Log decisions, preferences, follow-ups
  3. git push (persist everything)

This is the same model I described in From Markdown Notes to AI Chief of Staff, but with a key difference: the session start and end aren’t me opening and closing a terminal. They’re the Telegram bot receiving a message and going idle.

Git gives you version history for free. Every state change is a commit. You can diff what the agent changed, revert bad updates, see how context evolved over weeks. Try getting that from OpenClaw’s “persistent memory.”

The Telegram Bot

Telegram’s Bot API is one of the better-designed APIs out there. A minimal bot that forwards messages to an AI agent and returns responses is maybe 100 lines of Python. Add command handlers for specific workflows:

/morning — run the morning kickoff routine
/tasks — show today’s tasks
/email — triage inbox
/prep John — briefing before a meeting with John

Each command maps to a skill — a markdown file describing a multi-step workflow. The bot reads the skill, passes it to the LLM along with the relevant context files, and executes the steps.

The critical insight: the Telegram bot isn’t the agent. It’s a thin shell. The intelligence lives in the LLM, the context lives in git, and the workflows live in markdown files. The bot just connects them.

What You Get That OpenClaw Doesn’t Give You

Comprehension. Every component is something you built or configured. When something breaks — and it will — you know where to look. OpenClaw has hundreds of contributors and a codebase that’s growing faster than anyone can review. That’s fine for a web framework. It’s uncomfortable for something with access to your email.

Control. You decide exactly what the agent can access. Want it to read your calendar but not send emails? Don’t give it the email tool. With OpenClaw, you’re trusting that the permission model works as advertised across a plugin ecosystem built by strangers.

Simplicity. The entire system is a Telegram bot, a handful of CLI tools, and a git repo of markdown files. There’s no plugin registry, no social network for AI agents, no cryptocurrency tokens. It does what you need and nothing else.

Learning. This is the one people underestimate. Building your own agent teaches you how LLMs actually work in production — context windows, tool use, prompt design, state management, failure modes. Using OpenClaw teaches you how to configure OpenClaw.

What OpenClaw Gets Right

Credit where it’s due: the vision of an AI agent that operates across your entire digital life is correct. The idea that it should be open-source is correct. The emphasis on persistent memory is correct.

The disagreement is about approach. OpenClaw tries to be everything for everyone, which means it’s complex, hard to secure, and impossible to fully understand. The alternative is to build exactly what you need, understand every piece, and extend it when your needs change.

The Architecture, Concretely

Here’s what runs on the Pi:

Telegram bot (Python, ~200 lines) — receives messages, routes commands, returns responses
Agent orchestration — something like pi works well here. It’s a coding agent with tool use, extensions, and skill support that you can drive programmatically via its SDK. The Telegram bot spawns a pi session per conversation, hands it the context from the git repo, and lets it execute tool calls. Lightweight, hackable, and you can read the entire source.
CLI tools — small scripts for email (IMAP/SMTP), calendar (CalDAV), tasks, web search
Git repo — notes, compass files, skill definitions, references
Cron jobs — morning briefing, inbox check, follow-up reminders

That’s it. No Docker orchestration, no plugin framework, no vector database, no agent-to-agent social network. Each piece is simple enough to hold in your head.

Getting Started

If you’ve been following along with the note-taking system and operational layer, you already have the hard part: structured context that AI can work with.

The remaining steps:

Set up a Pi with SSH access
Clone your notes repo
Create a Telegram bot via BotFather
Write a minimal bot that forwards messages to an LLM API
Add git pull/push hooks around sessions
Add CLI tools one at a time as you need them

Start small. A bot that can answer questions about your notes and tasks is already useful. Add email access when you trust the setup. Add calendar integration when you need it. Each addition is a small, understandable increment.

The Point

OpenClaw proves there’s demand for personal AI agents. But it also demonstrates the standard industry pattern: take a fundamentally simple idea, add every feature anyone might want, and ship a system that no single person can understand or secure.

Personal infrastructure should be personal. You should be able to read every line of code that has access to your email. You should know exactly what gets persisted and where. You should be able to explain the entire architecture in a five-minute conversation.

A Raspberry Pi, a Telegram bot, and git hooks. That’s not a limitation — that’s the design.

From Markdown Notes to AI Chief of Staff

Sun, 01 Feb 2026 00:00:00 GMT

A couple months ago I wrote about a note-taking system built on plain markdown and 50 lines of bash. The thesis: when your entire system is transparent to AI, it can write whatever tooling you need. No plugins, no APIs — just files and scripts.

That thesis was correct. But it undersold what happens when you keep going.

The system isn’t notes anymore. It’s an operational layer. AI agents run morning routines, process meeting transcripts, triage inboxes, maintain their own memory across sessions, and flag when I’ve dropped a commitment. The same markdown files, the same transparency principle — but the ceiling turned out to be much higher than “better notes.”

The Cold Start Problem

The original system had a gap. Every conversation with AI started from zero. It didn’t know what I was working on, what I’d decided last week, or what mattered to me. I’d spend the first few messages re-establishing context, or worse — get advice that contradicted a decision I’d already made.

Notes existed, but they were organized for me to find things. Not for AI to orient itself.

The fix was a folder called compass/ with four files:

compass/
├── goals.md        # What I'm trying to achieve
├── context.md      # What's in flight right now
├── preferences.md  # How I work and communicate
└── decisions.md    # What's been decided and why

context.md — the session handoff

This is the most important file. It’s a queue of open threads — not a project wiki, not a journal. Three sections: In Flight, Blocked/Waiting, Needs Attention. Each entry is a one-liner: what it is plus current status.

## In Flight
- API redesign — open question on auth model blocks shipping
- Onboarding flow rewrite — new design approved, implementation started
- Hiring — backend role, two candidates in pipeline
 
## Blocked / Waiting
- Enterprise pilot — blocked on their internal security review
 
## Needs Attention
- Monitoring gaps — no alerting on payment failures, needs setup

When AI reads this at the start of a session, it knows what I’m dealing with. It can prioritize, notice conflicts, and avoid suggesting things I’ve already rejected. The file acts as a handoff between sessions — like briefing notes for an incoming shift.

preferences.md — behavioral memory

When I tell AI “don’t do X” or “I prefer Y,” that correction lasts exactly one session. Next time, same mistake. preferences.md makes corrections permanent:

## Communication
- Prefers brief, direct answers — no filler, no excessive caveats
- No emojis unless asked
 
## Decision Making
- Pragmatic over perfect — "good enough now" beats "ideal later"
 
## Behavioral Corrections
- 2026-01-15: "Don't summarize what I just said back to me" — context

The AI reads this file, adjusts its behavior, and the correction sticks. Over time the file becomes a fairly accurate profile of how you work.

decisions.md — the decision log

This one prevents the most insidious problem: re-litigating settled questions. When a significant decision gets made, it’s logged with reasoning and alternatives considered. Two weeks later when I’m second-guessing a technical or business decision, the AI can point to the entry and say “here’s why you decided this.”

References: Stable Context

Notes are dated and temporal. But some context is durable — it applies across sessions and doesn’t change with each meeting.

references/
├── professional-self.md     # Who I am, background
├── products.md              # Product descriptions
├── projects/                # Living project briefs
└── people/
    ├── internal/            # Team members
    ├── partners/            # External partners
    ├── candidates/          # Hiring pipeline
    └── customers/           # Customer accounts

Project briefs get updated in place as understanding evolves — architecture decisions, shifted strategy, changed ownership. They’re not notes about a project. They’re the canonical source of truth for what the project is.

People files are the same idea. When I’m about to meet someone, the AI can read their file and tell me: here’s who they are, here’s what we discussed last time, here’s what’s pending between us. No prep work on my part.

Skills: Workflows as Markdown

This is where it gets interesting.

A “skill” is a markdown file that teaches AI a multi-step workflow. Not code. Not a plugin. Just instructions that reference tools.

Here’s the structure of the morning kickoff skill:

# Morning Kickoff
 
Create a daily "today" page with an overview of the day ahead.
 
## Steps
 
1. Get today's date and determine the day of week
2. Create the daily file at `daily/YYYYMMDD--dayofweek.md`
3. Gather information using the relevant skills:
   - Check work calendar and personal calendar
   - Check today's tasks, upcoming tasks, deadlines
   - Check email for anything unread or requiring attention
4. Read all compass files to fully load context

That’s it. The AI reads this, follows the steps, calls the appropriate tools (email, calendar, task manager), and produces a daily briefing. No code to maintain. No API integration to debug.

I have about 20 of these now:

Morning kickoff — daily briefing with calendar, tasks, emails, and rehydrated context
Weekly review — audit progress against goals, clean up stale threads, identify stuck items
Inbox triage — process work email, personal email, and task inbox in sequence
Process meetings — pull transcripts from Notion, extract tasks, create meeting notes, flag Zettelkasten connections
People prep — briefing about a specific person before a 1:1
Follow-ups — scan notes and emails for commitments that aren’t tracked as tasks
Wrap-up — end-of-session routine that saves context and logs decisions

The realization that made this click: most operational work is a checklist. And checklists are exactly what AI is good at following — especially when the tools and context are transparent.

Why markdown instead of code? Because I can edit a workflow in 30 seconds. Adding a step, changing the order, adjusting what gets checked — it’s just editing a text file. And the AI reads it natively. No parsing, no compilation, no framework.

The Session-End Hook

The skills above all require me to invoke them. The session-end hook doesn’t.

It’s an extension that fires automatically when a session ends. It spawns a background AI agent that reads the session transcript and decides what’s worth persisting. It updates compass files, project briefs, and people references — without me asking.

The prompt for this agent is specific about what to look for:

Status changes — did a project move forward, get blocked, or resolve?
Decisions — was something significant decided?
Preferences — did I correct the AI’s behavior?
People info — did new context surface about someone I work with?

And specific about what to ignore: routine debugging, trivial file edits, implementation details.

This closes a loop that matters: the system maintains itself. I don’t have to remember to update context after each session. I don’t have to decide whether something was worth logging. The agent handles triage, and I review the results.

A Typical Day

Morning. I run /morning-kickoff. AI checks both calendars, pulls tasks and deadlines, scans email for anything urgent, reads compass files to rehydrate context. Produces a daily page I can glance at over coffee.

During work. When I ask for advice — on priorities, architecture decisions, how to handle a conversation — the AI already knows what I’m working on, what’s been decided, and how I prefer to operate. The advice is grounded, not generic.

After meetings. /process-meetings pulls transcripts from Notion, creates summary notes, extracts follow-up tasks, and checks them against my task manager. If a discussion maps to an existing Zettelkasten concept, it flags the connection.

End of day. Either I run /wrap-up explicitly, or the session-end hook captures context automatically. Open threads get updated. Decisions get logged. Nothing falls through the cracks — or at least, fewer things do.

Weekly. /weekly-review compares what actually happened against goals. Cleans up stale context. Surfaces stuck items I’ve been unconsciously avoiding.

The Principle

The original post argued that transparency enables tooling. That’s still true, but it’s the small version. The bigger version: transparency enables delegation.

When AI can see the full system, has persistent memory, and has procedures to follow — it’s not a chat assistant anymore. It’s an operational layer. It maintains its own context. It runs routines. It catches things you’d miss.

The building blocks are the same: plain text, simple conventions, no proprietary formats. But the capability ceiling is determined by what you teach it to do, not by what the tool vendor ships.

Tradeoffs

Same ones as before, plus a few new ones:

Terminal-native. If you don’t live in a terminal, there’s friction. I’m building a Telegram bot to extend this outside the IDE, but it’s not there yet.
Designing for AI is a skill. You’re not just organizing notes for yourself anymore. You’re thinking about what context an AI agent needs, how to structure handoffs, what’s worth persisting. It’s a different discipline.
Trust calibration. The session-end hook updates files autonomously. Mostly it’s accurate. Occasionally it overpersists or misreads significance. You need to review its work, especially early on.
Tool surface area. This works because I have CLI tools for email, calendar, tasks, and meeting transcripts. Each new integration is a new tool to build or configure. The system is only as capable as its access.

What’s Next

The gap I’m most interested in closing: proactive behavior. Right now, AI acts when I invoke it. The morning kickoff runs because I type the command. But the information is all there — calendar, tasks, deadlines, commitments. An agent could notice that I have a meeting in an hour with someone I haven’t prepped for, or that a deadline is approaching with no progress on the task.

Moving from “AI that helps when asked” to “AI that notices when something needs attention” is the next step. The foundation — transparent files, persistent memory, structured workflows — is already in place. The trigger just needs to shift from manual to event-driven.

Fifty lines of bash was the starting point. The system it grew into is unrecognizable from those origins. But the principle hasn’t changed: keep everything transparent, and AI will surprise you with what it can do.

📚 The Middle Passage — James Hollis

Sun, 01 Feb 2026 00:00:00 GMT

Provisional Personality

The identity you build in the first half of life isn’t really yours. It’s assembled from parental imprinting, cultural scripts, and adaptive strategies you developed as a child to stay safe, loved, and accepted. People-pleasing, overachieving, withdrawal, control — these survival mechanisms crystallize into personality traits you mistake for who you are.

The key point: this personality works. It gets you through childhood, school, career, relationships. It’s functional. But it’s a collage of other people’s expectations and your adaptive responses to them. The middle passage begins when this provisional self starts breaking down — when the strategies that got you here stop working, and the life you built feels hollow despite looking right from the outside.

The Midlife Crisis as Summons

Hollis reframes the midlife crisis not as a breakdown but as a summons. The provisional personality produces diminishing returns. Achievement fails to satisfy. The shadow — everything you repressed to maintain the functional self — accumulates pressure until it forces its way into consciousness.

The crisis presents a fork. Refuse the summons: double down on old strategies, numb out, chase a younger version of the same life. New car, new partner, same patterns. Or accept the summons: let the provisional personality collapse, endure the disorientation, and begin asking who you actually are beneath the scripts.

Depression and anxiety aren’t pathology here — they’re signals. The psyche saying the life you’re living is too small for the person you’ve become.

Individuation

Jung’s term for the process of becoming who you actually are. Not self-improvement — not optimizing the provisional personality — but dismantling it to find what’s underneath.

This means confronting your shadow (the repressed parts running your life from the basement), withdrawing projections (recognizing that what triggers you in others is often your own unowned material), and differentiating from complexes (the emotionally charged patterns like “I’m not good enough” that hijack behavior).

Whatever you sacrificed to maintain the provisional personality now demands its due. The creative impulse you buried, the values you suppressed, the way of being you abandoned to fit in. It’s grueling work, never complete. The unconscious is inexhaustible. The point is the ongoing relationship with it, not arrival.

📚 Falling Upward — Richard Rohr

Sun, 25 Jan 2026 00:00:00 GMT

Transcend and Include

Genuine development doesn’t replace earlier stages — it builds on and contains them. If you’ve authentically moved to a more mature perspective, you’ve necessarily passed through the earlier ones. They live within you.

This makes judgment structurally incoherent. You’d be condemning a version of yourself.

Flip side: if someone is superior or dismissive about their “advanced” views, that’s evidence they haven’t actually integrated the earlier stage. They’ve bypassed rather than transcended. True development brings humility automatically.

Teachings as Scaffolding

Religious teachings are entry points, not destinations. They get you “in the right ballpark” but eventually need to be transcended through contemplation.

The distinction:

Kataphatic — what we can say about God (doctrines, images, concepts)
Apophatic — what we cannot say, the mystery beyond language

This isn’t modern revisionism. It’s the older contemplative tradition — Desert Fathers, Meister Eckhart, John of the Cross. The conceptual frameworks eventually become obstacles if you cling to them.

Pushed vs Pulled

What motivates us shifts between the two halves of life:

First half: pushed from behind — by ego, wounds, fear, the need to prove yourself
Second half: pulled from ahead — by soul, meaning, calling

Early-life drive isn’t bad, but it’s often compensatory — proving worth, escaping shame. Second-half motivation feels different: less frantic, arising from abundance rather than lack.

📚 High Output Management — Andy Grove

Mon, 19 Jan 2026 00:00:00 GMT

Hybrid Organization

Neither pure functional (group by expertise) nor pure mission-oriented (group by objective) structures work at scale. Grove argues for combining both:

Mission-oriented units handle customer-facing work — speed and autonomy to respond to markets
Functional units provide centralized services where scale matters — HR, legal, specialized expertise

The question isn’t “which structure?” but “which structure for which work?”

Dual Reporting

The hybrid creates matrix management — individuals reporting to both a functional manager (expertise, standards, career) and a mission manager (priorities, delivery, outcomes).

This friction is a feature. It forces trade-off conversations to happen explicitly rather than letting one priority silently dominate. The matrix doesn’t eliminate conflict — it institutionalizes it. The alternative isn’t harmony, it’s hidden dysfunction.

Task-Relevant Maturity

Adjust management style based on someone’s experience with a specific task — not their general seniority.

Low TRM: Structured, directive. Tell them what, when, how. Monitor closely.
Medium TRM: Coaching. Two-way communication, set goals together, regular but less intensive monitoring.
High TRM: Delegative. Set objectives and constraints, then step back. Monitor at milestones.

A senior engineer might need high TRM style for backend architecture but low TRM style for their first people management responsibility. Mismatching style to TRM fails in both directions.

Pairing Indicators

Any single metric creates incentive to game it. Pair complementary indicators that create natural tension:

Quantity	Quality
Units produced	Defect rate
Tickets resolved	Reopened rate
Features shipped	Bug count

Neither metric can be gamed in isolation because optimizing one at the expense of the other becomes immediately visible.

Checkpoint vs Outcome KRs

Grove’s original Key Results were checkpoints — concrete milestones. Modern OKR literature reframes them as outcome measurements — metrics indicating progress.

Different failure modes:

Checkpoints force sequencing discipline but can become output-focused without asking if it mattered
Outcomes keep focus on impact but are harder to pace against

Execution mode (known path) favors checkpoints. Innovation mode (still searching) favors outcomes. Mismatch between style and context is why most OKR implementations fail.

Tension with Theory of Constraints

Grove measures productivity at the individual work unit level — improving any unit’s productivity improves the whole.

Goldratt would disagree. In any system with dependencies, only the constraint determines throughput. Optimizing non-constraints creates waste: inventory piles up, WIP increases, resources consumed producing things that can’t flow through.

Grove’s indicator pairing partially guards against this, but he never asks “is this work unit even the bottleneck?” before optimizing it.

📚 The Sleep Solution — Chris Winter

Sun, 18 Jan 2026 00:00:00 GMT

Sleep State Misperception

Many people who believe they “don’t sleep” are actually sleeping more than they realize. Light sleep and brief awakenings can feel like being awake, creating a gap between perceived and actual sleep time.

The test: if you feel reasonably functional the next day, you slept more than you think. True sleep deprivation makes functioning nearly impossible.

The anxiety about insomnia often causes more harm than the actual sleep loss.

Vigilance vs Sleepiness

Vigilance and sleepiness are separate neural systems, not opposites on a single spectrum.

Sleepiness is a homeostatic pressure that builds the longer you’re awake. Vigilance is an alerting response triggered by stimuli — novelty, stress, checking your phone, getting frustrated in bed.

“I’m not tired anymore” often means vigilance temporarily kicked in. The underlying sleep pressure is still there. Wait it out.

Insomnia ≠ Sleep Deprivation

Insomnia is difficulty sleeping when and how you want to. Sleep deprivation is actually getting very little sleep. They’re conflated constantly, but they’re fundamentally different.

True sleep deprivation (like lab studies where subjects are kept awake) causes severe impairment. Most people with insomnia are still getting meaningful sleep — the distress and preoccupation around sleep is often the core problem, not the sleep itself.

Importing scary research about sleep deprivation into your insomnia amplifies anxiety, which makes sleep harder, which increases anxiety.

The Insomniac Identity Trap

Building an identity around being “a bad sleeper” creates a self-reinforcing cycle:

Experience some sleep difficulty
Absorb alarming media about sleep deprivation consequences
Develop anxiety and hypervigilance around sleep
Anxiety disrupts sleep further
Adopt “insomniac” identity as explanation
Identity reinforces belief that good sleep isn’t possible
Return to step 3

Breaking it requires separating what you’re experiencing from who you are.

📚 It's Not Luck — Eliyahu Goldratt

Tue, 06 Jan 2026 00:00:00 GMT

The Thinking Processes

While The Goal gave us the Five Focusing Steps for physical constraints, It’s Not Luck introduces the Thinking Processes — a logical toolkit for solving complex problems, especially policy constraints and conflicts.

Five tools, five questions:

Current Reality Tree — What to change?
Evaporating Cloud — What to change to?
Future Reality Tree — Will it work?
Prerequisite Tree — What’s in the way?
Transition Tree — What actions?

Each produces output that feeds the next. Skip steps and you fail in predictable ways.

Why Known Problems Persist

If you know about a problem and it persists, something is stopping you from solving it. That something is almost always a perceived trade-off.

The interesting question shifts from “why does this exist?” to “why haven’t we solved it?”

Most answers reduce to: “I believe I can’t have X without losing Y.” That belief is either a valid constraint or an assumption worth challenging.

The Evaporating Cloud

Most conflicts aren’t real — they rest on hidden assumptions. The cloud makes them visible.

                A: Common Objective
                        │
             ┌──────────┴──────────┐
             │                     │
             v                     v
       B: Requirement        C: Requirement
             │                     │
             v                     v
       D: Position ◄── conflict ──► D': Position

Both sides want the same objective (A) but believe they need contradictory things (D vs D’). Surface the assumptions behind each arrow. Invalidate one, and the conflict evaporates.

CRT to EC: The Handoff

The Current Reality Tree maps cause-effect chains from symptoms to root cause. But finding the root cause isn’t enough — you need to understand why it persists.

Root causes persist because of perceived conflicts. The EC structures that conflict. CRT answers what, EC answers why we haven’t fixed it.

Future Reality Tree

The FRT is a flight simulator for your solution. Take the injection from the EC, trace its effects forward through cause-effect logic.

Two things to check:

Does it eliminate the undesirable effects from the CRT?
Does it create new problems (negative branch reservations)?

For every “what could go wrong,” either add a preventive injection or modify the solution. Critics become contributors — they’re stress-testing your logic, not attacking your idea.

From Objectives to Actions

The Prerequisite Tree captures obstacles (“we can’t because…”) and converts each into an intermediate objective — a state that overcomes the obstacle. Not actions, states.

The Transition Tree then breaks each intermediate objective into concrete action sequences with explicit cause-effect logic: given reality and need, if we do X, then Y results.

PRT answers “what conditions must exist.” TT answers “what exactly do we do.”

The Full Sequence

CRT ──► EC ──► FRT ──► PRT ──► TT ──► Implementation
 │       │       │       │       │
 │       │       │       │       └─ What actions?
 │       │       │       └───────── What's in the way?
 │       │       └───────────────── Will it work?
 │       └───────────────────────── What to change to?
 └───────────────────────────────── What to change?

Skip CRT: solve the wrong problem. Skip EC: fight symptoms. Skip FRT: solution creates new problems. Skip PRT: get stuck on obstacles. Skip TT: vague plan, inconsistent execution.

📚 The Phoenix Project — Gene Kim, Kevin Behr, George Spafford

Sat, 03 Jan 2026 00:00:00 GMT

The Three Ways

Three principles for high-performing IT:

First Way (Flow) — Optimize the whole system, not local work centers
Second Way (Feedback) — Fast, peer-to-peer feedback along the value stream
Third Way (Learning) — Continual improvement through experimentation and failure

The First Way establishes left-to-right flow. The Second Way adds right-to-left feedback. The Third Way creates conditions for improvement. Without flow, chaos. Without feedback, no course correction. Without learning, stagnation.

The First Way: Flow

Optimize work movement through the entire system — Dev through Ops to customer.

Local optimizations destroy system throughput. A faster upstream just builds bigger queues at the bottleneck. “Important” means business value, not what IT finds technically interesting.

Never pass defects downstream. Stopping to fix at the source is faster than letting problems propagate. It feels slower. It isn’t.

The Second Way: Feedback

Information flowing right to left so upstream learns consequences.

The emphasis: fast, direct feedback between work centers. Not via management noticing patterns. This is Toyota’s andon cord — worker sees defect, pulls cord, production stops. Local actors, immediate response.

Feedback value is inversely proportional to delay. Finding out your code breaks production three weeks later is useless — context is gone. Finding out in minutes lets you fix it while you still remember what you did.

The Third Way: Learning

Creating conditions where improvement happens.

Two forms: experimentation (trying things, accepting some will fail) and failure practice (incident response, blameless post-mortems, building resilience).

Both require time and safety. Time because improvement work needs protected capacity. Safety because people won’t surface problems or try risky ideas if failure means punishment.

The goal isn’t a system that never fails. It’s a system that learns faster than the competition.

The Brent Problem

When Brent takes on work “only Brent can do,” Brent gets smarter and everyone else misses an opportunity to catch up.

Every time the expert takes on work:

Expert gets better (more context, deeper understanding)
Everyone else stays where they are

The gap widens. More work flows to the expert. They become more irreplaceable. More work flows to them. Vicious cycle.

“Only Brent can do this” usually means “Brent can do this fastest right now.” The question: optimizing for this week or this year?

Breaking the cycle means letting others struggle. Slower throughput now for higher capacity later. Treating knowledge transfer as real work, not overhead.

Man, Machine, Method, Metric

The 4 Ms — what’s necessary for any work center to function:

Man — Person with right skills and availability
Machine — Tools and resources needed
Method — Documented process
Metric — How you measure and track

When work isn’t flowing: Do we have someone? Do they have tools? Is there a clear process? Can we measure what’s happening?

People assume the constraint is the person. Often it’s process or tooling.

Shrink Your Dependency Surface: Use Cases Over Services

Fri, 02 Jan 2026 00:00:00 GMT

Every time you add a method to a service, you expand the dependency surface for everyone who uses it. A TaskService with 20 methods means every consumer depends on all 20 — even if they only need one. As your application grows, this coupling compounds. Then the circular dependencies appear.

Services Grow, Surfaces Expand

The traditional service pattern starts clean:

type TaskService struct {
    repo *TaskRepository
}
 
func (s *TaskService) Create(title string) (*Task, error) { ... }

Then requirements arrive. Tasks need projects. Tasks need areas. Tasks need tags, recurrence, completion logic. The service grows:

type TaskService struct {
    repo           *TaskRepository
    projectService ProjectService
    areaService    AreaService
}
 
func (s *TaskService) Create(...) error        { ... }
func (s *TaskService) Complete(...) error      { ... }
func (s *TaskService) SetProject(...) error    { ... }
func (s *TaskService) SetArea(...) error       { ... }
func (s *TaskService) AddTag(...) error        { ... }
func (s *TaskService) SetRecurrence(...) error { ... }
// ... 15 more methods

Each new method potentially adds cross-domain dependencies. Every consumer of TaskService now implicitly depends on ProjectService and AreaService — even if they only call Create.

When Surfaces Collide

This gets worse when bounded contexts need each other. Consider:

Creating a task requires validating the project exists
Completing a project requires completing all its tasks

The traditional approach:

type TaskService struct {
    projectService ProjectService  // Need to validate project
}
 
type ProjectService struct {
    taskService TaskService  // Need to complete child tasks
}

Circular import. The compiler refuses.

The frustrating part: CreateTask only needs project lookup. CompleteProject only needs task completion. These operations don’t overlap. But because both are bundled into monolithic services, the entire service becomes the dependency unit.

Use Case Architecture

The fix: make each operation its own struct with exactly the dependencies it needs.

type ProjectLookup interface {
    Execute(name string) (*Task, error)
}
 
type CreateTask struct {
    Repo          *TaskRepository
    ProjectLookup ProjectLookup
}
 
func (c *CreateTask) Execute(title string, opts *CreateOptions) (*Task, error) {
    if opts.ProjectName != "" {
        p, err := c.ProjectLookup.Execute(opts.ProjectName)
        if err != nil {
            return nil, err
        }
        // assign task to project
    }
    // ... create the task
}

The interface is defined by the consumer, not the project domain. CreateTask doesn’t depend on everything projects can do — just on “something that can look up a project by name.”

Solving Circular Dependencies

With use case architecture, the circular dependency disappears:

CreateTask needs ProjectLookup (interface) → satisfied by GetProjectByName
CompleteProject needs TaskCompleter (interface) → satisfied by CompleteTasks

No circular imports because:

Neither domain knows the other’s full API
Interfaces are consumer-defined, not provider-defined
Each use case declares exactly what it needs

The dependency graph stays a DAG. Always.

The Wiring

Dependencies resolve in a neutral location—a composition root:

func New(db *database.DB) *App {
    taskRepo := task.NewRepository(db)
    areaRepo := area.NewRepository(db)
 
    getAreaByName := &areausecases.GetAreaByName{Repo: areaRepo}
    getProjectByName := &taskusecases.GetProjectByName{Repo: taskRepo}
 
    createTask := &taskusecases.CreateTask{
        Repo:          taskRepo,
        ProjectLookup: getProjectByName,
        AreaLookup:    getAreaByName,
    }
 
    return &App{
        CreateTask:       createTask,
        GetProjectByName: getProjectByName,
        // ... other use cases
    }
}

Each use case gets exactly what it needs. GetProjectByName satisfies the ProjectLookup interface that CreateTask defined. No service layer coordinates everything—use cases compose directly.

Trade-offs

This pattern has costs:

More files. One file per operation. A domain with 10 operations has 10 use case files instead of one service file.

Verbose wiring. The composition root grows as use cases multiply. In a real application, app.go might wire 30+ use cases.

Requires discipline. Use cases must stay focused. The temptation to add “just one more dependency” defeats the purpose.

The benefits:

Easy to test. Mock only the interfaces a use case declares. CreateTask tests mock ProjectLookup and AreaLookup—nothing else.

Easy to understand. Open a file, see one operation. No scrolling through a 500-line service to find the method you care about.

Changes stay isolated. Modifying CreateTask doesn’t affect CompleteTasks. Different operations can evolve independently.

No circular dependency risk. The architecture makes cycles impossible by construction.

The Core Insight

The unit of dependency should match the unit of operation. Services bundle unrelated operations, creating artificial coupling between consumers and producers. Use cases keep dependencies minimal and directional—each operation depends only on what it actually needs.

As your application grows, the dependency surface stays proportional to individual operations, not to the sum of everything each domain can do.

📚 Critical Chain — Eliyahu Goldratt

Wed, 31 Dec 2025 00:00:00 GMT

Critical Chain vs Critical Path

Critical Path considers task dependencies. Critical Chain considers task dependencies and resource dependencies.

Critical Path (ignores resources):
 
Task A ──────┐
             ├──→ Task C
Task B ──────┘
 
Duration: max(A, B) + C
 
 
Critical Chain (same person does A and B):
 
Task A ──────→ Task B ──────→ Task C
 
Duration: A + B + C

Critical Path gives you an optimistic fiction. Critical Chain gives you what will actually happen.

The Five Steps Applied to Projects

Identify — Find the critical chain: the longest path through both task and resource dependencies.
Exploit — Strip hidden safety from estimates. Eliminate multitasking on critical chain tasks. Never let critical chain resources wait for inputs.
Subordinate — Everything else serves the chain. Feeding work delivers before it’s needed. Resources prioritize chain work over everything.
Elevate — Add resources, parallelize, cut scope. Only after exploitation is maxed out.
Repeat — The chain shifts. Back to step 1.

The Buffer System

Instead of hiding safety in each task, consolidate it:

Project Buffer — End of the critical chain. Protects the delivery date.
Feeding Buffer — Where non-critical paths join the chain. Protects against delays in feeding work.
Resource Buffer — Alerts key resources their task is coming. Not time, just a heads-up.

Same total safety time, dramatically different outcomes.

Why Consolidated Buffers Work

Three reasons distributed safety fails:

Statistical aggregation. Uncertainties don’t all materialize. Pooled buffers let variations cancel out. A 10-day project buffer protects better than 10 × 1-day task buffers. Insurance math.

Behavioral effects. Parkinson’s Law: work expands to fill available time. Student Syndrome: people delay until pressure builds. Built-in safety gets consumed whether needed or not. Consolidated buffers are only touched when actually required.

Visibility. Distributed safety hides problems until too late. Consolidated buffers give you “60% consumed with 40% done” — a clear signal to act.

The Late Start Principle

Starting late is better than starting early. Counterintuitive, but the math checks out.

Early start problems:

Student Syndrome means early start just moves procrastination earlier
Parkinson’s Law means extra time gets used whether needed or not
Early finishes never get passed on — people polish, delay reporting, or start something else
More WIP means more context switching and resource collisions

Late start with buffers:

Minimizes work-in-progress
Reduces resource contention
Avoids premature work on requirements that might change
Buffer provides protection without the behavioral problems

The rule: start as late as possible while buffers absorb variation. Starting early feels safer. Starting late with buffers is safer.

📚 The Goal — Eliyahu Goldratt

Sun, 28 Dec 2025 00:00:00 GMT

The Core Insight

System throughput is determined by the bottleneck. Improving anything else is an illusion of progress.

[Wide] → [Wide] → [NARROW] → [Wide] → Output

Widening the wide sections changes nothing. This is obvious when drawn as pipes, yet organizations constantly pour resources into non-constraints and wonder why output doesn’t improve.

Five Focusing Steps

Identify — Find the bottleneck.
Exploit — Squeeze every drop of capacity from it without spending money.
Subordinate — Everything else serves the constraint. Deliberately hold back non-bottlenecks.
Elevate — Now invest. Only after exploitation is maxed out.
Repeat — A new constraint emerges. Back to step 1.

Step 3 is where most organizations fail. Holding resources back feels wasteful. It isn’t.

Finding Bottlenecks

Look for where inventory piles up. The constraint is always busy while downstream resources are starved. Follow the queues.

The constraint isn’t always a machine or person. It can be a policy, a skill (only one person knows how to do X), or the market itself.

Activation vs Utilization

Keeping everyone busy is not the same as being productive. A worker upstream of the bottleneck running at 100% just builds piles of WIP. A worker downstream starves regardless.

Idle time at non-bottlenecks isn’t waste. It’s the point.

Drum-Buffer-Rope

The hiking analogy: put the slowest kid (Herbie) at the front and tie a rope to him. The whole group moves at his pace without spreading apart.

Drum — The bottleneck sets the pace
Buffer — Inventory in front of the constraint so it never starves
Rope — Ties input to bottleneck capacity

Only regulate what enters the system. If the bottleneck processes 10 units/hour, release 10 units/hour at the start. Everything in between will sort itself out.

Throughput Accounting

Traditional cost accounting rewards overproduction — inventory is an asset on the balance sheet. Managers game this.

TOC flips it:

Throughput (T) — Revenue minus truly variable costs
Inventory (I) — Money stuck in the system
Operating Expense (OE) — Money spent to turn I into T

Net Profit = T - OE. ROI = (T - OE) / I.

The priority for decisions: increase throughput first (biggest leverage), reduce inventory second (frees cash), reduce operating expense last (often counterproductive if it touches the constraint).

A Note-Taking System That Works With AI, Not Against It

Thu, 11 Dec 2025 00:00:00 GMT

Obsidian, Notion, Roam — there’s no shortage of good note-taking apps. But I wanted something where AI could not just read my notes, but write tooling for them. So I built a system that’s about 50 lines of bash, and AI can extend it trivially.

The Idea

This isn’t anti-Obsidian. Obsidian is great, and it uses markdown too. The difference is what happens when you want a new feature.

In most apps, you either wait for it, find a plugin, or learn the plugin API and build it yourself. In a system that’s just files and scripts, you ask Claude “write me a script that does X” and it works. No plugins to learn, no API to integrate—just scripts that operate on files.

When the whole system is transparent, AI can reason about it completely.

The Structure

Notes are plain markdown files with date-prefixed names:

20251210--project-kickoff.md
20251211--api-design-notes.md

Organized in folders by project or area:

notes/
├── active/
│   ├── client-project/
│   ├── side-project/
│   └── personal/
├── archive/
└── scripts/

Active projects live in active/. When a project wraps up, it moves to archive/. The scripts/ folder holds the tooling.

Dependencies: fzf for fuzzy selection, bat for syntax-highlighted previews, ripgrep for fast search. Standard Unix tools.

Five Scripts I Found Useful

These are the actions I needed. Your list might be different — that’s the point.

nn — New Note

Pick a folder, enter a title, creates a dated markdown file and opens it.

#!/bin/bash
 
NOTES_DIR="/Users/daniel/notes"
ACTIVE_DIR="$NOTES_DIR/active"
PICKER="fzf"
 
folders=$(find "$ACTIVE_DIR" -mindepth 1 -maxdepth 1 -type d -exec basename {} \; | sort)
 
if [ -z "$folders" ]; then
    echo "No folders found in $ACTIVE_DIR"
    exit 1
fi
 
folder=$(echo "$folders" | $PICKER --prompt="Select folder: ")
 
if [ -z "$folder" ]; then
    echo "No folder selected"
    exit 1
fi
 
read -p "Title: " title
 
if [ -z "$title" ]; then
    echo "Title cannot be empty"
    exit 1
fi
 
date_stamp=$(date +%Y%m%d)
slug=$(echo "$title" | tr '[:upper:]' '[:lower:]' | tr ' ' '-' | tr -cd 'a-z0-9-')
filename="${date_stamp}--${slug}.md"
filepath="${ACTIVE_DIR}/${folder}/${filename}"
 
echo "# ${title}" > "$filepath"
echo "" >> "$filepath"
 
echo "Created: $filepath"
${EDITOR:-vim} "$filepath"

nf — Find Note

Fuzzy-find notes by filename with preview. Use -p to pick a specific folder first.

#!/bin/bash
 
NOTES_DIR="/Users/daniel/notes"
ACTIVE_DIR="$NOTES_DIR/active"
PICKER="fzf"
 
pick_folder=false
for arg in "$@"; do
  case $arg in
    -p|--pick) pick_folder=true ;;
  esac
done
 
if $pick_folder; then
  folders=$(find "$ACTIVE_DIR" -mindepth 1 -maxdepth 1 -type d -exec basename {} \; | sort)
 
  if [ -z "$folders" ]; then
    echo "No folders found in $ACTIVE_DIR"
    exit 1
  fi
 
  folder=$(echo "$folders" | $PICKER --prompt="Select folder: ")
 
  if [ -z "$folder" ]; then
    echo "No folder selected"
    exit 1
  fi
 
  search_dir="$ACTIVE_DIR/$folder"
else
  search_dir="$ACTIVE_DIR"
fi
 
notes=$(find "$search_dir" -name "*.md" -type f | sort)
 
if [ -z "$notes" ]; then
  echo "No notes found"
  exit 1
fi
 
filepath=$(echo "$notes" | $PICKER --prompt="Select note: " --preview="bat --color=always --style=plain {}")
 
if [ -z "$filepath" ]; then
  echo "No note selected"
  exit 1
fi
 
${EDITOR:-vim} "$filepath"

ns — Search Content

Full-text search across all notes. Opens at the matching line.

#!/bin/bash
 
NOTES_DIR="/Users/daniel/notes"
ACTIVE_DIR="$NOTES_DIR/active"
PICKER="fzf"
 
pick_folder=false
for arg in "$@"; do
  case $arg in
    -p|--pick) pick_folder=true ;;
  esac
done
 
if $pick_folder; then
  folders=$(find "$ACTIVE_DIR" -mindepth 1 -maxdepth 1 -type d -exec basename {} \; | sort)
 
  if [ -z "$folders" ]; then
    echo "No folders found in $ACTIVE_DIR"
    exit 1
  fi
 
  folder=$(echo "$folders" | $PICKER --prompt="Select folder: ")
 
  if [ -z "$folder" ]; then
    echo "No folder selected"
    exit 1
  fi
 
  search_dir="$ACTIVE_DIR/$folder"
else
  search_dir="$ACTIVE_DIR"
fi
 
result=$(rg --line-number --color=always . "$search_dir" | $PICKER --ansi --prompt="Search content: " --preview="bat --color=always --style=plain --highlight-line {2} {1}" --delimiter=: --preview-window=+{2}-10)
 
if [ -z "$result" ]; then
  echo "No match selected"
  exit 1
fi
 
filepath=$(echo "$result" | cut -d: -f1)
line=$(echo "$result" | cut -d: -f2)
 
${EDITOR:-vim} "+$line" "$filepath"

np — New Project

Create a new folder in active/.

#!/bin/bash
 
NOTES_DIR="/Users/daniel/notes"
ACTIVE_DIR="$NOTES_DIR/active"
 
read -p "Folder name: " folder
 
if [ -z "$folder" ]; then
  echo "Folder name cannot be empty"
  exit 1
fi
 
slug=$(echo "$folder" | tr '[:upper:]' '[:lower:]' | tr ' ' '-' | tr -cd 'a-z0-9-')
 
mkdir -p "$ACTIVE_DIR/$slug"
 
echo "Created: $ACTIVE_DIR/$slug"

na — Archive Project

Move a folder from active/ to archive/.

#!/bin/bash
 
NOTES_DIR="/Users/daniel/notes"
ACTIVE_DIR="$NOTES_DIR/active"
ARCHIVE_DIR="$NOTES_DIR/archive"
PICKER="fzf"
 
folders=$(find "$ACTIVE_DIR" -mindepth 1 -maxdepth 1 -type d -exec basename {} \; | sort)
 
if [ -z "$folders" ]; then
  echo "No folders found in $ACTIVE_DIR"
  exit 1
fi
 
folder=$(echo "$folders" | $PICKER --prompt="Select folder to archive: ")
 
if [ -z "$folder" ]; then
  echo "No folder selected"
  exit 1
fi
 
mkdir -p "$ARCHIVE_DIR"
 
mv "$ACTIVE_DIR/$folder" "$ARCHIVE_DIR/"
 
echo "Archived: $folder"

I also have Neovim commands that mirror these :NoteNew, :NoteFind, :NoteSearch, and so on. AI generated those too.

AI Can Write the Rest

This is the key part. When you need a new feature, you don’t search for plugins or learn an API. You describe what you want.

“Write me a script that generates a weekly summary of notes created in the last 7 days.”

“Write me a script that finds notes containing a specific phrase and lists them with context.”

“Write me a script that exports a project folder to a single markdown file for sharing.”

Claude can see the folder structure, understand the naming convention, and write a working script in seconds. No abstractions to navigate, no plugin architecture to satisfy. Just files and bash.

You don’t need to anticipate every feature. When you need something, ask.

Tradeoffs

There are real downsides:

No native mobile app. A simple markdown viewer / editor like 1Writer is enough for me though.
No sync built-in. I use git. Syncthing would also work. But you have to set it up.
No graph views. I never found these useful, but if you did — AI could generate one.
Requires terminal comfort. If fzf and vim aren’t in your vocabulary, there’s friction.

Why This Works

The system isn’t better because it’s minimal. It’s better because it’s transparent.

AI works best when it can see and modify the full system. No hidden state, no proprietary formats, no plugin APIs to reverse-engineer. Just files and scripts that do exactly what they say.

Five scripts was enough for me. Yours might be different. That’s the point.

Building the First Open Source Legal MCP Server for German Law

Sat, 29 Nov 2025 00:00:00 GMT

German law is technically public. Every federal statute sits on gesetze-im-internet.de, maintained by the Federal Ministry of Justice.

But “public” doesn’t mean “accessible.” Finding relevant law means knowing which of 6,800+ legal codes to search, which section applies, and how to parse dense legal German. Legal professionals spend years building this mental map.

AI assistants could help—if they had structured access to the data. That’s what legal-mcp does: it gives AI semantic search over German federal law via the Model Context Protocol.

The Problem: German Law Is a Labyrinth

The Data Source

Gesetze-im-internet.de is Germany’s official legal database. It contains every federal law, ordinance, and regulation—6,852 legal codes as of this writing. The data is structured XML, downloadable as ZIP archives, following a DTD called gii-norm.dtd.

This sounds usable. It isn’t.

Why It’s Hard to Use

The official site offers only exact string matching. No semantic search. This creates a fundamental mismatch between how people think about legal questions and how the system expects them to search.

Consider: you want to know about returning a defective product. Is that in the BGB (Civil Code)? The HGB (Commercial Code)? The VVG (Insurance Contract Act)? Even if you know it’s the BGB, do you search § 437 (Buyer’s Rights), § 439 (Subsequent Performance), or § 323 (Withdrawal from Contract)?

The answer is all of them, in conjunction. German legal texts are dense with cross-references: ”§ 433 BGB in Verbindung mit § 275 BGB.” The structure assumes you already know the structure.

Then there’s Rechtsdeutsch—legal German. It’s precise but archaic, using vocabulary that doesn’t match everyday language. Searching for “return defective product” won’t find “Nacherfüllung” (subsequent performance), even though that’s exactly what you need.

The Real Challenge

Keyword search fails for legal text because legal precision and natural language operate on different axes.

What people ask: “What are my rights if a product is broken?”

What the law says: “Der Käufer kann als Nacherfüllung nach seiner Wahl die Beseitigung des Mangels oder die Lieferung einer mangelfreien Sache verlangen.”

No shared keywords. Same concept. This is a vector search problem.

The Solution: MCP + Vector Search

What MCP Does

Model Context Protocol is Anthropic’s standard for giving AI structured tool access. Instead of dumping law into prompts (context limits make this impossible anyway), you give AI tools to search on demand.

Legal-mcp exposes four tools:

@mcp.tool()
async def search_legal_texts(
    query: str = Field(description="The search query text"),
    code: str = Field(description="Legal code identifier (e.g., 'bgb', 'stgb')"),
    limit: int = Field(default=5, ge=1, le=20),
    cutoff: float = Field(default=0.7, ge=0.0, le=2.0),
) -> List[LegalTextResult]:
    """
    Perform semantic search on German legal texts.
 
    Searches through legal codes using semantic similarity to find relevant
    sections based on the query text. Lower similarity scores indicate better matches.
    """

The others: get_legal_section for exact retrieval, get_available_codes for discovery, and import_legal_code for adding new codes to the database.

Why Vector Search Matters

Vector embeddings capture semantic meaning. The embedding model (Qwen3-Embedding-4B, 2560 dimensions, running locally via Ollama) converts text into high-dimensional vectors where similar concepts cluster together.

“What are my rights if a product is broken?” produces a vector that’s geometrically close to vectors for warranty and defect provisions—even without shared words.

This transforms legal search from “hope you know the exact terminology” to “describe what you’re looking for.”

The Architecture

flowchart TB
    mcp["MCP Server<br/>(FastMCP, port 8001)<br/>Tools for AI assistants"]
    store["Store API<br/>(FastAPI, port 8000)<br/>Search, import, manage legal texts"]
    pg["Postgres<br/>pgvector"]
    ollama["Ollama<br/>embed"]
    scraper["Scraper<br/>XML parse"]

    mcp -->|HTTP| store
    store --> pg
    store --> ollama
    store --> scraper

Three deployable components. The MCP server is a thin HTTP client that exposes tools. The Store API contains all business logic—search, import, embedding coordination. PostgreSQL with pgvector stores the legal texts and their embeddings.

The separation matters. MCP servers should be stateless tool interfaces. The heavy lifting happens elsewhere.

Parsing German Legal XML

The gii-norm.dtd format is comprehensive. It’s not just paragraphs—it’s nested structures with specific legal semantics:

@dataclass
class Metadaten:
    """Represents metadata for a legal norm"""
    jurabk: List[str]  # Legal abbreviation(s)
    amtabk: Optional[str] = None  # Official abbreviation
    ausfertigung_datum: Optional[str] = None  # Promulgation date
    fundstelle: List[Fundstelle] = field(default_factory=list)
    kurzue: Optional[str] = None  # Short title
    langue: Optional[str] = None  # Long title
    gliederungseinheit: Optional[Gliederungseinheit] = None
    enbez: Optional[str] = None  # Section designation
    titel: Optional[str] = None
    standangabe: List[Standangabe] = field(default_factory=list)
 
 
@dataclass
class FormattedText:
    """Represents formatted text with structure preserved"""
    content: str
    paragraphs: List[str] = field(default_factory=list)
    tables: List[Table] = field(default_factory=list)
    footnote_refs: List[str] = field(default_factory=list)

Multiple title formats (short, long, official). Footnotes that are legally significant. Tables in legislation (yes, really—tax brackets, fee schedules). Status information tracking amendments and repeals.

The parser handles about 20 distinct element types across 800 lines of code. Most of that complexity is invisible to end users, but it’s necessary to extract clean, searchable text from the source.

Extracting Structure

German legal texts use a consistent notation: § 1, § 2, and so on. Subsections are marked with parenthetical numbers in the text itself:

def _extract_sub_section(self, section: str) -> str:
    # if section number is present, the str begins with (n)
    if section.startswith("("):
        return section.split("(")[1].split(")")[0]
    return ""

Simple, but it preserves the granularity that makes legal search useful. You can retrieve § 433 BGB (the whole section on purchase contracts) or § 433 BGB Abs. 1 (just the seller’s obligations).

The Embedding Pipeline

Batch Processing

Legal codes vary wildly in size. The Grundgesetz (Constitution) has 146 articles. The BGB (Civil Code) has over 2,300 sections. The Sozialgesetzbuch (Social Code) is split across twelve books.

Embedding everything at once hits request size limits. The solution is straightforward: process in batches.

async def generate_embeddings(self, texts: List[str]) -> Sequence[Sequence[float]]:
    all_embeddings: List[Sequence[float]] = []
    batch_size = self.settings.ollama_batch_size  # default: 50
 
    for i in range(0, len(texts), batch_size):
        batch = texts[i : i + batch_size]
        logger.info(
            f"Generating embeddings for batch {i // batch_size + 1}/"
            f"{(len(texts) + batch_size - 1) // batch_size} ({len(batch)} texts)"
        )
 
        response = await self.client.embed(
            model=self.model,
            input=batch,
        )
        all_embeddings.extend(response.embeddings)
 
    return all_embeddings

Default batch size is 50. Configurable up to 500 for systems with more memory. The tradeoff is straightforward: larger batches are faster but risk 413 errors from nginx or Ollama’s request handling.

Why Local Inference

Using Ollama for embeddings means no API costs and no rate limits. Legal text volumes are substantial—importing the 267 codes relevant for public administration involves embedding tens of thousands of sections.

The tradeoff: you need a GPU for reasonable performance. On an M1 Mac, embedding the BGB takes about 10 minutes. On a system with a dedicated GPU, it’s under a minute.

Search in Practice

pgvector stores embeddings as native PostgreSQL columns. Search becomes a query:

async def semantic_search(
    self,
    query_embedding: Sequence[float],
    code: str,
    limit: int = 10,
    cutoff: Optional[float] = None,
) -> List[Tuple[LegalTextDB, float]]:
    """
    Uses cosine distance to find the most similar legal texts.
    Results are filtered by code and ordered by similarity (closest first).
 
    pgvector's <=> operator returns cosine distance where:
    - 0 means identical vectors
    - smaller values mean more similar
    - 2 means completely opposite vectors
    """
    distance_expr = LegalTextDB.text_vector.cosine_distance(query_embedding)
 
    query = (
        select(LegalTextDB, distance_expr.label("distance"))
        .filter(LegalTextDB.code == code)
        .order_by("distance")
        .limit(limit)
    )
 
    if cutoff is not None:
        query = query.filter(distance_expr <= cutoff)
 
    result = await self.session.execute(query)
    return [(row[0], float(row[1])) for row in result.all()]

The cutoff parameter lets users control precision vs. recall. Lower values (0.3-0.5) return only very similar results. Higher values (0.7-1.0) cast a wider net.

Once imported, search is fast. pgvector supports IVFFlat indexing for approximate nearest neighbor search, scaling to millions of vectors without degradation.

What This Enables

Example Query Flow

User asks Claude: “What are my warranty rights under German law?”
Claude calls search_legal_texts(query="warranty rights defect product", code="bgb")
MCP server embeds the query, searches vectors, returns relevant sections
Claude synthesizes a response citing § 437, § 439, § 323 BGB

The AI doesn’t need to know German legal structure. It asks in natural language and gets structured results it can reason about.

Use Cases

Legal research assistance. “What does German law say about…” questions that would otherwise require knowing which code to search.

Compliance checking. “Does this business practice comply with…” queries that need to find relevant regulations.

Public administration. The repository includes a curated list of 267 legal codes relevant for government work—procurement law, administrative procedure, data protection, and so on.

Trade-offs and Limitations

What This Isn’t

It’s not legal advice. The system retrieves text; it doesn’t interpret it. Legal interpretation requires understanding context, case law, and scholarly commentary that isn’t in the raw statutes.

It’s not comprehensive. Federal law only—no state law, no case law, no EU regulations. Those would require different data sources and parsers.

It’s not real-time. You import codes as snapshots. When laws change, you re-import.

Performance Considerations

Initial import is slow. Embedding a large legal code like the BGB involves thousands of API calls to the embedding model. Plan for this during setup, not during queries.

Search is fast once indexed. The pgvector IVFFlat index handles similarity search in milliseconds, even with thousands of sections.

Local inference trades money for hardware. No API costs, but you need a capable GPU for reasonable import times.

Why Open Source

Legal information should be accessible. The data is public; the tooling to use it should be too.

Others can extend it. State law, case law, other countries’ legal systems—the pattern (MCP + vector search + domain data) is reusable.

The specific implementation matters less than the approach: structure domain knowledge so AI can search it semantically, then expose that capability through a standard protocol.

The Pattern

German law is public but not accessible. Semantic search + MCP makes it usable by AI.

The same pattern applies anywhere domain expertise gates access to public information. Medical literature. Patent databases. Building codes. Any corpus where keyword search fails because the vocabulary of experts differs from the vocabulary of questions.

The repository is at github.com/ayunis-core/ayunis-legal-mcp.

Hexagonal Architecture Makes AI Assistants Extremely Efficient

Thu, 20 Nov 2025 00:00:00 GMT

The pitch for hexagonal architecture usually centers on testability and flexibility. Swap out your database. Mock your external services. Keep your domain pure.

Those arguments are valid. But there’s an underappreciated reason to adopt this pattern: it makes AI coding assistants extremely efficient.

I’ve been using Claude and Copilot across various codebases. They’re useful everywhere. But in Ayunis Core—a NestJS backend built on strict hexagonal principles—they’re noticeably faster and more accurate. Less fumbling. Fewer wrong guesses. More code that works on the first try.

The structure removes ambiguity. AI spends less time searching for where things live and makes fewer incorrect assumptions about how components interact.

Hexagonal Architecture in 60 Seconds

Quick refresher for context.

Ports are interfaces that define what your application needs from the outside world. A repository port defines persistence operations. An email port defines how to send messages.

Adapters implement those interfaces. A PostgreSQL adapter fulfills the repository port. An SMTP adapter fulfills the email port.

The domain sits at the center, containing pure business logic with no knowledge of infrastructure.

The dependency rule: everything points inward. Infrastructure depends on application. Application depends on domain. Never the reverse.

flowchart TB
    P[Presenters] --> A[Application]
    A --> D[Domain]
    I[Infrastructure] --> A
    I -.-|implements| ports([Ports])
    A -.-|defines| ports

That’s the essential mental model.

The Ayunis Core Structure

Here’s how this looks in practice. Ayunis Core is a NestJS backend with 1,106 TypeScript files across 47 modules. The structure:

src/
├── domain/           # 15 domain modules
│   └── agents/       # Example module
│       ├── domain/           # Pure business logic
│       ├── application/      # Use cases + ports
│       │   ├── ports/
│       │   └── use-cases/
│       ├── infrastructure/   # Adapters
│       │   └── persistence/
│       └── presenters/       # HTTP layer
│           └── http/
├── common/           # Cross-cutting infrastructure
└── iam/              # Auth & org management

Every domain module follows the same four-layer pattern. Every layer has a clear purpose. No exceptions.

Some metrics that matter for AI context:

Average domain entity: 67 lines
Average use case: 50-100 lines
Each use case lives in its own directory with its command/query and tests

Small, focused files. Predictable locations. Explicit contracts everywhere.

Why AI Assistants Love This Structure

The File Tree Is Documentation

AI can understand the entire system topology from the directory structure alone.

domain/agents/application/use-cases/create-agent/
├── create-agent.command.ts
├── create-agent.use-case.ts
└── create-agent.use-case.spec.ts

The path domain/agents/application/use-cases/create-agent/ tells you exactly what that code does. No searching. No “let me look for where agents are created.” The structure is the documentation.

Same principle applies everywhere:

infrastructure/persistence/local/ → local database adapter
application/ports/agent.repository.ts → the contract for agent persistence
presenters/http/dto/ → HTTP request/response shapes

When I ask Claude to add a feature, it reads the directory tree first. In Ayunis Core, that tree provides a complete map. In less structured codebases, the tree is noise—you need to read actual files to understand relationships.

Small Files Keep Context Lean

LLMs have context limits. Every token matters. Smaller files mean more relevant code fits in the context window.

Compare:

Ayunis Core: Domain entities average 67 lines. Use cases are single-purpose, 50-100 lines.
Typical NestJS: Service files commonly hit 500+ lines, mixing CRUD operations, business logic, and infrastructure concerns.

When Claude reads create-agent.use-case.ts, it gets everything relevant to creating an agent and nothing else. No scrolling past 400 lines of unrelated operations to find the logic that matters.

This compounds. When AI can hold more relevant files in context, it reasons better about how they interact.

Abstract Ports Constrain the Solution Space

Here’s a port definition from Ayunis Core:

export abstract class AgentRepository {
  abstract create(agent: Agent): Promise<Agent>;
  abstract findOne(id: UUID, userId: UUID): Promise<Agent | null>;
  abstract update(agent: Agent): Promise<Agent>;
  abstract delete(agentId: UUID, userId: UUID): Promise<void>;
}

When AI sees this abstract class, it knows exactly what operations exist. It can’t accidentally use a method that doesn’t exist. It can’t hallucinate a findByName that was never implemented.

The payoff: AI suggests this.agentRepository.create(agent) and it’s guaranteed to work. The type system and the port contract together eliminate entire categories of errors.

In codebases without clear abstractions, AI frequently suggests methods that don’t exist or calls with wrong signatures. It’s guessing based on common patterns. With explicit ports, there’s nothing to guess.

Strict Layering Means Predictable Dependencies

The dependency rule isn’t just architectural preference—it’s a constraint AI can observe and follow.

When adding a new feature, AI correctly:

Creates the domain entity in domain/
Adds the port in application/ports/
Creates the use case in application/use-cases/
Implements the adapter in infrastructure/
Wires everything in the module

It doesn’t try to import TypeORM decorators into a domain entity. It doesn’t call infrastructure code from a use case. The visible structure makes violations obvious.

I’ve watched Claude implement features in Ayunis Core with zero guidance on the architecture. It infers the pattern from what exists and follows it. In less structured codebases, the same assistant needs explicit instructions about where to put things—and still makes mistakes.

Mappers Make Transformations Explicit

Every layer has its own data shapes. Domain entities aren’t the same as database records. DTOs aren’t the same as domain entities. The mappings between them are explicit:

// Domain ↔ Database
class AgentMapper {
  toDomain(record: AgentRecord): Agent {
    /* ... */
  }
  toRecord(agent: Agent): AgentRecord {
    /* ... */
  }
}
 
// Domain ↔ HTTP
class AgentDtoMapper {
  toDto(agent: Agent): AgentResponseDto {
    /* ... */
  }
}

No magic. No auto-mapping. No implicit conversions that work until they don’t.

AI sees exactly what shape data has at each layer. It doesn’t need to guess how a database record becomes a domain entity—the mapper is right there, explicitly named, in a predictable location.

A Concrete Example

Adding a feature to Ayunis Core with AI assistance.

Scenario: Add “agent templates”—predefined agent configurations users can clone.

I describe what I want. Claude:

Reads the agents/ module structure
Creates agent-template.entity.ts in domain/
Creates AgentTemplateRepository port in application/ports/
Creates CreateAgentFromTemplate use case
Implements LocalAgentTemplateRepository in infrastructure/persistence/local/
Adds the controller endpoint in presenters/http/
Wires the providers in agents.module.ts

It followed the pattern because the pattern was visible. I didn’t need a prompt explaining hexagonal architecture or a reference document describing where files go. The structure taught the structure.

The Trade-offs

Being honest: hexagonal architecture has costs.

More Files, More Boilerplate

Creating a new use case means at minimum:

The command/query class
The use case class
A test file
Module provider registration

Simple CRUD operations feel over-engineered. For a basic “update user email” feature, you’ll create several files where a single service method might suffice in a simpler architecture.

The setup cost is real. Don’t hexagonal-ify a weekend project or a script you’ll run twice.

Learning Curve

Developers unfamiliar with the pattern need time. “Where does this go?” is a common question for the first few weeks.

But here’s the thing: the structure eventually answers those questions. Once you internalize the four layers and the dependency rule, the answers become obvious. The explicit structure that feels like overhead early becomes self-documenting later.

NestJS Friction

NestJS wants services. Hexagonal architecture wants use cases. They can coexist, but there’s ceremony:

Abstract classes for ports (to support NestJS DI)
Module providers mapping ports to adapters
Some awkwardness around request-scoped dependencies

It works. But it’s not what the framework was designed for.

Getting Started

If you want to try this:

Start small. Pick one module to restructure. Establish the four-layer pattern:

feature/
├── domain/
│   └── feature.entity.ts
├── application/
│   ├── ports/
│   │   └── feature.repository.ts
│   └── use-cases/
│       └── create-feature/
├── infrastructure/
│   └── persistence/
│       └── local-feature.repository.ts
└── presenters/
    └── http/
        └── feature.controller.ts

Name things consistently:

*.entity.ts — domain models
*.use-case.ts — application logic
*.port.ts or *.repository.ts — abstractions
*.record.ts — database schemas (separate from entities)
*.mapper.ts — layer transformations

Add ports for external dependencies. Any time you’d inject a service that talks to the outside world—database, API, file system—create a port first. Implement the adapter separately.

Let it grow organically. You don’t need to restructure your entire codebase. The pattern pays off module by module.

The Argument

The traditional case for hexagonal architecture: testability, flexibility, separation of concerns. All valid.

The newer case: it maximizes what you get from AI assistants.

Structure your code so AI can navigate it efficiently—explicit contracts, small focused files, predictable locations, no magic—and you’ll spend less time correcting its guesses and more time shipping.

Parse, Decode, Validate: A Clean Pattern for Form Handling in Go

Sun, 30 Mar 2025 00:00:00 GMT

Form validation in Go handlers tends to sprawl. One handler validates immediately, another waits until halfway through the business logic, a third barely validates at all. The inconsistency creates bugs and makes the codebase harder to reason about.

The fix is a consistent three-step pattern: Parse → Decode → Validate. Put it at the beginning of every handler, every time.

Implementation: A Practical Walkthrough

DTOs as the Foundation

Define a struct for each form. Place it at the top of the handler file - it serves as documentation for what the endpoint accepts:

type taskUpdateForm struct {
    Title  string `schema:"title" validate:"required,min=1,max=200"`
    Status string `schema:"status" validate:"required,oneof=pending in_progress completed"`
}

The schema tags map form field names to struct fields. The validate tags enforce constraints: title is required with length limits, status must be one of the allowed values.

The Validation Workflow

Step 1: Parse

if err := r.ParseForm(); err != nil {
    h.log.Error().Err(err).Msg("error parsing form")
    http.Error(w, "Bad request", http.StatusBadRequest)
    return
}

Extract form data from the request. Fail early if parsing fails.

Step 2: Decode

decoder := schema.NewDecoder()
var form taskUpdateForm
if err := decoder.Decode(&form, r.PostForm); err != nil {
    h.log.Error().Err(err).Msg("error decoding form")
    http.Error(w, "Bad request", http.StatusBadRequest)
    return
}

Gorilla Schema maps form fields to struct fields, handling type conversions.

Step 3: Validate

validate := validator.New()
if err := validate.Struct(form); err != nil {
    h.log.Error().Err(err).Msg("validation error")
 
    validationErrors := err.(validator.ValidationErrors)
    errorMessages := make([]string, 0, len(validationErrors))
 
    for _, e := range validationErrors {
        switch e.Tag() {
        case "required":
            errorMessages = append(errorMessages, fmt.Sprintf("%s is required", e.Field()))
        case "oneof":
            errorMessages = append(errorMessages, fmt.Sprintf("%s must be one of: pending, in_progress, completed", e.Field()))
        default:
            errorMessages = append(errorMessages, fmt.Sprintf("Invalid value for %s", e.Field()))
        }
    }
 
    http.Error(w, strings.Join(errorMessages, ", "), http.StatusBadRequest)
    return
}

Go-Playground Validator runs the validation rules from your struct tags. Extract specific error details to return useful messages instead of generic failures.

Complete Example

type taskUpdateForm struct {
    Title  string `schema:"title" validate:"required,min=1,max=200"`
    Status string `schema:"status" validate:"required,oneof=pending in_progress completed"`
}
 
func (h *Handler) HandleTaskUpdate(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    taskID := chi.URLParam(r, "taskID")
 
    // parse
    if err := r.ParseForm(); err != nil {
        h.log.Error().Err(err).Msg("error parsing form")
        http.Error(w, "Bad request", http.StatusBadRequest)
        return
    }
 
    // decode
    var form taskUpdateForm
    if err := h.decoder.Decode(&form, r.PostForm); err != nil {
        h.log.Error().Err(err).Msg("error decoding form")
        http.Error(w, "Bad request", http.StatusBadRequest)
        return
    }
 
    // validate
    if err := h.validate.Struct(form); err != nil {
        validationErrors := err.(validator.ValidationErrors)
        errorMessages := make([]string, 0, len(validationErrors))
 
        for _, e := range validationErrors {
            switch e.Tag() {
            case "required":
                errorMessages = append(errorMessages, fmt.Sprintf("%s is required", e.Field()))
            case "oneof":
                errorMessages = append(errorMessages, fmt.Sprintf("%s must be one of: pending, in_progress, completed", e.Field()))
            default:
                errorMessages = append(errorMessages, fmt.Sprintf("Invalid value for %s", e.Field()))
            }
        }
 
        http.Error(w, strings.Join(errorMessages, ", "), http.StatusBadRequest)
        return
    }
 
    // business logic starts here
    task, err := h.taskService.Update(ctx, taskID, form.Title, form.Status)
    if err != nil {
        h.log.Error().Err(err).Msg("error updating task")
        http.Error(w, "Internal server error", http.StatusInternalServerError)
        return
    }
 
    render.JSON(w, r, task)
}

The structure is always the same: extract context, parse, decode, validate, then business logic. Validation errors never leak into your core logic because invalid requests return early.

Why This Works

The pattern does a few things well:

Consistency - Every handler follows the same structure. Easy to read, easy to modify.
Separation - Validation happens before business logic. Invalid data can’t corrupt state.
Testability - Clear boundaries make unit tests straightforward.

It’s not clever. That’s the point. Simple patterns survive contact with growing codebases.

A Four-Part Framework for AI Prompts

Fri, 28 Mar 2025 00:00:00 GMT

While building an in-product AI assistant, I went down the prompt engineering rabbit hole. Elaborate role-playing scenarios, complex formatting tricks, instructions so detailed they could pass as legal contracts. Our prompt grew to 2,000 words with nested bullet points three levels deep.

The result: the same errors kept happening. Complexity wasn’t solving the fundamental issues.

After stripping it back to first principles, a simple four-part structure emerged that worked better than everything we’d tried before:

Context about role and environment
Tools and when to use them
Common mistakes to avoid
General guidelines and boundaries

That’s the entire framework. Here’s how each part works.

Part 1: Context and Role

Orient the AI to its environment and purpose. Answer: What platform is this? Who uses it? What’s the assistant’s role? What problems should it help with?

What makes this effective:

Factual, not fictional. Don’t ask the AI to “imagine” it’s something. State reality: “You are the X Assistant, designed to help users do Y.” No elaborate personas.

Relevant, not comprehensive. Don’t explain every feature. Focus on what’s relevant to the assistant’s function.

Scope boundaries. By stating what the assistant helps with, you implicitly define what’s outside scope.

Keep this section short—a paragraph of context, 3-4 sentences on supported use cases. Too much detail here is counterproductive. Domain knowledge belongs in tools or a knowledge base, not the system prompt.

Part 2: Tools and When to Use Them

Most approaches either list tools without context or bury usage guidelines in complex instructions. The key distinction:

Tool definition: What it does (functionality)
System prompt: When to use it (contextual guidance)

Example tool definition:

Receives resource data as csv file. The output of this tool is the filepath and the csv header.

Same tool in the system prompt:

- Use when a user asks about a specific resource to find its ID
- Use to help users find available resources matching certain criteria
- Use to verify if a resource exists before attempting operations on it

The first explains functionality. The second ties it to user scenarios.

Without this contextual guidance, the assistant often chose inappropriate tools or failed to use tools when helpful. For example, when asked “Can I book the conference room next Tuesday?”, the assistant would try to create a booking without first checking availability—a clear case for GetResourceData followed by GetAvailability.

Adding contextual usage guidance dramatically improved tool selection. Describe tools not just by what they do, but by the user needs they address.

Part 3: Common Mistakes to Avoid

A straightforward list of behaviors you’ve observed the assistant doing wrong. Format:

- DON'T [specific behavior]. INSTEAD [correct approach].

Examples:

- DON'T create a booking without checking availability. INSTEAD, use GetAvailability first.
- DON'T provide generic responses to error reports. INSTEAD, ask for specific error messages.
- DON'T guess resource IDs or permissions. INSTEAD, use lookup tools to verify.

Why this works:

Direct. No ambiguity in “DON’T do X.”
Specific. Each item addresses observed behavior, not hypotheticals.
Actionable. Pairing “don’t” with “instead” redirects toward correct approaches.

Issues we struggled to fix through complex prompt adjustments disappeared when explicitly called out in this format. The assistant was guessing at information it didn’t have—we added “DON’T guess how features work, always check the help center first” and the problem vanished.

This section grows organically from real usage. Start with 3-4 items addressing the most pressing issues. Ours has grown to about ten.

Part 4: Guidelines and Boundaries

How the assistant should behave across all interactions: tone, boundaries, handling limitations.

- Address users formally
- When uncertain, state limitations rather than guessing
- If a request falls outside capabilities, explain and offer to connect with support
- Focus on helpful and accurate over verbose

The critical element: how to handle limitations. Every AI has boundaries. Without explicit guidance, assistants make up answers or overstep their role.

Example: our assistant receives questions about municipal regulations that vary by location and change over time. Rather than guess (and provide potentially incorrect information), guidelines instruct it to acknowledge when a question requires specialized knowledge and suggest appropriate resources.

This section doesn’t need to be long—5-6 bullet points covering core principles for behavior across all scenarios.

Why This Works

Separation of concerns. Each section addresses one aspect: context, capabilities, pitfalls, behavior. Adding a new tool only touches the tools section. A new mistake only touches the mistakes section.

Practical over theoretical. The “common mistakes” section evolves from observed failures, not hypothetical edge cases.

Maintainable. Clear structure makes it obvious where new information belongs.

Results after implementing this:

30% more user requests completed without human intervention
60% reduction in incorrect information
25% increase in user satisfaction

The simplicity also makes the prompt accessible to the whole team. Customer support notices a problem? They can suggest a specific addition to the mistakes section without needing to understand “prompt engineering.”

The Takeaway

Effective prompting isn’t about complexity—it’s about clarity and structure. Four sections: context, tools, mistakes, guidelines. Update based on observed failures. Keep each section focused on its specific purpose.

The best prompt isn’t the one that anticipates every scenario. It’s the one that provides clear guidance and evolves easily as needs change.

Role-Based Access Control in Go with Chi

Fri, 21 Mar 2025 00:00:00 GMT

Most RBAC implementations are overcomplicated. Hierarchical roles, conditional permissions, attribute-based controls—useful in some contexts, but overkill for many applications.

This implementation is minimal: roles and permissions as enums, a map between them, and middleware that checks permissions. No external dependencies. The entire system fits in one file.

The RBAC Package

Roles and Permissions

Define roles and permissions as typed string constants:

package rbac
 
type Role string
 
func (r Role) String() string {
	return string(r)
}
 
const (
	RoleAdmin   Role = "admin"
	RoleManager Role = "manager"
)
 
type Permission string
 
func (p Permission) String() string {
	return string(p)
}
 
const (
	PermissionManageAccess      Permission = "manage_access"
	PermissionManageReleaseNote Permission = "manage_release_note"
)

Role-Permission Mapping

// rolePermissions maps roles to their granted permissions
var rolePermissions = map[Role][]Permission{
	RoleAdmin: {
		PermissionManageAccess,
		PermissionManageReleaseNote,
	},
	RoleManager: {
		PermissionManageReleaseNote,
	},
}
 
// HasPermission checks if a role has a specific permission
func (r Role) HasPermission(p Permission) bool {
    permissions, exists := rolePermissions[r]
    if !exists {
        return false
    }
 
    for _, permission := range permissions {
        if permission == p {
            return true
        }
    }
    return false
}

The Middleware

// RequirePermission creates middleware that checks if the user has the required permission
func RequirePermission(permission Permission) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            // Get user role from context (set by auth middleware)
            userRole, ok := r.Context().Value(„userRole“).(Role)
            if !ok {
                http.Error(w, „Unauthorized“, http.StatusUnauthorized)
                return
            }
 
            // Check if the role has the required permission
            if !userRole.HasPermission(permission) {
                http.Error(w, „Forbidden“, http.StatusForbidden)
                return
            }
 
            // User has permission, proceed to the next handler
            next.ServeHTTP(w, r)
        })
    }
}

The middleware extracts the user’s role from the request context (set by auth middleware) and checks if that role has the required permission.

Using It with Chi

Auth Middleware Integration

Your auth middleware needs to add the user’s role to the request context:

// AuthMiddleware authenticates the user and adds their info to the context
func AuthMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Get token from request (e.g., from cookie or Authorization header)
        token := getTokenFromRequest(r)
 
        // Validate token and get user info
        user, err := validateToken(token)
        if err != nil {
            http.Error(w, „Unauthorized“, http.StatusUnauthorized)
            return
        }
 
        // Create a new context with the user role
        ctx := context.WithValue(r.Context(), „userRole“, user.Role)
 
        // Call the next handler with the updated context
        next.ServeHTTP(w, r.WithContext(ctx))
    })
}

Protecting Routes

func SetupRoutes(r *chi.Mux) {
    // Apply authentication middleware to all routes
    r.Use(AuthMiddleware)
 
    // Public routes accessible to all authenticated users
    r.Get(„/release-notes“, handlers.ListReleaseNotes)
    r.Get(„/profile“, handlers.ViewProfile)
 
    // Routes that require specific permissions
    r.Group(func(r chi.Router) {
        // Only users with user management permission can access these routes
        r.Use(rbac.RequirePermission(rbac.PermissionUserManagement))
        r.Get(„/users“, handlers.ListUsers)
        r.Post(„/users“, handlers.CreateUser)
        r.Put(„/users/{id}“, handlers.UpdateUser)
        r.Delete(„/users/{id}“, handlers.DeleteUser)
    })
}

Chi’s router groups let you apply RBAC middleware to entire groups of routes. The permission check happens in middleware, so handlers focus solely on business logic.

Request Flow

Request hits AuthMiddleware → validates token, adds role to context
Request hits RequirePermission middleware → checks role has permission
If permitted, request reaches handler

Clean separation: authentication (who are you?), authorization (what can you do?), business logic (how do we do it?).

Why This Works

Simple. A few dozen lines, no dependencies.
Explicit. Route definitions show required permissions at a glance.
Extensible. New roles/permissions = update constants and map. Middleware unchanged.

The pattern works with any Go router. Roles, permissions, and middleware-based checks are the core concepts—the specific router doesn’t matter.

Rolling Your Own Authentication in Go

Thu, 13 Mar 2025 00:00:00 GMT

“Don’t roll your own auth.”

The advice exists for good reason—authentication mistakes can be catastrophic. But the warning has morphed into something else: a belief that authentication is inherently complex and best left to third-party services.

It isn’t. Most auth libraries require adapting your entire application to their patterns. For standard username/password authentication with sessions, that’s overkill.

The core components are straightforward when broken down. Two resources were particularly useful:

This isn’t about reinventing cryptography. It’s about correctly implementing well-established patterns.

Core Components

An authentication system answers one question: “Are you who you claim to be?” Here’s what that requires:

Password Storage - Bcrypt hashing. Store the hash, never the password.
Secure Random Generation - Unpredictable tokens for sessions and CSRF.
Session Management - Remember authenticated users across requests.
Rate Limiting - Prevent brute force attacks.
CSRF Protection - Ensure requests originate from your application.
Auth Handlers - Registration and login endpoints.
Auth Middleware - Gate access to protected routes.

Each component has a single responsibility. No monolithic auth service to adapt your application around.

Password Storage

Never store passwords in plain text. Use bcrypt.

Unlike general-purpose hashing (SHA-256), bcrypt is intentionally slow. This makes brute-force attacks computationally expensive. The cost factor is adjustable—increase it as hardware gets faster.

package password
 
import (
	"golang.org/x/crypto/bcrypt"
)
 
func HashPassword(password string) (string, error) {
	passwordBytes := []byte(password)
	hashed, err := bcrypt.GenerateFromPassword(passwordBytes, bcrypt.MinCost)
	if err != nil {
		return "", err
	}
	return string(hashed), nil
}
 
func DoPasswordsMatch(hashedPassword, test string) bool {
	if err := bcrypt.CompareHashAndPassword([]byte(hashedPassword), []byte(test)); err != nil {
		return false
	}
	return true
}

HashPassword creates the hash at registration. DoPasswordsMatch verifies it at login.

Secure Random Generation

Session tokens need to be unpredictable. Use crypto/rand, not math/rand. Standard random functions are predictable if you know the seed—cryptographically secure generators aren’t.

package random
 
import (
	"crypto/rand"
	"crypto/sha256"
	"encoding/base32"
	"encoding/hex"
)
 
func CreateRandomToken() string {
	bytes := make([]byte, 15)
	rand.Read(bytes)
	token := base32.StdEncoding.EncodeToString(bytes)
	return token
}
 
func EncodeToken(token string) string {
	byteId := sha256.Sum256([]byte(token))
	return hex.EncodeToString(byteId[:])
}

CreateRandomToken generates random bytes and base32-encodes them. EncodeToken hashes the token before storage—if someone gets your database, they can’t use the stored session IDs directly.

Session Management

HTTP is stateless. Sessions maintain state across requests by associating a token with user data on the server.

The flow:

User logs in successfully
Server generates a random session token
Server stores user info associated with that token
Server sends the token as a cookie
Client includes the token in subsequent requests
Server validates and retrieves the associated user

package session
 
import (
	"your/project/random"
)
 
func (s *service) CreateToken() string {
    // create a random token through our random package
	token := random.CreateRandomToken()
	return token
}
 
func (s *service) CreateSession(token string, userId uuid.UUID) error {
    // create a new session record from a random token and user id
	sessionId := hashToken(token)
	expiresAt := calcNextExpiry()
	session := Session{ExternalID: sessionId, ExpiresAt: expiresAt, UserID: userId}
	return s.repository.Save(&session)
}
 
func (s *service) ValidateSession(token string) (*Session, error) {
    // returns a session from a token, if there is any
    // and resets the expiry if successful
	sessionId := hashToken(token)
	session, err := s.repository.FindByExternalId(sessionId)
	if err != nil {
		return nil, err
	}
	if sessionIsExpired(session) {
		if err := s.repository.Delete(session.ID); err != nil {
			return nil, err
		}
		return nil, s.repository.db.ErrRecordNotFound
	}
	session.ExpiresAt = calcNextExpiry()
	if err := s.repository.Save(session); err != nil {
		return nil, err
	}
	return session, nil
}

Key details:

Hash the token before storing it. If your database leaks, attackers can’t use the session IDs directly.
Sessions expire. Reset expiry on each request so active users stay logged in.
Storage can be database, Redis, or in-memory (though in-memory won’t survive restarts).

Rate Limiting

Without rate limiting, attackers can try thousands of passwords per minute. Rate limiting caps attempts per user/IP within a timeframe.

This implementation uses a token bucket algorithm—each identifier gets a bucket that refills over time. Each attempt consumes from the bucket. Empty bucket means wait.

package ratelimit
 
import (
	"errors"
	"math"
	"sync"
	"time"
)
 
 
type Bucket struct {
	count      float64
	refilledAt int64
	mu         sync.Mutex
}
 
func (b *Bucket) consume(cost, maxValue float64, refillIntervalMillis int64) bool {
	// lock the bucket
	b.mu.Lock()
	defer b.mu.Unlock()
	// calculate refill until now & refill bucket
	now := time.Now().UTC().UnixMilli()
	refill := float64(now-b.refilledAt) / float64(refillIntervalMillis) * maxValue
	b.count = math.Min(b.count+refill, maxValue)
	b.refilledAt = now
 
	// deduct cost if possible
	if b.count >= cost {
		b.count -= cost
		return true
	}
	return false
}

Each email address and IP address gets its own bucket. Check capacity by attempting to deduct:

type TokenBucketRateLimit struct {
	refillIntervalMillis int64
	maxValue             float64
	buckets              map[string]*Bucket
}
 
func (tbr *TokenBucketRateLimit) Deduct(bucketLabel string, cost float64) error {
	now := time.Now().UTC().UnixMilli()
	// check if the label already has a bucket
	bucket, ok := tbr.buckets[bucketLabel]
	if !ok {
        // no bucket yet -> create one, then deduct again
		bucket := Bucket{count: tbr.maxValue, refilledAt: now}
		tbr.buckets[bucketLabel] = &bucket
		return tbr.Deduct(bucketLabel, cost)
	}
    // bucket exists -> consume
	if ok := bucket.consume(cost, tbr.maxValue, tbr.refillIntervalMillis); !ok {
		return errors.New("rate limit reached")
	}
	return nil
}

The bucket approach allows brief bursts (mistyped passwords) while blocking sustained attacks.

CSRF Protection

CSRF attacks trick authenticated users into submitting requests they didn’t intend. Your browser automatically includes session cookies, so a malicious site can submit forms to your app on behalf of logged-in users.

The common fix is CSRF tokens in forms. A simpler alternative: check the Origin header.

if request.Method != "GET" {
    originHeader := r.Header.Get("Origin")
    if originHeader != "https://yourapp.com" {
        w.WriteHeader(403)
        return
    }
}

This works when state-changing operations are restricted to POST/PUT/PATCH. For more complex scenarios, use proper CSRF tokens.

These handlers bring all the components together.

package handler
 
// !! logging, error handling and validation
// left out for brevity and focus
 
type registerForm struct {
	Email           string
	Password        string
	ConfirmPassword string
}
 
// allow 5 requests per minute
var registerRateLimiter = ratelimit.New(60, 5)
 
func (h *Handler) HandleRegister(w http.ResponseWriter, r *http.Request) {
	userService := user.NewService(*user.NewRepository(h.DB))
	sessionService := session.NewService(*session.NewRepository(h.DB))
 
	if err := r.ParseForm(); err != nil {
		http.Error(w, "Error parsing form", http.StatusBadRequest)
		return
	}
 
	req := registerForm{
		Email:           r.FormValue("email"),
		Password:        r.FormValue("password"),
		ConfirmPassword: r.FormValue("confirm"),
	}
 
    // check rate limit
	if err := registerRateLimiter.Deduct(req.Email, 1); err != nil {
		http.Error(w, "Too many requests. Please try again later.", http.StatusTooManyRequests)
		return
	}
	if err := registerRateLimiter.Deduct(req.OrgName, 1); err != nil {
		http.Error(w, "Too many requests. Please try again later.", http.StatusTooManyRequests)
		return
	}
 
    // check password guidelines
	if err := password.IsValidPassword(req.Password); err != nil {
		http.Error(w, err.Error(), http.StatusBadRequest)
		return
	}
 
    // check if passwords match
	if req.Password != req.ConfirmPassword {
		http.Error(w, "Passwords do not match", http.StatusBadRequest)
		return
	}
 
	// check if user already exists
	existing, err := userService.GetByEmail(req.Email)
	if err != nil && !errors.Is(err, h.DB.ErrRecordNotFound) {
		http.Error(w, "Error creating user", http.StatusBadRequest)
		return
	}
	if existing != nil {
		http.Error(w, "User already exists", http.StatusBadRequest)
		return
	}
 
	// create user
    pwHash := password.HashPassword(req.Password)
	user, err := userService.Create(req.Email, pwHash)
	if err != nil {
		http.Error(w, "Error creating user", http.StatusInternalServerError)
		return
	}
 
    // create session
	token := sessionService.CreateToken()
	if err := sessionService.Create(token, user.ID); err != nil {
		http.Error(w, "Error creating session", http.StatusInternalServerError)
		return
	}
 
	w.WriteHeader(http.StatusCreated)
}

Registration validates input, checks for existing users, hashes the password, creates the user, and starts a session.

package handler
 
// !! logging, error handling and validation
// left out for brevity and focus
 
type loginForm struct {
	Email    string
	Password string
}
 
var loginRateLimiter = ratelimit.New(60, 5)
 
func (h *Handler) HandleLogin(w http.ResponseWriter, r *http.Request) {
	userService := user.NewService(*user.NewRepository(h.DB))
	sessionService := session.NewService(*session.NewRepository(h.DB))
 
	if err := r.ParseForm(); err != nil {
		http.Error(w, "Error parsing form", http.StatusBadRequest)
		return
	}
 
	req := loginForm{
		Email:    r.FormValue("email"),
		Password: r.FormValue("password"),
	}
 
	// Check rate limit
	if err := loginRateLimiter.Deduct(req.Email, 1); err != nil {
		http.Error(w, "Too many login attempts. Please try again later.", http.StatusTooManyRequests)
		return
	}
 
	// validate credentials
	user, err := userService.GetByEmail(req.Email)
	if err != nil {
		if errors.Is(err, h.DB.ErrRecordNotFound) {
			http.Error(w, "Wrong credentials", http.StatusUnauthorized)
			return
		}
		http.Error(w, "Error accessing user", http.StatusInternalServerError)
		return
	}
	if match := password.DoPasswordsMatch(user.Password, req.Password); !match {
		http.Error(w, "Wrong credentials", http.StatusUnauthorized)
		return
	}
 
	// create session
	token := sessionService.CreateToken()
	if err := sessionService.Create(token, user.ID); err != nil {
		http.Error(w, "Error creating session", http.StatusInternalServerError)
		return
	}
 
	// set session cookie
	http.SetCookie(w, &http.Cookie{
		Name:     session.AuthCookieName,
		Value:    token,
		HttpOnly: true,
		SameSite: http.SameSiteStrictMode,
	})
 
	w.WriteHeader(http.StatusOK)
	return
}

Note the error handling: don’t tell users whether the email exists or the password is wrong—just “wrong credentials.” This prevents username enumeration.

Auth Middleware

Middleware gates access to protected routes. Extract the session token, validate it, attach user info to the request context.

package mw
 
type contextKey string
 
const (
	SessionIdKey          contextKey = "sessionId"
	UserIDKey             contextKey = "userId"
)
 
func (h *Handler) Authenticate(next http.Handler) http.Handler {
	sessionService := session.NewService(*session.NewRepository(h.DB))
	userService := user.NewService(*user.NewRepository(h.DB))
 
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		// get session cookie
		cookie, err := r.Cookie("<cookie-key>")
		if err != nil {
			http.Redirect(w, r, "/login", http.StatusSeeOther)
			return
		}
 
		authToken := cookie.Value
 
		session, err := sessionService.ValidateSession(authToken)
		if err != nil {
			if errors.Is(err, h.DB.ErrRecordNotFound) {
				http.Redirect(w, r, "/login", http.StatusSeeOther)
				return
			}
			http.Error(w, "Error validating session", http.StatusInternalServerError)
			return
		}
 
		user, err := userService.Get(session.UserID)
		if err != nil {
			http.Error(w, "Error getting user", http.StatusInternalServerError)
			return
		}
 
		ctx := r.Context()
		ctx = context.WithValue(ctx, SessionIdKey, session.ID.String())
		ctx = context.WithValue(ctx, UserIDKey, session.UserID.String())
 
		r = r.WithContext(ctx)
		next.ServeHTTP(w, r)
	})
}

Apply it to any route:

r.With(mwHandler.Authenticate).Route("/dashboard", func(r chi.Router) {
    r.Get("/", handler.HandleDashboard)
})

Authentication tells you who the user is. Authorization (checking roles/permissions) tells you what they can do—a separate concern.

When to Roll Your Own

Roll your own when:

Standard username/password with sessions is enough
You want full control and minimal dependencies
You have time to implement it correctly

Use an existing solution when:

You need MFA, social logins, or enterprise SSO
You’re on a tight deadline
Security expertise on the team is limited

The Tradeoffs

Rolling your own means owning security updates and edge cases (password resets, account recovery, session invalidation across devices). Third-party auth means adapting to their patterns and depending on their uptime.

Neither choice is universally right. For standard auth requirements, building it yourself is simpler than the conventional wisdom suggests. The complexity isn’t in the implementation—it’s in not skipping steps.

From NextJS to Go

Sun, 09 Mar 2025 00:00:00 GMT

I built a side project—in-product release notes—with NextJS and Supabase. Development was smooth. Production was not.

Pages that loaded instantly in dev took seconds in production. Before I’d even launched, I hit Vercel’s image processing limits and Supabase’s edge function ceiling. The “lightweight” side project was heading toward €50/month in hosting costs.

So I rewrote it: Go with HTML templates, HTMX, and Alpine, hosted on a €5/month VPS. I also built authentication from scratch.

This wasn’t because NextJS and Supabase are bad—they’re not. They just weren’t right for this project. The performance issues and unexpected costs were the practical reasons. Curiosity about Go was the other reason.

Learning Go

Resources that worked well, in order:

Go’s directness was immediately apparent. Where JavaScript offers a dozen ways to do the same thing, Go typically presents one or two. This felt constraining at first, then liberating. The emphasis on “idiomatic” patterns—one right way to do things—made the language easy to pick up.

Rolling my own auth

Supabase Auth’s abstractions made debugging painful. So I built authentication from scratch.

Two resources made this straightforward:

The implementation: a session service, authentication middleware, rate limiting, and CSRF protection. More work than using a provider, but when something breaks, I know exactly where to look. I wrote more about this in Rolling Your Own Authentication in Go.

Project structure

In NextJS, client and server code blur together. In Go, the boundaries are clear: receive request, process, return response.

Standard structure:

Handlers (HTTP requests)
Services (business logic)
Repositories (database)
Models (data structures)
Templates (HTML)

Libraries: Chi for routing, validator for validation, Gorilla schema for form handling.

A consistent pattern emerged for handlers: parse → decode → validate → business logic → response.

The hardest part was replacing React’s component model with Go templates. No state management, no component abstraction—just template partials and inheritance. It forced me to think differently about UI structure.

What surprised me

Speed. The first time I loaded the Go version, I thought it was broken—pages appeared so fast I assumed they hadn’t loaded. A dynamic app with auth, database queries, and business logic, responding instantly. I knew static sites could be fast. I didn’t expect this from a full application.

Simplicity. In NextJS, I was constantly thinking about client vs server components, server actions, data fetching strategies, client/server state interplay. In Go: receive request, process, return response. Debugging got easier. Development got faster.

AI compatibility. Go’s consistency made AI tools significantly more reliable. Point an LLM at a Go handler, ask it to create another one in the same style—it works almost every time. The lack of abstraction layers means less for the model to misunderstand.

On abstractions

Implementing features in Go—routing → parsing → validation → business logic → templates—made something clear: this is what NextJS does under the hood, just with layers of abstraction on top.

Client components, server components, server actions, hooks, context, reducers—they’re abstractions over request-response. Powerful for large teams building complex applications. Overkill for many projects.

Go templates have real limitations. No component model, no state management. Some interactions are harder to build. But the constraint forced me to understand web fundamentals more deeply—browser events, form submissions, AJAX—knowledge that transfers regardless of framework.

Not everything in Go-land is perfect. GORM (the ORM I used) occasionally behaved in surprising ways. Sometimes I wished I’d just written raw SQL.

Will I use React again? Yes—it’s the right tool for certain projects. But I’ll be more critical about whether its abstractions solve problems I actually have.

The result

The Go version is faster, cheaper (€5/month vs €50+), more maintainable, and easier to debug. When something breaks, I know where to look.

The migration started as a practical fix for performance and cost problems. It became a useful reminder that simpler tools often work better—and that understanding what happens beneath the abstractions makes you better at using them.