A few weeks ago, Christoph Janz — one of the most respected VCs in Europe — documented his 13-day experiment building a “Knowledge Hub” for his firm using vibe coding. The result: 43,800 lines of Python, none written by a human. It pulled data from Gmail, Slack, Drive, and CRM systems, vectorized it, stored it, and made it queryable through an LLM.
He claimed that it worked. Sort of. You could ask, “How is Company X doing?” and get a reasonable answer with sources.
His honest conclusion was disarmingly straightforward: “My guess is that we’ll eventually end up using a product built by someone who knows what they’re doing.”
On his Linkedin post, my friend Ernests Stals (founder of Starwatcher) — who has been building and writing deeply about knowledge systems and published essays that first pulled me seriously into this space last year, responded Christoph’s post with a quote that has stayed with me ever since:
“If you think in terms of ‘sources,’ you’ll build a system of sources. Knowledge isn’t static. It evolves. Aggregating and retrieving information is not the same as understanding it.”
I think both of them are right. They’re looking at the same problem from different ends. But the gap between them — the thing neither quite named — is what this essay is about.
Christoph built a system that retrieves. Ernests described a system that creates. The missing layer is encoded judgment. Not data, not answers — the reasoning itself. The score is just a handle. The reasoning chain is the knowledge.
I didn’t arrive at this conclusion abstractly. I built something to test it.
The Experiment
About six months ago, I couldn’t hold back from tinkering with Claude Code. I’d been playing with LLMs for a while, but something about working directly in the terminal — giving an AI agent instructions, watching it build, breaking things, rebuilding — got under my skin.
During that tinkering phase, I came across markdown files as a way to give persistent context to an AI agent. Not code. Not databases. Just structured text files that tell the system how to think. I started experimenting with frameworks I’d borrowed from Ernests — “How We Think,” “How We Work,” “How We Decide” — and layered them into a set of markdowns that shaped how my Claude Code assistant engaged with me.
Then something unexpected happened. It morphed. What started as a note-taking scaffold turned into a system that started to call me out. I’d feed it a decision I was mulling over, and instead of agreeing or summarizing, it would push back. “You’ve tagged this as observed, but where’s the evidence?” or “You score at a confidence that is 0.8 — what is the evidence given that the current one is more inferred more than observed?”
At first, my instinct was to tell it to f**k off. :) Genuinely. Nobody enjoys being questioned by their own tool. But after that initial ego moment — and after many rounds of debates, discussions, poking at assumptions and tracking what actually happened — something shifted. I started running it against a quarter’s worth of real decisions.
The results were uncomfortable.
After dissecting the decisions I’d made and how confident I was in each, a pattern emerged. Where my gut screamed 0.8 — obvious, no need to think harder — reality resolved closer to 0.5. Things I’d claimed as observed turned out to be more inferred than not. The confidence wasn’t earned. It was inherited from habit.
We audit expenses down to the cent, yet rarely audit conviction at all.
Everyone’s Favorite Black Box
“Judgment” is one of the most frequently invoked and least operationalized words in business. In venture capital, we invest in the “judgment” of founders. In startups, product decisions supposedly require “good judgment.” In AI discussions, we claim machines can do everything except judgment.
Yet ask anyone to decompose what that word actually means — to name its components, describe how you would test them, or explain how they might be improved deliberately — and the conversation stalls.
Judgment stays mystical because it is also where ego lives. It’s the last defensible moat of the individual. To examine it closely feels like submitting your identity to scrutiny.
But what happens if you treat judgment not as a personality trait, but as a system? Not to diminish it, but to instrument it.
What Happens When You Try
Three things emerged from the experiment.
One: Judgment decomposes. Not neatly, but meaningfully.
Two: Different layers tolerate different degrees of formalization.
Three: The formalized layers protect space for the irreducible ones.
Here’s the stack I ended up with — six layers, from most to least formalizable.
Epistemic Hygiene
What it is: Every claim gets tagged as observed, inferred, or hypothesized.
Back in August, I wrote about the distinction between facts and wishful thinking — how easily we confuse what we know with what we want to be true. This essay turned out to be the seed of this entire system.
In practice, it works like this. Before any decision, I force myself to list the core assumptions and label each one. “We know this because we saw it” versus “we believe this because it feels right” versus “we’re guessing and should say so.” The exercise sounds trivial until you actually do it. In one case, I listed eight assumptions behind a strategic move and discovered that two were untested guesses I’d been treating as facts for weeks. One of them was load-bearing — the entire thesis rested on something nobody had verified.
Most strategic documents are elegant structures built on unlabeled assumptions. The fix isn’t more data. It’s honest labeling of what you already have.
Decision Mechanics
What it is: Reversibility gates, structured scoring, and pre-written kill signals.
Before committing to a path, I define the conditions that would terminate it. Not “reconsider if X happens.” Kill if X happens. On more than one occasion, a kill signal triggered weeks later, and a past, less emotionally invested version of myself made the hard call before sunk costs could pile up.
Calibration
What it is: Scoring applied to your own confidence levels.
This is where the ego takes the hardest hit. My first calibration review was sobering. When conviction was decomposed into factors — market timing, execution risk, regulatory uncertainty, team dynamics — it became obvious that I was least calibrated in exactly the domain I had assumed was my strength.
Without measurement, experience quietly turns into mythology.
Taste and Discernment
What it is: Partially formalizable. Running parallel reasoning chains across options, each scored on identical dimensions, to expose distortions.
On one occasion, the option I would have chosen instinctively scored lowest once examined systematically. The eventual winner wasn’t the most clever. It was the simplest. The system didn’t replace taste — it revealed where taste had drifted.
Moral Clarity
What it is: Fail-closed defaults and escalation triggers. Tolerates structure only to a point. When two legitimate values collide, no model resolves the tension.
Vision
What it is: Cross-domain pattern recognition and the conviction that something matters before data confirms it. Resists formalization almost entirely — and that resistance is not a flaw. It’s the boundary condition that gives the rest of the stack meaning.
Where This Gets Concrete
The stack above might still sound abstract. Let me make it specific.
Two analysts at a VC fund evaluate the same Series A in a digital therapeutics company. Both have the same data room, the same clinical evidence, the same management calls. Both recommend investing. Both feel confident.
But when you surface their scoring weights, the picture shifts.
Analyst A weights team execution at 5 out of 5, regulatory pathway at 3 out of 5. Analyst B flips it — team at 3, regulatory at 5. The disagreement isn’t about the company. It’s about which risk each person is unconsciously discounting.
Analyst A once worked with a strong team that navigated regulatory hell and came out fine. So she trusts teams and discounts regulatory complexity. Analyst B watched a weak team collapse despite a clean regulatory path. So he trusts the pathway and discounts execution risk.
Neither is wrong. Both are under-examined.
Without visible weights, this disagreement stays at the level of opinion. Whoever has more seniority — or more conviction in their voice — wins. With visible weights, the conversation changes entirely. Instead of arguing about conclusions, participants examine why one factor was weighted more heavily, and whether that weighting is justified by evidence or by autobiography.
This is what I mean by encoding judgment. Not replacing intuition with spreadsheets. Making the invisible visible so it can be examined, challenged, and improved.
Where It Breaks
The system catches maybe 80% of decision errors — the ones driven by laziness, ego, or sunk costs. The remaining 20% resist instrumentation.
One decision I faced cleared every threshold. Confidence was high. No kill signals had triggered. The structured evaluation supported proceeding. Yet something felt off. I overrode the system and paused.
In retrospect, that hesitation was correct. I still can’t fully articulate why.
This is the risk of over-formalization: following the system when the system is wrong. A scaffold can become a cage if you forget it was meant to support judgment, not replace it. The system can’t tell you when to override it — and that limitation is precisely what keeps the human layer necessary.
So What?
Christoph built a system that retrieves. Ernests argued that retrieval alone isn’t knowledge. My thinking is that the missing layer between their positions is encoded reasoning.
I think the next generation of knowledge systems won’t simply store documents or surface answers. They’ll record how decisions were made — which assumptions were labeled, what confidence was declared, what conditions would terminate a path, and what ultimately happened. In other words, they’ll store judgment.
But here’s what I didn’t expect — the personal side effect.
Over the past few months, I’ve noticed something shifting. Less anxiety around decisions. More clarity about what actually matters versus what’s just noise. A kind of selective hearing that I didn’t have before. When you operate in a world where everything moves fast and every headline screams for attention, it’s easy to get overwhelmed. Most of us cope with that speed by either freezing or reacting. This system gave me a third option: examine, label, decide, and move on.
I can’t claim it’s perfect. But I can say I’m making fewer decisions I regret and spending less time second-guessing the ones I’ve made. That alone has been worth the experiment.
Judgment used to exist for me as a black box. Now it looks more like a stack. Some layers can and should be codified. Others resist structure, and that resistance is itself informative. The frontier isn’t a contest between artificial and human judgment — it’s a question of which layers can be systematized and which remain irreducibly personal. More of it can be instrumented than I once assumed. Just not the parts that make it uniquely human.
The tooling is deliberately simple: markdown scaffolds, a Claude Code orchestrator, and the discipline to use them against live decisions. I’ll keep you posted on how the experiment evolves.
S.