So what is context engineering?

It's kind of like de-abstracting the abstractions that have been layered on top of RAG, memory, and agentic history. At the end of the day, they're all different ways to pass tokens into a model.

What is a smart zone, and what is a dumb zone?

The less context window you use, the better outcomes you'll get, always.

A new paradigm that is spreading is loop engineering. What do you think is bad about it?

The problem with loops is, like, at a certain point, you're going to generate so much code that you can't read it anymore. We built a lights-off software factory in July of 2025, and by November, we had shut it down.

Can we talk about what you mean by "token harder" and "token smarter"?

I'm in a group chat called hyperengineering, and it's all, like, people trying to max out their cloud subscriptions. That's my idea of "token harder," and the goal is—

What happens when you let AI agents ship code for months and no developer reads a single line? Today's guest tried exactly that. He built a lights-off software factory and, four months later, he had no choice but to shut it down as things just stopped working.

Dex Horthy is the founder of Human Layer and the person who coined the term "context engineering" days before Andrej Karpathy and Tobi Lütke made it famous. He spent the last two years talking to hundreds of AI engineers about what actually works when you build with LLMs, and is testing the most extreme ideas with his own team.

In today's conversation, we discuss context engineering, what it is, and the physics of context windows, including what the "dumb zone" is; loop engineering, from the Ralph Wiggum technique to the slow loops that Dex's team runs every night to wake up to code cleanup PRs; the rise of software factories, from a NATO conference in 1968 through DevOps to today's agentic factories; spec-driven development, and why specs always drift from the code itself, and many more.

If you want to understand increasingly important concepts like context engineering and harness engineering, or want to know how far you can push the "let agents build everything" idea from someone who pushed it further than almost anyone, then this episode is for you.

This episode is presented by Antithesis. If you work with agents, your job is no longer just writing code. It's specifying and testing it, and Antithesis is the most effective method of verifying agentic code today.

Today's episode is brought to you by Buildkite, the CI platform trusted by OpenAI, Anthropic, Cursor, Nvidia, Uber, Canva, and more. Today, we're talking about pushing the right context into models so that they write better code. Right after that starts working, your agents will write more code—a lot more. Trusting that code avalanche is where many teams face a challenge today. Every change that an agent makes still has to be built, tested, and proven safe before it ships. "Worked on my machine" is not enough.

So, you obviously need CI. But when agents are pushing 5, 10, or 50 times the commit volume into your pipelines, faster CI runners won't save you. Shaving 30 seconds off a single build is meaningless when the queue is 100-plus jobs deep. What you really want is a CI system that gets faster as the volume grows, and CI that offers instant parallelization to give you unlimited concurrency and to intelligently route changes at runtime. This is what Buildkite does and why global software leaders continue to rely on it.

The same architecture that absorbed the scale of Shopify and Uber a decade ago now runs about 1.4 billion job minutes a week across Cursor, Meta, and Snowflake. While the rest of the CI world is cracking under the weight of rearchitecting their platforms, Buildkite continues to reliably grow. Agents running on your infrastructure or Buildkite's—any cloud, any chip, your secrets, your scale—every artifact and log is captured. So when something fails, either you or your agents have immediate insight into why.

As you're ensuring the context you'll give to your agents, think about how you'll verify what they hand back. If your system is buckling under the increased volume, head to buildkite.com/pragmatic for a 30-day all-access trial, no credit card, and an actual human engineer on standby. His name's Ola, and he's very helpful.

So, Dex, welcome to the podcast.

Super stoked to be here, dude.

Before we get into some of the context engineering and some of the more spicy stuff as well, how did you get into tech? How did you fall in love with computers?

Oh, man. So, I was doing undergrad as a physics major, and I realized that I didn't like academia. There are basically like two or three paths out of physics: basically, you go get a PhD, or you go into finance, or you go do programming. At that time—this was 2011, 2012, when I was in the middle of undergrad and deciding what to do—I had done an internship when I was in high school. I was working with NASA researchers at the Jet Propulsion Lab in California. They had just gotten this really high-fidelity, like the most fine-grained dataset of altitudes, like a topographical map of the south pole of the moon.

And the south pole of the moon is really interesting because some of the craters there are so deep because of the angle it has. It got hit by meteor storms like no other part of the moon, so there are very deep craters that have never seen sunlight.

And so there's frozen liquid water in there from the formation of the moon. Scientists were really interested in getting down there and exploring. We had this really fine-grained map and it's like, "Okay, cool. Let's build software so that I have point A to point B. I know the limitations of my rover: maximum incline up is this, maximum incline down is that. Find a path from point A to point B that doesn't break those rules of the incline."

So I was 17. I had never cracked a CS textbook, so I basically wrote a really naive, bad version of Dijkstra's algorithm for pathfinding. When I was in college, I was like, "I don't know if I want to do the academics thing, but I really enjoyed programming back in the day." So I decided to go get half of a CS minor and then started working on an API platform team at a software company in Chicago, and—

Sprout Social, right?

Yes, and basically never went back.

Yeah. And then where did you go from there? Where did you pick up the parts of the trade? Because very early on—your first job—that's not really common. You were doing platform engineering back more than a decade ago.

From that point, it took me about two or three months to notice that the most valuable work being done in the company was being done by—of course it's obvious—the first couple of engineers who know everything and understand where everything is. You spend a day on a support ticket from a customer, and they solve it in five minutes, but you have to solve it so you learn and whatever. And I realized the most valuable people in the company were the people who were building the developer platform, CI/CD, sandbox environments, and preview stuff. So that was my first step into the journey, and I've basically been obsessed with software factories since about three or six months into my first job.

We talk about software factories now, but you were talking about software factories back then. So you were starting to already think that this is how we can produce better software inside. This is a pre-AI world, right?

Well, and I'm always surprised—there's a huge class of developers that say, "I don't want to work on CI/CD. I hate CI/CD." I'm like, "Really?" Because building the thing that builds the thing, and building the thing that builds the thing that builds the thing... as software engineers, we're lazy. We want to do the most high-leverage thing that makes our job easier. If we can build a thing that helps us build a thing that helps us move faster, then that's the best use of my time as a lazy engineer.

And then you went to another startup, Aspiration?

Aspiration, yeah.

Aspiration, also platform engineering.

Yeah. I was brought in, and then three months into the job, the VP of engineering who hired me quit or got fired. I don't know, there was some drama about it—I probably shouldn't talk about it. I was there for about a year and was kind of like acting CTO for a while. I hired a couple of people, helped hire the new VP of engineering, but then I was out of there. I don't think I'll ever do consumer again. I think I'm actually a B2B guy.

Good to know. And then you went to Replicated, where you spent a good, solid four years, and went from engineer to forward-deployed engineer to product manager.

Yeah, I did core engineering for like two years. We were building a container orchestrator before Kubernetes, before Docker Swarm was really a thing. We built our own orchestrator. The founders had this vision that, "Oh, Docker is going to make it much easier to ship on-prem software." And when I say on-prem, I don't mean literally like a rack in a colocation facility. It's more like, "Hey, look, bring the app to where the data is, rather than sending the data up to some cloud vendor."

And Docker makes it much easier to package up apps and move them around. So they had this thesis that, basically, you could build a platform where you get the experience of using GitHub Enterprise—which is like you install it and it has this admin panel, but then you just get GitHub running in your data center and your code never has to leave your data center. Suddenly, you could build a generic SaaS where everybody could have that.

So I did two years as an engineer there. And then we parted ways with our head of sales. Honestly, I was having a lot of arguments about the software factory with our CTO, and it was almost like a "too many cooks in the kitchen" kind of thing. I'm sure many listeners have had this experience of like, "Well, yeah, I know I have these tickets to build, but CI sucks. I've got to fix CI because it's too slow or there are too many different builds and it's always breaking. I'm going to fix that and then I'm going to do the tickets." And in the end, it's just like, "Dex, I need you to stop fixing the build pipeline and do the tickets I gave you." I'm sure you've had this experience, perhaps.

Yeah. And was this what led you to forward-deployed engineering?

Yeah. So, I really loved our customers. Our customers were HashiCorp, DataStax, Puppet—all these really cool engineering brands like Travis CI, CircleCI. I was like, "Yeah, I actually love working with our customers. Our customers were awesome." It was a great way to get in the trenches with a lot of really good engineers who were solving the hardest problem at the company, which is: how do we take this three-to-five-year-old SaaS platform and package it all up so that someone who knows nothing about our architecture can run it reliably in their own AWS VPC, in their own on-prem data center, whatever it was?

So I spent time as our first customer-facing engineer. In about three months, I met with every customer that was in the pipeline but wasn't moving sales-wise, and we closed something like 12 deals in three months. The CEO was like, "Holy crap, Dex. The investors are taking my calls again. I know you want to get back to coding, but I need you to go hire three people and build this team out because I think you might have been born for this."

Wow.

Yeah, so I did that for about four years, built that org to like 25 people, and then ZIRP ended—or happened—and it got a lot smaller. We kind of realized like, "Hey, we have a product that's pretty good." We had been solving things the way lots of early startups do, like, "Okay, there are some usability issues. We'll get a bunch of smart people and throw them in the trenches with our customers." That's great for sales and great for retention, but we realized the margins on that aren't good enough. So we basically were like, "Cool, we actually just need to make the product way more usable, do a more PLG-shaped thing, make it product-led growth."

Product-led growth.

Make it a little more self-service so you don't need an expert to teach you how to use it. And I was like, "Cool. If that's the most important thing, then I want to go be a product manager because I have tons of opinions. I've now spent four years in the trenches with our customers, and I have a laundry list of roadmap things that I think would make the product way easier to use, adopt, implement, and deploy."

And now you went the full arc; you went towards the dark side.

Exactly. Yeah, I did. I was like, "This is going to kill my street cred, isn't it?" But I was really glad. I think a lot of engineers are afraid that if they go do a customer-facing thing, they lose all their credibility. And yes, I wasn't coding for 10 hours a day; I was coding for like three or four hours on a Saturday for fun. But we were helping people build YAML, we were building CLIs, and we owned a lot of the tooling that customers used, but it was the last-mile delivery side of it, not the core platform.

On a more personal note, I had spent most of my 20s feeling a little bit introverted, a little bit socially awkward, which is what a lot of engineers, I'm sure, experience. I talked to my uncle, who is a music producer. He used to work with Randy Newman and a bunch of really famous musicians.

Oh, wow.

Yeah, this guy Mitchell Froom. I was sitting at dinner with him at some point when I was—I think it was when I was still in undergrad—and he gave me this lecture. He was basically like, "If you want to be really good at something, you have to make it the only thing you do. The guy playing guitar nights and weekends trying to get his band off the ground will probably never achieve greatness. The people who become great are the people who basically make it so that if they don't play guitar, they don't eat. You go and you sit on the street all day, and you play for 14 hours a day. That's the only way to become great."

So I said, "Okay, instead of trying to read self-help books about how to be less introverted and less socially awkward, what if I just made it my freaking job to just talk to people, make friends, help people, and solve their problems?" I think it worked out. I recommend it. I think everyone should spend a year or two at least doing something customer-facing.

Did you do this because you felt being introverted was holding you back, or what? I know you got the motivation from the musician story—I get that on one part—but what was it that made you say, "I'm going to do a customer-facing thing"? Because clearly, you were pretty great at writing code by that point. You could argue you were doing it night and day. So why did you think customer-facing would get this introvert off of you? Did you feel it was holding you back, or did you just want to be good at it?

It was just kind of a thing that was interfering with my general life satisfaction.

And I'm also not a very Type A person; I'm very disorganized. I don't know if people call it, "Okay, I have ADHD now," and that's why I can run 30 threads in parallel or whatever it is. But I was really bad at email, calendars, and spreadsheets—I just didn't care about them and didn't understand them. Another side effect of this was that it forced me to be organized and keep a lot of things going. So there are weird benefits you get from stepping outside your comfort zone and learning industrial disciplines that are separate from what you've been doing. The opportunity presented itself, and I was like, "Oh, I like working. I'll try this for a little bit." It started going really well, and I was like, "Cool, let's see how far this thread goes."

And then afterwards, you're now in your second startup. You became a founder, and you also got involved in AI pretty early, even before it was so obvious how it would change how we develop software, right?

Well, I would say I was later than I could have been, because we started the company—me and a buddy in Chicago started a company in the data engineering space in about November 2020. We decided in like August of 2020—

This is Metalytics.

Metalytics. Technically, it's still the same company as Human Layer; we just pivoted the mission. But the advice I got from every angel investor—people who just knew CTOs I'd worked for before and stuff—they were just like, "Look, you're hitting a lot of headwinds." I don't know if you know the whole dbt data engineering fever, that whole arc where there was this huge party and tons of investor money going into all these different companies, and then by 2021, 2022, there was the ZIRP thing and just this general realization that the TAM for those sorts of tools is not as big as everyone thought.

Yes, the Total Addressable Market for those sorts of tools was not quite as big as we all thought it was.

So it was a hard place to raise money. It was a hard place to get customers.

Yeah. And then I met you while you were at Human Layer in SF at an event. We actually chatted afterwards. By that point—this was about a year ago—you started to have some really strong opinions on using AI. And one of them was this now-famous "12-Factor Agents" manifesto.

Are we calling it a manifesto now?

I'm calling it a manifesto. It's a manifesto. Let's talk about this. This was 12 engineering principles to build reliable, production-ready apps. How did you come up with this, and maybe we can talk about some of them?

Yeah. So, I'll go back to around August. The co-founder I was working with kind of burned out and left. We were on good terms; it was very mutual. I decided to start messing with AI stuff and was building AI agents. What was really in vogue right then was LangChain, CrewAI—these agent frameworks. It seemed like there was a ton of momentum. You go into the CrewAI Discord, there are 10,000 people. It's like, "Okay, this feels like the right shape, and there's clearly this ecosystem."

You go into every single one of those projects and they have a ChromaDB plugin, they have a Composio plugin. There's clearly a shared interface that everybody is building for. I said, "Okay, what's missing from all of this?" The agents can call tools, but it's really hard to control which tools they call. If it's a chatbot, obviously, you can show "approve/deny" in the UI of your application. But I was kind of obsessed with what I would call outer-loop agents or proactive agents. These are agents that run in the background, triggered by events. OpenDevin is basically the biggest manifestation of this—you have a heartbeat, it wakes up, it sees if there's any work to do, and it tries to do stuff. My thought was, "I'm not going to trust that agent to do anything meaningful."

If I can't get a Slack message or an iMessage or something when it wants to do something, and kind of guarantee deterministically that I can approve or deny that—or deny it with feedback and say, "Actually, no, do it like this"—then I won't trust it. We played in that space for a while and talked to a lot of founders, founding engineers, and builders.

We came into YC in the fall of 2024 with this idea. We were building out this API platform, and it was sort of like PagerDuty, but it wasn't about who's on call to fix the servers; it was about who's on call for this routing mechanism—like who needs to approve this agent, and can they escalate it, delegate it, or defer it? We built it for this ecosystem—CrewAI, LangChain, Griptape, there were so many at that time.

Then I talked to tons of AI engineers who were actually building really interesting things, making money, doing six-figure contracts shipping AI to the enterprise. All of them had tried that stuff for a month or two, thrown it out, and were just writing all their API calls by hand. They were building things that looked more like pipelines and workflows than these hands-off "call tools in a loop" configurations.

I talked to a hundred people. I spent a lot of time hanging out with one of my best friends, Vibhu from Boundary. They built a programming tool—like Protobufs for AI—and I think they're about to launch their full-fledged, Turing-complete programming language. He had this way of thinking about agents, models, and inference where it was much more about understanding what structured output really is under the hood.

Every single step in your AI workflow is just tokens in, tokens out. Your job as an engineer is to figure out, "Okay, what tokens do I need to put in to maximize the chance that the tokens out are going to be good?" I distilled all these ideas into about 12 principles, wrote about it on GitHub, posted a 12-page GitHub repo, threw it on Hacker News, and it was on the front page for about two days. I think it really resonated with a lot of people.

Yeah. So, I'll just quickly read the 12 principles, and then let's talk about one or two that resonate. The 12 are: natural language over tool calls, own your prompts, own your context window, tools are just structured outputs, unify execution state and business state, launch-pause-resume with simple APIs, contact humans with tool calls, own your control flow, compact errors into the context window, small focused agents, trigger from anywhere, meet users where they are, and make your agent a stateless reducer.

Yeah, the "stateless reducer" one—someone actually hit me up on Twitter and corrected me. It's actually a transducer, because there are technically multiple steps in the workflow, but there we go.

I spent most of March writing it and published it in April. Then Swix hit me up from AI.engineer and said, "Hey, do you want to come talk about this?" So I gave this talk, "12-Factor Agents," on June 6th, I think. It was a small room—it was packed, but maybe a hundred people.

That was the year at AI.engineer where, physically, on the second basement floor, it was all the super corporate stuff. You go up a level and it's a little bit more, and then on the top floor, it was all the weird, cutting-edge startup stuff that you probably shouldn't care about yet. So we were up there on the top, talking about this weird way of thinking about agents.

About a week or two later, Tobi Lütke from Shopify said, "I really like this idea of context engineering." And I was like, "I wrote about this two months ago!" This is great, Tobi gets it. Then a week later, Andrej Karpathy was like, "Well, I think what we should think about is not prompt engineering, but context engineering." And I was like, "Yes, that's mine!"

Anyways, if you ask Gemini, depending on what day it is, it will tell you either me, Tobi, or Andrej came up with context engineering. You can't really own a word; no one remembers who invented the word "prompt engineering." But of all the factors, factor three—"own your context window"—is the most important. Whether it's agentic or a single step in a pipeline, the only way you can impact the quality of your output from AI is by caring a lot about the inputs and crafting them.

Let's talk about context engineering, which I am going to credit you with coining. I did some research and I think you were earlier by a few days. So there we go, you coined it. We're adding to your SEO juice—we'll have it in the transcript: "Dex coined context engineering."

Well, an asterisk on that is basically that I learned about context engineering from talking to these hundred engineers and founders. I just looked at what was common across what they were all doing and put a name on it. I didn't invent doing it; I was just like, "I think there's this thing." Vocabulary and names are really important, and having clean ways to talk about the problem—especially when a lot of the content about AI right now is so much hype and jargon that is meaningless—is useful. I thought there was a word here that is helpful to builders to explain how they should be thinking about building their software.

So what is context engineering?

It's kind of like de-abstracting a lot of the abstractions that have been layered on top. You have RAG, you have memory, you have agentic history, you have structured output—you have all these things that are different ideas in the frame of agentic programming. At the end of the day, they're all different ways to pass tokens into a model and ask it to produce, usually, some structured output.

Understanding that is a lot more powerful than trying to learn "memory" or trying to pick some agent framework and memory framework off the shelf. Those things are all really good if you want to get to 80% or if you want to get a really good demo. But when you have to go from 80% to 95% or 99%, you need to go down a level and think about everything we are putting into the context window. What order is it going in, depending on which model we're using? All of this stuff matters. You have all of these levers that you can pull, and it just felt like the right abstraction for thinking about how to get AI to do the thing I want as accurately as possible.

Why has context engineering started to become more talked about? About a year ago, did it have to do with the context window that we could pass to LLMs? Did it start to expand, or did we just start to realize that we can do a lot more? The easiest one is, of course, system prompts, but whenever you build an LLM app, behind the scenes you will pass additional context as well, not just the prompt from the user—you will add a bunch of stuff. That's a dirty secret of any LLM app. Why do you think the focus is moving to, "All right, context is important"?

I think it always was important. What had to happen is a ton of smart people—again, like all these builders I talked to—had to focus really hard on producing software they could sell. They wanted to make something accurate enough that they were proud of and could sell to an enterprise that would be happy with it.

The easiest way to get to really high-quality AI applications is by thinking at that token level, and thinking about a string of different LLM calls rather than just tools in the loop, which is open-ended and flexible but not very reliable. Think of agents as workflows, as pipelines, as some mix between a couple of tools in a loop, versus just, "Hey, I have my tools, I have my model, and I have my system prompt, and these are the only levers I have."

Actually, you have way more levers. It's going to take more work, and you're going to have to understand the LLM with a deeper intuition. It was a thing we always needed, and it just took time for people building with this technology to figure out that this is the layer of abstraction that allows you to break through the quality ceiling.

And how are cost and context engineering connected?

Yeah. I was talking about this with someone this morning. When you're working with LLMs, one of the things I like to say is: "make it run, make it right, make it fast." See if the world's best LLM at the time—at the time we did a podcast episode, it was like o3—can solve your problem. Then give it to people and see if they want that. If people want it and use it a lot, then go do a bunch of context engineering.

Your engineering time is always the bottleneck. Humans trying to figure out and solve problems, build evals, improve, and try different dimensions is always going to be more expensive than just using a smarter model, until you have millions of requests a day. Then it's like, "Okay, we're going to do a bunch of context engineering, break this up into three calls, and get it to work on GPT-4o."

The point is, for a certain task in your workflow, can you get an open-source 12B model—which is 1/1,000th of the cost of Opus—to solve parts of the problem so that the tokens and things you're using the smartest frontier models for are just the things where you really need that level of intelligence? But you shouldn't go build all of that and over-engineer it until you've proved that you need it and that it's valuable.

This gets us to Eliyahu Goldratt and his book, *The Goal*, which was about how to model your factory. I'm sure we'll get to that when we talk about software factories. What is the bottleneck in your system? One day it will be latency and cost, but it's probably not that when you first start out. Context engineering is how you add human effort to the equation to improve the efficiency, the speed, and the cost efficiency of your system.

Interesting. And then one thing that came up more recently is harness engineering. What is harness engineering?

I made a post in October or November about how there's this new thing that I see, and I'm calling it harness engineering. My definition at the time is not actually what this guy, Viv, who's at LangChain now and does a lot of really good writing on agents, wrote. He had written something called "harness engineering" a couple of weeks before me, but I hadn't read it at that point.

My take was basically: when you build an agent, you use context engineering. We gave this talk in August of 2025 about how to apply context engineering to how you use coding agents, and that evolved into this idea of: how do you take a harness like Claude Code or Cody, and how do you engineer against the integration points of that harness?

So, commands, MCPs, skills, how you organize your codebase—how do you optimize the environment that the coding agent runs in to get the best results? In the same way that with context engineering you optimize the inputs to every single prompt, harness engineering is about raising the floor so that, on every single turn of this thing, the results are as good as possible.

The term got super blurry. Some people think harness engineering means building a harness, and some people think harness engineering means building around a harness. I actually like what Martin Fowler came up with—as usual, he's very good at naming things. He defined it as: you have the LLM, and then you have the "inner harness," which is the tool definitions and integration points that Claude Code, Cody, or an MCP actually exposes. That's your inner harness. Then you have the "outer harness," which is the stuff that you, the human, do to customize that for your specific needs, your codebase, your languages, etc. That's the best definition we have for harness engineering.

It's interesting how naming is still so important, isn't it?

As soon as you name anything, people adopt it. I'm actually surprised that context engineering still means the same thing to most people that it did a year ago, and that it's even still relevant. That's honestly the craziest thing to me. Think about how many things written about AI 15 months ago still matter, are still interesting, or still have good advice baked into them. Stuff changes.

I think context engineering has been so long-lived because it's grounded in the fundamentals of how transformer attention works. Until we have post-transformer models, linear attention, or whatever is next—which, who knows when that's going to happen—context engineering will be interesting and important to anyone building on AI.

Can we talk about the physics of context? You had a tweet—this one on the "context reality check." It's a graph showing that as you get to 1 million context, the quality drops. It goes down.

What do we need to know about the context window? We now have models that have a 1 million context window, and maybe we'll have even longer ones, but when you start to just put more stuff into the context, it starts to become less efficient. What do we know so far from a practical perspective for someone who is using the context window to add a bunch of stuff, whether that's MCPs, tools, skills, or other things?

Yeah. I mean, the longer context windows are good. You can talk to it for longer. They're doing a good job. But at the end of the day, especially when you had models like Opus, whether it was Opus 4.5 and then Opus 4.5 1M, or 4.6 and 4.6 1M, you're not actually getting a smarter model.

The intelligence of the model is what drives its ability to attend to all of the tokens in the context window, to figure out on the next turn which parts of this 100k or 200k context window are the most relevant to making the decision of, "What is the next tool we call?" and doing that over and over again in a loop.

There was some study that came out in 2025 which found that—and again, these are older models, so inflate your numbers—frontier LLMs can follow about 150 to 250 instructions before it starts to drop off. Their ability to follow all the instructions drops off pretty quickly. I haven't actually looked at the data, but they did a study with the next-generation models a year later, and it looks like it's much better in terms of the number of instructions you can get in.

In any case, I split context engineering into two categories. Most people think about the information budget: "Okay, I can do RAG, and I can pull out chunks of this document rather than putting the entire book into my context window. I can just go grab the pages that matter."

But there is also your instruction budget. If you give the model too many instructions, especially too many conflicting instructions in your initial prompt, or if you have a conversation where you start going down a path and then you change your mind and say, "Actually, I don't want to do any of that, I want to do this," that is a lot of computation the model has to do to notice that it has to ignore that whole thing. When both of those things are far back enough in the context window that they're only half-attended to, the likelihood that it's actually going to remember the exact instructions you gave it 100,000 tokens ago goes down quite significantly.

This is all very interesting because as engineers, when we're AI engineers—which is now a lot of software engineers, meaning you just use LLMs to build software, there's an LLM layer somewhere, you're an AI engineer, congratulations—the expectation is different. To be a good software engineer pre-AI, you needed to understand how to write good code, and it helped to understand the underlying layers, though we didn't need to do that as much over time.

But right now, we're in this phase where to be an engineer who can write an efficient AI system that uses LLMs, you need to understand the dynamics of the context. You need to understand why stuffing your context one way or the other can introduce latency and compute costs. It sounds like it's more of an intuition, though there is some understanding. From talking to you, it's like, "Well, it does this computation." You know that because you tried it out, right?

Yeah, I'm not a PhD in machine learning—I couldn't actually go draw a mathematical proof of how this works. But we know attention is quadratic, and the more stuff you put in, the more it has to spread this attention out over everything.

This just feels like an absolute new area, and very different from what we're used to in traditional software engineering, which is pretty black and white, right? It compiles or it doesn't compile.

That's true. There's a different kind of intuition that you develop over years as a software engineer. There are many categories of it, but the one I'll call attention to is something that you cannot teach, you cannot do, and you cannot learn in a textbook. The only way to learn it is: "I know bad patterns in software because I have debugged them at three in the morning." My buddy Jake from Netflix said this in his talk at the AI Engineer conference. There's no better way to learn what is good and what is bad, what works and what doesn't, than suffering through the thing that doesn't work.

Well, speaking of suffering through the things that don't work, a new paradigm that is spreading is loops—loop engineering. The idea is that instead of writing prompts, you just write loops. Set up your loops. This all started with the Ralph Wiggum technique, which is an early version of loops. Now we're hearing some of the biggest labs talking about how they're actually just doing looping.

What is your take on this? Have you done some looping yourself? Have you set up some loops? And what do you think is good about it, and what do you think is bad about it?

Yeah. So, I think of loops—I mean, I could ramble on this for 10 minutes, this is an entire talk—but I'll try to lay out some high-level stuff, and then we can dig in wherever you think is most interesting.

We had Ralph Wiggum. It was actually a year and four days ago that I first saw the Ralph Wiggum demo. Jeff Hunley was visiting SF, and he just came through and dropped everybody's jaws with, "Yeah, I just ran Sonnet around the clock, spent $6,000 in six weeks, and built an entire Gen Z programming language." And look, it compiles, and it has a stage-two compiler where the compiler for the language is written in the language itself, and all this insane stuff.

The core lesson from all of that, I think, was the idea of backpressure—which is basically how to let the model check its own work. How do we automate the process of getting feedback into the model? There are lots of different flavors of this. You can have deterministic linters; you can have unit tests. Part of what made the programming language easy to build with Ralph Wiggum is that a programming language can be infinitely verified. You write the code in the language, and you compile it. If the compiler fails, you go fix the compiler. You run the program; if the program fails, you go fix the compiler. It's very verifiable.

I think the lesson in loop engineering is: if you can make a problem very verifiable, you can treat it like a black box.

And then have it loop, because it will keep improving itself because the verification loop is already there.

Exactly. You can do this with CI/CD. I do this every time I'm doing a release. I'm like, "I'm tired, the CI/CD is slow. Cool, go research the codebase, make a change, make a pull request, run the test, see if it's faster, try again, push to the branch, and check again." If it can verify its own work in a loop—instead of you being really back-and-forth saying, "Let's try this approach" or "Let's try that approach"—you just say, "My goal is to make CI faster."

Then you tell the model, "Here are the five steps: you're going to write some code, you're going to commit it, you're going to push it, you're going to launch a sub-agent to watch the job until it's finished, it's going to tell you what happened, and then you're going to decide what to do next." That's the simplest example I have of designing loops.

And you just set the goal—which, I think Claude Code and Cody both have shipped—where you just set the goal and it iterates until it reaches it, or as long as it makes progress towards it.

Exactly. If it's verifiable, if you can measure it... this is like AI research, too. AI research is like, "Hey, go make this model twice as fast," and it's just a prompt that tells the model to go do it over and over again, trying things until it actually gets good results. That's what I think of as loop engineering.

We do a very interesting kind of loop engineering. The challenge is, it's very easy to get excited about building the thing that builds the thing, or building the thing that builds the thing that builds the thing. People say, "Oh, we need to redo everything as this big, agentic-first factory—maybe even a dark factory." They're redesigning their entire setup to be their infrastructure for the next five years.

One thing we know in engineering, and especially *pragmatic* engineering, is: how can you make this more incremental? How can you make it more continuous? A lot of people don't have the option to say, "Hey, I ran a Ralph Wiggum loop for three days, and it fixed every lint error in our codebase. Here's a 60,000-line PR. Who wants to review it, sign off on merging and deploying it, and guarantee there won't be any bugs?" Nobody.

The thing I'm most excited about is what we call "iterated loops" or "slow loops." We basically have a cron job. The structure of the loop is really easy: run this linter, fix one thing, commit and push. We run that every night in our GitHub Actions, and we wake up every morning to one PR that makes the codebase a little bit better.

I like the slow loops.

Yeah, and it has two dimensions. We have a blueprint for it now, and Kyle actually just shipped a skill so that you can build these yourself. You can add more feedback mechanisms. For example, we have React Doctor for the front end. We have another anti-pattern that has no deterministic tooling, but Kyle's just like, "Here's what good looks like, here's what bad looks like. Go fix one thing and bring it back."

It's like prop narrowing, basically. We have a bunch of optional props, and most of them don't need to be optional. It tells the agent, "Here's how to make the prop not optional so that the code is cleaner and easier to reason about." You can add more conditions, more things like, "Fix one thing, I want to wake up to a PR." Now we wake up to like four PRs because there are four separate tasks running.

The other dimension you can do here is, as you gain confidence, you can increase the scope: instead of fixing one thing, fix four things. These are other ways to think about loops where something that is not a human triggers it to start. Whether it's an alert from Sentry, user feedback on a support ticket, a PM writing a ticket, a failing test, or a cron running on a schedule—the trigger should be something you don't have to press a button on. There's a defined workflow, and it makes everything a little bit better.

Dex just described letting agents fix things without a human pressing a button. But what if a bug is too difficult not just for an agent, but also for a human to reproduce, let alone fix? This is where our presenting sponsor, Antithesis, comes in.

I was recently pairing with the Antithesis team, where we did a walkthrough of how they helped fix a nasty bug in etcd, the open-source key-value store used by Kubernetes. The team noticed that the linearization validation assertion failed during the regular Antithesis runs. This is not good, because linearization guarantees strong consistency, so this needed to be fixed.

What the etcd team did was run a causality analysis inside Antithesis. This generates a bug probability graph. Here, the x-axis is virtual time, and the y-axis is probability. We see that something happened just before virtual time 24 that caused a huge jump in the probability that the bug would occur.

Going deeper, we can look at the entire set of timelines. Vertical lines going down represent events branching off from the same state, and the purple dots are where the bug happens. If we look closely enough, we see that all of the failures come from one parent branch. Gotcha. This is such a useful debugging tool.

In the end, the team was able to figure out that process pauses were causing the bug using all these Antithesis debugging tools. This non-deterministic bug was diagnosed in a deterministic way. How cool is that? And this is an actual bug that then got fixed in etcd—you can see the bug and the fix in etcd's GitHub repo. Honestly, the tools that Antithesis built for debugging feel pretty darn futuristic, but they are also really powerful. Head over to antithesis.com/pragmatic to learn more.

I'd also like to talk about our season sponsor, Sentry. Sentry is a tool I use for application monitoring on all of my projects, including the Pragmatic Engineer backend. I've used it for 10 years now, starting when I worked at Uber.

A neat Sentry feature I'm liking is their Seer AI agent, which helps investigate production errors. For example, here's an actual error I had in my application. I can just ask Seer what might be the root cause, and it brings context. It can also make a plan to fix it right from the web interface. A nice thing is how Seer also works great from Slack as well, not just from the web.

One place I find even more handy to use Sentry is from Cody or Claude Code using Sentry MCP. Also, you can set up neat automations, like when a resolved Sentry issue resurfaces, you can kick off a Cursor agent or GitHub Copilot agent to investigate the regression, read the relevant code, and open a PR with a suggested fix.

I'm not a fan of using AI tools just for the sake of it, but I really like practical integrations where I can fix errors faster and with more context. Check out Sentry at sentry.io/pragmatic and start monitoring and fixing regressions today. And with this, let's get back to Dex and to agentic loops that trigger themselves.

Now, you said we can get more ambitious and we can add more things to it, but I'm going to quote you on one of your tweets which says, "This may surprise you that this is coming from me, but I think we're in for a one-to-three-year period where stuff might break at 3:00 a.m., and you're relying on loops to fix it, and nobody understands what's under the hood, and you're looking at an existential threat to your company."

Yes. Yeah, that one was great. That one did a lot of numbers.

It resonated.

Here's the other side of it: I think that today, with today's models, today's programming languages, and today's infrastructure, you might get away with not reading the code. But the problem with loops is that at a certain point, you're going to generate so much code that you can't read it anymore. This is the SWE-agent dark factory, or harness engineering—just spend as many tokens as possible.

We tried this. We built a lights-off software factory in July of 2025, and by November we had shut it down. I think it takes about three to six months of you shipping all the time with nobody reading the code before you realize, "Wow, this is getting way worse, and it's easier to start over than it is to fix it." The models have made the codebase so bad that it is actually going to be easier to just rethink this from scratch.

Maybe that's okay, because we have AI and it's easier to rebuild things from nothing. Usually, when engineers say, "Oh, we can't fix this, we have to rebuild it," the feedback they get is, "No, just refactor in place. Just constantly keep the codebase getting better."

But you'll notice what I said was *not* to use loops to ship the features that users want. We use loops to actually improve the codebase quality, and we read all the code because we care about how it's architected. We care not just about the system architecture, but what I would call the program design—which I think is something people are missing. Where are the interfaces? Where are the seams? How are we doing dependency injection? All of these things make your codebase more maintainable over time and keep you from falling into the trap of, "Okay, well, now if I change something over here, I broke something over here."

This is the classic problem of software engineering. Software engineering was invented in the 1970s because we realized we needed techniques for avoiding that problem of a giant ball of spaghetti. I don't think the models are smart enough, and I don't think we actually have the training, benchmarking, and eval techniques to get models to write code that is more maintainable over time, because they're all trained on SWE-bench and SWE-bench-looking things, right?

All of the benchmarks are basically: "Here's a commit in Django. Here's an issue that was filed around that time. See if you can create the fix that the human created." It's Django, Apache, and a hundred repos in Go, C++, TypeScript, Java, and all these different languages. But the problem with training models on maintainability is that the cost function of bad architecture and bad program design can't be evaluated by running a unit test, because it hits you three to six months later when you're like, "Holy crap, this software has become so hard to change."

Is this not similar to why it took years for someone to become a senior software engineer? Typically, and in some environments, you could become a senior faster—typically fast-moving startups where there's a bunch of issues and you have to keep fixing things. Sometimes some people work in the same place for 10 years and they're still not at that level.

The point is, it just takes time for you to understand how a small mistake you make right now snowballs into something disastrous later. You get hit by it, and you realize like, "Okay, testing matters, architecture matters, tech debt can actually be a killer." We don't talk about it as much anymore, but we used to talk about how tech debt kills or slows down companies so badly pre-AI that their competitors would overtake them while they were stuck with a two-year refactor not shipping any new features, and the competition shifts a bunch of other stuff and now they're ahead.

And I will say, it is possible that GPT-7 will fix this. But if you are turning the lights off in your software factory and saying, "Hey, you know what? We're not going to read the code. It's fine, the models are smart enough. If we give it the right feedback and just throw enough tokens at the problem, it will keep getting better"—this is what led to that tweet.

That might work for a bit, but if nobody has read the code in three months, and you replace all of your code review with loops—like, "Hey, if a user complains, we give it to an agent. If something crashes, we give it to an agent. If a PM writes a ticket, we give it to an agent. If a CEO writes an obnoxious essay in Slack about what we should be building, we give it to an agent."

Yeah.

And then you stop reading the code because it's going to produce way too much—no one can read it. The PR reviews become the bottleneck, so you replace that with agentic testing and agentic code review. But none of these things have intuition for software architecture, because we haven't trained it in yet.

So you're going to wake up one day and you're going to have an issue. This happened to us. We got through it, and at the time, it was still worth it. It was like, "Okay, we spent three weeks onboarding back into the codebase that we had stopped reading three months ago, because no matter how much sophisticated, expert prompting we did, we could not get Opus—I think it was Claude Opus at the time—to actually find the root cause." We had to go spend several days digging through the code and figuring out, "Oh, there's actually a primary key being routed through this whole thing that needs to be changed to a different type of object."

This actually happened to you?

This happened to us, yeah.

And when it happened, I was like, "You know what? That sucked. That was terrible. But we did it, we solved it." And back then I thought, "It's still worth not reading the code most of the time, at the cost of having to spend two weeks fixing an issue by hand every once in a while." But I don't believe that anymore, because I think the amount of code we're able to write now has 10xed or 100xed, and I think the problem is just getting worse.

So let's talk about software factories.

Yeah.

In your mind—because I feel it's an overloaded word—what do you think of a software factory before AI, and now post-AI?

Do you know the first definition of a software factory, the first time it was used?

No.

It was a NATO conference in 1968.

Oh, Grady Booch would know about this.

Yeah, exactly. You should ask Grady about it. They talked about the idea of, "Okay, you actually need to build a system of steps, just like a factory floor." You have the coding part, the testing part, the validation part, and the integration part. We had no CI/CD, we barely had version control, but you needed a factory.

It was then adopted by Toshiba and a bunch of other companies. The next major moment was DevOps, where you had this idea of: "Okay, we're going to do CI/CD, we're going to automate." Chef, Ansible, Puppet—all of these technologies made it so that instead of having guys running around data centers resizing disks or clicking around the AWS console...

Yeah, exactly.

It was like, "Cool, we build loops: the server hits 90% disk space, that sends an alert to Nagios, Nagios triggers a Chef run, Chef makes the disk bigger." Feedback loops, right? This has been around for a while.

In 2018, this guy Nicolas Chaillan, who was the Chief Software Officer of the Air Force, wrote this 100-page essay saying, "Hey, the DoD needs a software factory."

The Department of Defense.

Yeah, the Department of Defense and the Air Force. He called it the DevSecOps factory. He said, "We need all the things that all of the good enterprises are using. We need Jenkins, code quality scanning, security scanning, CI/CD. We're shipping once every three months or once a year—we need to be able to ship every day like all these other companies."

The way we do that is by embracing all these automations and technologies, so that 90% of the issues are caught by automations instead of people manually checking, manually reading the code, or manually integrating modules together.

Wow. Talk about forward-thinking in the government.

I know. I was surprised, like, "Oh, nice. This is cool." Part of it was, "Hey, look, we're falling behind." I imagine it was also about attracting really good talent: "Hey, look, if we have a modern software stack, we're building things fast, we care about efficiency, and we care about using people's time well." We want them spending time on the hard parts of the job, not manually looking for SQL injections, because you can automate that. So this was software factories pre-AI.

Pre-AI.

Now, we hear the term a lot more because of AI.

Yeah. Is it the same, or is it different?

This is really hard to explain without a drawing, but I'll try to draw it out. At the core of a software factory, you have a source of work. You can imagine Linear, Jira, or whatever the source of truth is—you have what stages the work is in.

Yep. And pre-AI, you would maybe do some architecture review planning, some sprint planning, and then people would take tickets off the queue and go build them. Then you would make a pull request, people would review it, and you would run CI checks. Then you would send it to prod, and then it would make contact with your users.

If your users complain, that goes to your support team and back into your work tracker. If it crashes, that goes into your monitoring stack and back into your tracker—and that was your loop. People would take stuff off the tracker based on priorities—product managers, engineering managers, and engineers prioritizing work.

The first change is this long-winded process with lots of phases. This is why when a developer ships a bug, by the time it comes back to you, it might be two or three months, and by the time it gets fixed, it might be a year or two. When you're using a piece of software and you hit that annoying bug, you talk with customer support, but there are just very long latencies at each part of the factory, if you will.

Yeah. And the step where someone pulls a work item off a queue and starts working on it is a couple of hours to a couple of days before it actually gets integrated and touches users—and that's in a great world, right? Sometimes you build it, merge it, and then it actually gets released three months later.

But we're going to assume we're in a fairly modern world—somewhere like Netflix or Meta, where engineers are capable of shipping a hundred or a thousand times a day, but it still takes two or three hours to do the work.

Now, with an agentic factory, what you do is take out the person building the thing and replace them with an agent. You have orchestration to trigger things, a sandbox, an LLM, an inner harness, and an outer harness—which is the dev environment you build for the agent. Maybe you give it a browser or a video recorder, if you use things like Cursor's background agents. They've built this outer harness around the inner harness of the coding agent, and you make PRs with that.

The problem there is that now it takes 10 minutes to do a build instead of two hours or two days, so the bottleneck becomes code review. So, you say, "Okay, let's throw a bunch of AI agents at code review, and let's do agentic testing so that we can basically catch a lot of the easy stuff, and humans only focus on the most important, critical, core parts of the codebase."

Then the next level up of your agentic factory is the top loop: it gets deployed, goes to prod, and if a user complains, you hook your support queue right up to the agent. Someone complains about something, the agent tries to fix it, and instead of looking at a ticket and triaging, you just close that loop. Every time something goes wrong, you just get a PR. Every time something crashes in Sentry or Datadog, it goes into the tracker, gets picked up by an agent, and you get a PR.

This is the ramp-and-inspect thing. The only difference is that then you have so much code to review that people say, "Well, let's try turning the lights off. Let's just take all the human testing and review steps out. If users complain, then it's broken; if users don't complain, then it's working. We're not going to read the code—we're going to treat the whole system as a black box."

So, you said you tried this out back when it was Claude Opus, and you built a software factory that was running beautifully until it just blew up in your faces. How do you think of this model? I can see an ideal world where it works, but clearly we're not in an ideal world. Where do you think we are, and could some of this actually work at some point? What progress are you seeing right now, and what is the situation today in terms of how much of this we can or should automate?

Yep. If you know me and follow my stuff, you know I stand for three things. Number one is cutting through the hype and the jargon, trying things, talking to people who are using them, and figuring out which parts of this actually work and are valuable.

Number two, we talked about words: I try to find and protect useful bits of language because I think it helps us all move forward. When you take a useful word like "agents" or "software factory" and semantically diffuse it—which is another Martin Fowler term—you make it so that because everybody likes the word, it all becomes hype, and eventually "agents" means nothing. It could be a chatbot, a Slackbot, a coding agent, or tools in a loop. I like to protect important, useful words to help elevate the conversation out of that hype and jargon.

Number three, I care a lot about going one level down beneath where I'm generally working. It's the same thing with context engineering: I was rarely building or training LLMs, but knowing how they are trained and how transformers work informs how you build one layer up.

For the software factory, my version of that is spending the last couple of weeks going really deep on reinforcement learning with verifiable rewards (RLVR). RLVR is a very productionized approach—unlike RLHF, which is still fairly academic. RLVR is a machine in these labs for how we train these models.

I'm studying the benchmarks for coding agents, the techniques for training them, and how we give the model a small problem, have it solve it, revert the changes, apply a test patch, and see if it passed. Even this year, we have new benchmarks like Frontier Code and Marathon, which are supposed to be better at evaluating a model's ability to maintain a codebase over time and write maintainable code. They are better, but I don't think they're sufficient.

But it's basically the idea that the only thing that made Claude Code good was reinforcement learning. The dimension along which it got good was that they trained the model and the harness together. The model got really good at calling the specific tools in that harness—reading files, writing files, searching for files—by doing these problems. That was what made it feel so much better than all the other CLI coding agents that came before it.

People are like, "Okay, that was so much better," and they're just going to keep getting better. But it got really good in only one dimension. The dimension they're not getting better in—because it's hard and expensive, and maybe we need to get a lot more creative with how we design these verifiers and benchmarks—is: "How do I make code that in three months is going to improve the productivity of humans and agents—mostly agents—in the codebase, instead of making it worse over time?"

And so you think that part is just missing? We haven't seen too much improvement.

I haven't seen... obviously, no one knows what the labs are doing internally because it's all very secret. But I think if we look at where the benchmarks tend to reflect, that's where the labs are, right? If there is no benchmark that can convey to me, "Did this model write code that is going to make my codebase better or worse?"

The best we have is, I think, Frontier Code from the Cognition team. It is really interesting. They have like, "Did the test pass?" and then they have like two layers of model review. So they have a judge model that checks, "Okay, is the patch the model made similar to the patch that is like the golden answer set?" So even if the model didn't write the exact code that the benchmark was expecting, was it functionally equivalent? And the next one is like a code quality review from another judge model. And like, that's better, but it's not sufficient.

And this is why I also think agentic code review is like... yes, it will catch things and it will raise your floor, but I don't believe the model writing the code is the same model reading the code. And if you ask a model, "Hey, is this code good?" it's going to be like, "Oh yeah, it's great! Comprehensive, it's got unit tests." You've tried this, I'm sure.

And you say, "Okay, review this PR that my coworker wrote and tell me everything that's wrong with it." And it's like, "Oh, it has this problem and this problem." This is sycophantic, and they want to tell you what you want to hear. And so, it's really hard for me to trust a model to evaluate the quality of code that's written.

And so, I have some ideas on like, "Okay, can you build a benchmark where the model builds 20 features in a row and maintains the codebase the whole time, and it doesn't know what features are coming?" You treat it like a real product team where you don't know what you're going to build next week until you get there and you find out what's most important, and then can we try to evaluate—can we build a problem like that that's hard enough that most frontier models fail by issue six or seven?

Is it fair to say that, you know, we've had the software factory? Before AI, it was just like lots of loops. It was like the PM giving the ticket to the dev, the dev building it, deploying to production, users and customers using it, customer support getting tickets, and then creating a PM ticket, triaging, and it kind of goes around in this loop.

Is it fair to say that the software factory of how a company or a team builds and maintains software, that is changing because now everyone's replacing some parts of it? You know, maybe the least advanced teams will just be devs starting to use Claude Code or Copilot to write faster. They're not spending as much time on there. Some others are also having the deployment feedback. Some actually have the agents already one-shotting bugs.

So, is it fair to say that the software factory is just changing everywhere? Maybe at different speeds, but I think every team who is building production software, they're frantically experimenting, trying, and everyone's at a different pace. You'll have the AI-native startups where most of this will have agents in them, and you'll have the laggards or the more cautious ones who have agents in a few places but not in the others.

Well, and I think that's the key. If you want to do loops engineering, you should build one loop at a time, and you should keep them small and contained. Basically, I think everything except "stop reading the code" is really good advice. Take support tickets and turn them into tickets in your system, and then maybe turn those into PRs. Great.

The advice that I have and what we are chasing at Human Layer is like, how can I add another checkpoint in that factory? So instead of having one human review point where you're reviewing PRs (and sometimes they're 100 lines and sometimes they're 1,000 lines, but it's quite a lot of effort, especially if it's bad, especially if it needs rework)... it's quite a lot of effort for a human to be like, "Okay, this is wrong. Go change it in this way," and then you loop back to the agent, and then you come with another one.

Doing a lot of loops on there, once the direction has been committed to, it's really hard to steer off. You're better off just restarting from scratch. How do you build controls and mechanisms around that?

And my take is, if you do a little bit of human-agent planning and discussion before you hand it to the implementer—whether it's... I mean, planning and specs, whatever you want to call it. Again, "spec-driven development" is another word that has become very muddled as far as what it means.

But basically, how can we spend an hour before we start building so that the PR, when we read it, only takes 20 minutes because the code is perfect, instead of not touching it and literally saying every user-reported issue becomes a PR through the loop, and then we read that PR and it takes six hours because there's back-and-forth and we have to make changes and things? It's all... I'm all about, "Let's find leverage." And so, you basically have three options in the software factory world.

If you're going to go all in on agentic software factories, you can turn the lights off and just let everything flow and pray that you don't create too much slop and pray that the next generation of models comes fast enough before you create a giant pile of ash.

You can slow way down and read every PR and read every line of code, and then you're only going to really get modest benefits from AI because that becomes... I think you should expect maybe a 30% to 50% lift in productivity, which is kind of what I see when we go into teams.

Or you can find the right leverage points where humans can actually... an hour spent over here in planning can save you four hours in implementation in terms of fixing and going back and getting the design right. And that's what I call seeking leverage. If you can find the right leverage points for the agents to guide the work, then you can actually move two to three times faster while maintaining like 99% accuracy to, if the humans were carefully writing this code by hand, how would it come out?

Now jumping a little bit back to ideas. I will come back to this. This was earlier, maybe it was last year, but you had the Research-Plan-Implement. Can we talk about the original Research-Plan-Implement framework, and then also what you've learned about this approach? What you got wrong about it?

Yeah, sure. Yeah. So, I mean, the first time we talked about RPI was in August of 2025. And it was basically like... the research was this thing of, "Hey, before you go build anything, go read lots and lots of code. Use a bunch of sub-agents in parallel, understand all the code." It was this technique that worked really well for hard problems in complex codebases.

If you just ask Claude to do a thing, it would read three files and make a change; it would have no context. So, you start the research. You don't even tell it what you're working on. You just tell it, "Hey, can you tell me how this system works, and this system, and how they connect together?" And then you get a markdown doc out.

And this is the context engineering part: that would take 100,000 tokens of context, but you would get a 10k token doc out of it that summarized it. Then you would start a new context window and you would do planning.

And the planning would be... actually, I realize the plans that we were building last summer were actually terrible. But it would basically be this: you would say, "Okay, now here's what we're building. Here's the research doc. Build a plan to implement it."

In retrospect, now that we see everyone is obsessed with, "How do I get agents to work for longer?" I think the reason why in May, June, July, August of 2025 that a lot of people became really interested in planning was it was a very powerful lever to get agents to work for longer. If you said, "Build me a B2B SaaS for burrito delivery," you'd get like a homepage and that's it. But if you said, "Build me a plan," it would build out this big plan.

And then in the next context window, you'd say, "Hey, here's the plan. Here's all the changes we're going to make. Go implement." It would actually keep going until the plan was done. So the plan was a really good way to anchor an agent and remind it that, "Hey, you're not done until this is all finished."

So that was the original RPI. And the plan doc—what was bad about it is it didn't give you leverage. The plan was every single line of code that was going to change, like in diff blocks and all the new stuff to write. And so people would review these plans. We recommended this; we told people to read the plans. We read all our plans, and then eventually I found myself... I just kind of skimmed the plans.

And so you're not really using it as a way to re-steer the agent; it's just kind of there. And then you go write the code and there's a crap... some people would review the plans and the code, and it's like, "Okay, well, the plan took you 20 minutes to read, and then the pull request takes you 20 minutes to read, and they're different." And so you actually doubled the amount of time you're spending reading code instead of doing less of it. You've anti-leveraged.

And hang on, was spec-driven development not related to this? The one that Amazon Q, for example, and GitHub Workflows again, a year ago, did, which was... it also first generated a plan, and it had the human review it, and then it started to... and you could edit it as well, and then it went off and implemented this part. It looked beautiful on the surface; it should have worked great, but it's tossed into the garbage outside of some maintenance projects. I think it just didn't work. All the feedback I got, people just stopped using it because it just didn't really work that well. It just rhymes with the RPI framework a little bit, the original one, right?

Well, so our thing too... the biggest difference between RPI and spec-driven development (and some people refer to RPI as spec-driven dev because for some people, SDD, all it means is, "I use a bunch of markdown files while I'm coding and forget what's in them. I just write those as my specs, and I'm using them to drive development").

There was this OpenAI researcher who talked about spec-driven dev and like, "Hey, stop reading the code, just write the specs, and treat the coding part as compiling specs into code." That part never really materialized—maybe with GPT-7, you know?

But the challenge... I'm on a GitHub issue in spec-driven dev that has been open for a year, and every couple of weeks, I get a new email on the thread of people complaining about this problem of, "Okay, I edit my specs and then I edit the code, and then the code drifts, and how do I keep the specs up to date as the code is changing?" And basically, you now have two sources of truth, and it stops being useful.

And so, that's why with RPI, the idea of the docs is... they were, for a while, we kept them around. But after two or three months, we're like, "Oh, these are actually tactical execution docs. I do the research, I do the plan, I do the implementation, I throw the docs out."

And the next time I need research, I just do it from scratch because tokens are cheap, and my time is expensive, and the amount of time I might waste if I reuse a research doc that is no longer in sync with the real state of the codebase... so we just create it live every time. This is why context engineering still matters.

Creating artifacts that compress the state of the codebase and compress the intent of the builder into small things that can be reused in the future for the scope of a task is a very powerful tactical approach. But it's not a thing... I have very few opinions on what sorts of docs you should leave lying around your codebase that are evergreen.

I've seen people try to maintain parity between documentation or specs and the code itself, and I don't think anyone actually found it very useful. Like, you can do it and it works, but the ratio of the effort it takes to keep them up to date—and trivially, you could do this with AI probably, but I've never known anyone who was like, "Yeah, this is great, and we're glad we have it."

Like, you could do it and it might help, but I don't think anyone found it useful enough to maintain a system to keep the specs and the code in sync versus just using the code as the source of truth always.

Now you mentioned something interesting, which is with context engineering, you need to sometimes compact. And you've previously talked about intentional compaction—that when context is noisy, deliberately compress the useful part into a clear markdown artifact, verify it, and then start a fresh conversation. Can we talk about this kind of compaction and why it's important? It sounds like it's going to be a building block—or it already is—for context engineering, right?

Yeah. No, frequent intentional compaction is *the* building block. It completely comes from context engineering. Context engineering is like: how do we get the most out of today's models? How do we change what we're putting into the model, into the context window, into the agentic chat? How do we control that in such a way that we get the best results possible, which means doing as much work as possible in the smart zone (the first 100,000 tokens of the context window).

And this intentional—frequent intentional compaction—is basically like: okay, the research step, we're going to go read a bunch of code and turn it into a doc. That's our compaction. We take that forward in the next session. We're going to read the ticket and the intent, and turn that into a design document that we call... okay, here's the high-level spec of what we want to do. Here's a high-level current state, desired end state, and then a bunch of design questions. The model has kind of like a very thorough, maybe even overengineered, plan mode.

And then you take the research and the design, and you do a new session, new context window. You're like, "Cool. You've compressed the intent and you've compressed the state of the codebase so that you can then do your planning." Like, okay, we know what the end state looks like. We know where we're going. Now, let's break down how we're going to get there.

All of these different steps of the process exist because models have shortcomings in each of these phases. So, the research is pretty hands-off. I don't read the research docs. It's just like, "Go read a bunch of code and then make a doc out of it." Models are pretty damn good at that. If you ask it to find a bug and have opinions about the codebase, that's different. But if you just ask it what is the intent and how does this stuff fit together, that's usually pretty straightforward.

But designing the end state of the software—the architecture and the program design—models are not great at. They make decisions, and sometimes they're right and sometimes they're wrong. So we want to have a human in the loop there.

And then the steps to get there... we talked about this before, but models love making what I call horizontal plans. If you ask a model, "Build a plan of steps to go build this app," it's like, "Cool. We're going to do the database, and then we're going to do the services layer, then we're going to do the API, and then we're going to do the front end."

It's like, well, that actually kind of sucks because we're going to be on the other side of 2,000 lines of code—and let's imagine this is an existing codebase, right? We're going to make changes to all these different parts of the system. I can't test it till the end.

And so what I would do is: okay, how would I have built this if I were building by hand? Well, okay, I would probably create a mock API endpoint with fake data. And then I would go get the front end kind of how I want it to look. And then I would actually go build a services layer and actually wire the data through. And then I would make a database migration and make my new table. And then I would actually add a lot of business logic. And then I would add a bunch of error handling.

And it's completely orthogonal to how models would write the database layer and all the error handling without ever... like, anyone's ever touched or seen the code or whatever it is. And so this is another place where we like to have humans involved because humans have really good taste and judgment. Like, I would rather read five separate little mini diffs of things that I can manually verify and explore, than read 2,000 lines of code and be like, "Well, it's not working, and I don't know where." You don't know where because you wrote the code! You were supposed to get it right.

We talk about compaction, context engineering... it's like, how can you stay in the smart zone of the context window? Which is... again, the dumb zone. I will say, disclaimer, it's really good training wheels if you don't have intuition about this.

So let's just define these things. What is a smart zone and what is a dumb zone?

So, it's a little bit blurrier than I would like it to be. I think in November we talked about the first 40% of the context window, but then we had million-token context windows.

Yeah, then we had million-token context windows. So then I changed it to like—

So then I changed it to like the first 100,000 tokens. If it's a really like Claude 3.5 Sonnet, I usually will go up to like 200k. But basically, the thing Jeff Huntley had and Ralph Wickham was: the less context window you use, the better outcomes you'll get.

And basically, the smart zone meaning, if you have context in that first part, it should work a lot better. And then the dumb zone is like, once you have stuff there, it's kind of... forget about it. It'll be confused, it'll degrade.

Yeah, and there are times—and this is an intuition thing—I will often go up to 300k or 400k tokens. Four hundred is rare, but I will go up to 250k or 300k tokens for certain types of work where my intuition tells me that I can keep working without degrading the performance.

But if you don't have good LLM intuition, 100k for smaller models, 200k for these really beefy like Sonnet and Opus 3.5 models is usually a good training wheel guideline of: if you pass there, your quality of results may be degrading.

The biggest tell I see for this is often the model's trying to get the test to pass and you're at a 200k token window. "Well, let me try this. Okay, let me try that." And it's trying a bunch of stuff, and it's getting more and more extreme, and it's like, "Oh, let me delete your `.env` file and try again." This is where things get really, really weird.

And so it's like, if you start to see certain types of... if I'm like, "Oh, we're at 300k tokens and I need to fix the unit test," I'm like, "Cool. Write everything we did to a file (or even I'll just do a built-in compaction depending on the model)." And then I'm starting a new session at 30k or 50k tokens, and I'm like, "Cool, we're going to do a hard thing, which is you're going to get this freaking test to pass, and you're not going to be stupid about it."

One thing that you said about the model being dumb is, you said that if the model ever tells you, "You are absolutely right," you should start over. And we've all had that, when it tells me, "Oh, you know, you're absolutely right," and I'm like, we just get annoyed. But why should we start over? What's happening there in your observations?

Yeah, that's great. Yeah, and the new "You're absolutely right," I think, is: "You're right to push back on that." Right? Yes.

That's Opus, right?

Yeah, Opus is like, "You didn't run the test, did you? You're right to push back on that." "I totally did it!" But no, for me, "You're absolutely right" was always what the model would respond if you were like, "That's totally wrong. You did it." Like, if you said something where you were angry or frustrated or just wanted to point out that it's done something wrong, it would respond with, "You're absolutely right."

And most of us have had the experience of: it says that, and then it continues to do the wrong thing. So, once it starts doing dumb things... because there are four things in your context window that matter. There's like the size of it (how many tokens?), there's the quality of the information (is there any incorrect information? Like if the model had some thinking trace where it decided the wrong thing was true), is there missing information? (does this have context missing that it should have?), and then there's the trajectory. And the trajectory is very subtle, but you may have had sessions...

The trajectory meaning your prompting?

The actual history of everything. I call it trajectory—the actual history of what the agent has done in the past.

And so if I say, "Hey, make this change," and the agent makes the change and then it runs the test and then they're broken and then it fixes the test, I have very high confidence the next change I asked it to make, it's going to follow that path again because it's like, "Okay, here's a conversation, and the last time the user asked me to do a thing, I made the change, I ran the test, the test was broken, I fixed the test, and then I told the user."

But if I say, "Make a change," and it makes a change and it doesn't run the tests, then I'm on a different trajectory. And if I say, "Okay, make another change," they are autoregressive, so they're predicting, "What's the next message in this conversation?"

And so the example we talked about in *No Vibes Allowed* was, of course, like: hey, the model makes a mistake, and then you yelled at it, and then it made another mistake, and then you yelled at it. And then it's like, "Cool, what's the next message in this conversation? Well, look, if I read the history, I should probably make another mistake so the human can yell at me." So I was like, "Okay, that's a great example of time to start over."

Let's talk about some observations on how software engineering is changing. One thing you talked about recently on the evolution of the coding meta is going from "token harder" to "token smarter." Can we talk about what you mean by "token harder" and "token smarter"?

Yeah. So, token harder is... I mean, I'm in a group chat called Hyperengineering, and it's all people trying to max out their Claude subs.

Oh, wow. Okay.

It's just like—

That sounds like a fun... is it a fun place?

It's a fun place, but it's like all token harder. It's like, "Look at all the side projects I built. Look at everything that I've gotten my Claude tokens for. I've got six Claude accounts. I've gotten all of them maxed out every 5-hour period. I've timed it out so I always use all the tokens, and it starts up immediately when the limit resets."

And so it's like—I mean, getting into Eli Goldratt and *The Goal*—it's like optimizing for utilization and efficiency of one node in your factory, rather than the end-to-end goal of: how do we ship value and things that people like, that are stable and will last a long time? But that's my idea of token harder. And it's the same thing with the dark factory thing: if you remove humans from code review, you can push more tokens through the system.

So we talk about software factories, but what is the dark factory?

Ah, so the dark factory... this comes from this idea of: there are factories where everything is automated by robotics. So you can imagine a car factory where it's all robots building the cars, and they don't have lights because there's no humans.

Oh, so that's where it comes from.

The dark factory. Yeah. You walk in, there are no lights. There are not even light switches.

So, it would be the fully automated software factory where there would be like no human input, basically.

No human input. Raw materials go in, cars come out.

Yep.

And I think in a micro-way, you can have many loops that are dark in your factory. Like: hey, if the code review agent comes back with a problem, you loop that back to the builder agent, it fixes it and comes back, and that's dark. You don't need a human in the loop for that.

But the full dark factory where you don't read any code? Yeah, it's a good way to maximize your token utilization. And if your belief is: "My job is to extract as much intelligence out of the machine god as I can, because that's how I get the most value and the most leverage on my time," then token harder.

But my take is basically what we talked about before, token smarter: okay, how do I move faster? How do I get as much value out of AI as I can without having to turn the lights off, while still maintaining control, taste, judgment, understanding the system architecture, and applying my hard-won opinions through 10 years of software engineering to the design of the program so that I can feel confident that the code's going to get better and more maintainable over time?

It's the same thing of... you look at the SRE team inside Google. They brought out this book, *Site Reliability Engineering*, and the whole take was: we're going to go from one data center to five data centers, and we need the same six-person team to be able to manage five data centers. And we need the same six-person team to be able to manage 50 data centers next year.

And it's basically: how do we apply software to this problem so that instead of scaling linearly (like: "Okay, every data center needs five DevOps people, so we need to scale the people with the things"), how do we continually automate the parts that we don't need? So, a little bit orthogonal and maybe even contradictory to what I just said, but this idea of: how do you find leverage?

Well, I think what you were saying there is like, when Google did that, they never sought to remove those SREs from the process at all. They just said, "Look, can we think ahead and scale yourselves?" And they actually grew the team—it wasn't actually six people, it was more like... I think Google specifically said, "Okay, we have five data centers. Next year we'll have 50. There's six of you. We do not want to have 60 people and then a management layer and all that."

It's like: how can we do it with like 12 or like 10, and then when we'll have 500? And now, actually, their SRE team has grown, but—

Of course, yeah.

But they never... you know, I think as engineers, we feel pretty threatened when someone says, "All right, we just want to have zero engineers." I mean, that's not a fun place to work at. But what it sounds like—

It's not a possible place to work at. If they have zero engineers, neither of us can work there, right?

But do I understand that "token smarter" is like: let's keep humans in the loop, let's keep adding value and figure out what are the parts which are not as relevant, boring, or where we don't need them? And so, one developer can probably do more than before, but you are built to be part of this whole thing, and the lights are on in the factory.

Yeah. And it's basically... I think what I'm trying to get to, the connection here is like SRE builds a thing where headcount scales at a square root function or a logarithmic function, whereas their output scales linearly. And you want the same... the way you do that is with good architecture and good program design.

And so, in order to avoid this problem where you have to throw more people or more tokens at the problem, if you design good software in such a way that it gets more maintainable and more scalable over time... just today, it doesn't feel like... basically, you need humans in the loop to be able to do that.

Let's talk about AI slop. At one point you wrote, "Yeah, AI can write your code, but it can also write your specs and PRDs. But the same rule is always: slop in, slop out. If you outsource your thinking, you're going to get garbage."

Yep. So, yeah, that's basically the idea. The way we think about getting high-quality outputs is... yeah, you could write the code by hand, or you could sit with a model and work back and forth and go maybe a little bit faster, and you have control. And every time it makes a change, you go read the change, and if it's bad, you tell it, "Nope, we want it like this," and you kind of incrementally, slowly...

This is kind of the stage two or stage three version of working with agents where the agent is writing all your code, but you're very much in the loop. And this will make you go faster, but it won't make you go *that* much faster. It won't make you go anywhere near... there's that level, and then there's the maximum speed you can go while still caring about the code. And then there's the maximum speed you can go if you turn the lights off.

And so we always think about it in terms of leverage: okay, let's take... everything starts with a sentence or a voice note ramble, like, "I want to build this thing, it's going to work like this," or whatever it is, to, let's say on average, two sentences. "I've got to fix this thing," or there's a support ticket, "I've got to fix this thing."

If you can turn that with AI into a one-pager, and then turn that one-pager and make sure that's correct, and then turn that one-pager into a three-pager and make sure that's correct, and then turn that three-pager into a 10-page detailed outline, then you can write a 100 pages' worth of code.

And it's maybe not perfect—you shouldn't sweat over these documents and make sure they're perfect—but you're increasing the chance that... you're decreasing the uncertainty of the outputs. It's like you have a line of where it's going, and then you have the probabilities of where it might go in that range.

If you are reviewing along the way as you get more and more detailed into what you're building and how you want it to be built, you kind of collapse the uncertainty and the set of end states that you could land in. That's me doing the physics thing of: you've got to superimpose all these probabilities.

And I don't know, I have this thing that... I think people who really like playing real-time strategy games are probably going to be really good with AI because you kind of have to... I don't know, Matt Piccolella was just talking about fog of war and things that are at the frontier of: "There's stuff we don't know about this problem yet. How can we find that out, and how can I make the best decision now, knowing what I have seen?" There are a couple of pieces of information, so there's a 30% chance it's this, and there's a 40% chance it's this. How could I get more information so in my head I can recalculate those probabilities and decide what's the most likely path that's going to lead us to success?

Speaking of the most likely path that leads you to success, let's talk about your company that has just come out of stealth: Human Layer. What is Human Layer, and what is the probability that you're setting up for success?

That's a good question. One hundred percent! One hundred percent probability—maybe 110. But no... so Human Layer is an AI IDE, a collaboration platform, and building blocks for your software factory.

And the basic pitch is: for engineers solving hard problems in complex codebases, basically, there are two categories of builders. There are vibe coders building side projects, and then there are people building production software where the stakes are high, and if something breaks, we're going to get fined millions of dollars or we're going to lose millions of dollars of money for the company. And there's a whole spectrum in between there.

But if you're in the left half of that spectrum, building software that matters, and it has to last and be around for a while, then helping people like that solve problems two to three times faster without descending into slop—how do you maintain that near-human level of quality and move two to three times faster?

And what were the ideas that you built and that you came with?

One idea that we're really excited about right now... I mean, it all comes from this RPI and using specs to... I've kind of been hinting at it this whole time, right? Like: okay, cool, start really high-level and zoom in layer by layer, and re-steer and find that leverage that helps you move faster and increase the chance that your agent's going to build exactly what you want or something that's really high quality.

The other thing I think that's really interesting... I just posted yesterday, I said, "Hey chat, should we kill the pull request?" And that's something I can't talk too much about, but basically, the idea is the IDE of the future needs to be rethought from the ground up for agents.

And it might not even be... a lot of editors kind of started with the text field and bolted on an agents tab. And then eventually, you've seen like Cursor... I can't even find the text editor! I know it exists; people have told me you can get to a text view of files, but it's also very agent-first.

And so we started from the ground up of: what is an IDE designed for helping a developer interact with and manage the work of agents? And then we zoomed out and said, "How do we make this collaborative and build in a sync engine, durable streams, and all of these pieces of tech that enable me to get human input and feedback on what I'm doing with agents in real time, rather than waiting for the pull request time?"

And great engineering teams have been doing this for decades. Like, "Hey, we're going to have a design review where we talk about how we're going to build the thing as a two-page Google Doc (or a 10-page, however, PRD or ADR/Architecture Requirements Document)." And then you go to sprint planning, and you break it down into little tickets, and you decide who's going to do what.

AI can help with all of this. If you're just using AI to write the code, you're missing out on a lot of the benefits that AI can bring to your SDLC. And a lot of people say, "Well, we don't need any of those meetings anymore because we have the loop. We have the dark factory. Things just fly around the loop." But it's like, "Okay, but if you want to actually move faster and maintain quality, then you should have these checkpoints before you go to actually write the code, and you should use AI to help with that."

So, we built this cloud platform that has a Google Doc-style component where you can comment, and the agent can surface mockups, Mermaid diagrams, HTML, and all these things. So, basically, how do we make agents like Figma style? Everything's in the cloud. Everything's collaborative. I see all my coworkers' sessions; they see all of mine.

It's almost like the benefit that Slack had over email was that you didn't have to be in every conversation to know what was happening. You could see all these channels light up. You could check on them. "Okay, I don't care about any of that." But if you saw a conversation that you cared about, you could jump in on that.

And it's: how do we do that for engineering work? Because we really had these very strict—even when we called it agile, it's very waterfall-like—PRDs, ADRs, tickets, everyone goes and builds for a day, and then you get the PR back, and then one person reviews it.

How do you create this more just like soup, and what is the data model for that world where you have agentic traces, you have documents, you have tasks and projects that group these things, you have actual git diffs being streamed everywhere? Why would I review all the code at once when everybody's work lives in a shared environment that anyone can go interact with?

What it reminds me of is what GitHub did for software teams. Before GitHub and its competitors, you might have a tracker somewhere, but most teams were just working inside the company. You didn't know what one team was doing. I remember pre-GitHub: you had individual teams, some of them had a board with stickers, but no one else in the company knew what they were doing. They were all working in isolation.

And now when you have GitHub (or even the internal version of GitHub inside a company), you can always see, when you go to a team, the pull requests flying. You can join in, you have history, it's all connected and came together. And now it's like, duh, you're going to use GitHub, or people will copy it.

So, do I sense that you're trying to build something like this workflow for when you have the software factories—which are like dark factories and loops at a bunch of places—how can we have this new way of working which will feel natural, but coming up with it is hard work and counterintuitive?

How can we do something that accomplishes what GitHub did, but like 10x better? More specifically, more continuous, more real-time, and more collaborative than these discrete units of work that is the pull request.

Well, I'm now starting to understand why you're saying maybe we should kill the pull request. Because the pull request was invented by GitHub, right? It is not part of Git, but they did it as a way for you to do a code review and merge before it goes in, and be able to modify it or just reject it, etc.

And it's probably a lot better than whatever we had before, which I guess was like emailing your git patch to Linus and asking him to merge it into the kernel or whatever.

They still do that! It works for them. That's the point, but it only works for them.

Yeah. I don't know anybody else who does that. I mean, I'm sure even before GitHub, you guys had what, like CVS or—

CVS? Or if you had a lot of money, Microsoft Visual SourceSafe. They made us use Subversion in undergrad because the guy who invented Subversion was a UChicago guy. The year after I graduated, they switched everybody to Git, and I was like, "Damn, I learned a useless thing just for somebody's ego."

Specifically for AI startups, or startups building on top of AI or building AI products: how important do you think location and network is? Especially since you are based in the Valley, we see research that AI startups are more frequently funded from here than normal startups as well. Do you see this advantage, and do you see some disadvantages of being in a specific area, whether that be Silicon Valley or elsewhere?

I don't have really strong opinions on this, actually. Paul Graham gave a talk in Sweden about why SF is cool, and rather than just regurgitate that, I will forward people onto that one—we can put it in the show notes or whatever.

But he talks about all of the dynamics of Silicon Valley, the pay-it-forward culture, and how people take you way more seriously just because you're based here. I lived in Chicago for a long time. I have a lot of really good friends from high school, from college, and growing up in LA. And never before have I felt so locked in with my people. Never have I felt more seen, more connected.

There's just so many people here. Again, talking about the founder thing: people who care deeply, who are incredibly competent, who... we have all the same types of problems, we love all the same types of things. Like, I don't do LAN parties where we play video games, but all my buddies will come over and we'll sit in the office till 11:00. We'll just do co-working and hack on cool, fun projects and stuff. And you can't do that anywhere else. There's not enough critical mass for that to just happen organically everywhere you go, and I absolutely love it. I wouldn't trade it for anything.

Yeah, I think critical mass nails it on the head. When it comes to hiring, what types of folks are you hiring for specifically? Because I'm interested in how hiring changes, what a standout engineer means for you, and how you are trying to confirm that those traits exist.

In general, we are looking for people who have really strong software fundamentals—so, understand distributed systems, understand the core fundamentals of CS and operating systems, and these kinds of things. I mean, you don't have to be a PhD in freaking kernel design or whatever, but it's a lot easier. We can teach somebody, I think, to be a really good AI developer in a few months. You can build enough intuition where you are accelerated off the ground, and you can keep growing there. It's really hard to teach someone a CS undergrad program in three months.

And what's a problem space that you're excited about in software engineering, or even product engineering or building products, that you think in the next few years is going to be one of the interesting things that you're going to be attacking?

My co-founder could talk more about this, but there's a lot of interesting things happening in real-time in cloud, sandboxes, sync, and using these new building blocks that have gotten really solid in the last couple of years. We're big fans of the ElectricSQL team, where users have durable streams.

It's like: how can you build systems that are a lot more spread out and distributed, and almost like decentralized? This is really interesting for coding because you want to be able to run coding agents anywhere. You want to be able to run them for a short time, for a long time, on demand, on a schedule, all these things, and have them all be part of this kind of like brain.

So, parts of what we're doing are really boring—like all our data is in Postgres—and then parts of what we're doing are really interesting. But there's a lot of distributed systems problems. There's a lot of infrastructure problems.

We are building tools for AI, but there's a lot of problems in building collaboration platforms that are really, really hard, and there's a lot of new tech that makes it easier and more interesting, but it's still far from an easy problem.

It sounds like what you're saying is the infra layers, to some extent, a new infra is being built, and it'll take some time. But it'll be just new blocks, and it will eventually become the primitives. Like for cloud, we have primitives already, but it took freaking a decade to get those together or more.

Yeah. You had AWS in what, like 2006, 2008? Yeah. And then you got Kubernetes a decade later.

Yep. And as closing, what's a book or reading that you would recommend? Something that you personally enjoyed.

Nowadays, we talk a lot about *Refactoring* by Martin Fowler—classic. I think it's because we spend a lot of time improving the design of existing code and trying to figure out how to get models to build code that is easy to maintain, easy to read, easy to understand, and easy to build on.

I feel like I probably have a better answer than that, but that's what's top of mind these days. We're reading a lot of classics of software engineering: *Refactoring*, *Clean Code*, *The Pragmatic Programmer*... all that stuff, I think, is more relevant now than it has ever been.

Love it. Well, Dex, thanks so much. This was fun.

This was a blast, dude. Thanks for having me on. This was great. I had a lot of fun. I don't know about you, but I really enjoyed this conversation.

Dex is such a big believer in agentic coding. Yet, he's the one warning us that if you stop reading the code, you have about three to six months before your codebase becomes easier to rewrite than to fix. And this comes from firsthand experience—his team built a light software factory, ran it, and then had to shut it down.

I also like the idea of the slow loop. "Loop engineering" feels like a somewhat meaningless term to me. What Dex's team does is actually pretty boring: a cron job runs every night, fixes one issue or one anti-pattern, and opens one small pull request. The team wakes up to a codebase that's a little bit better every morning, and devs still need to review and approve it. This is a practice that, honestly, any engineering team could just adopt today.

Finally, I really enjoy the history lesson. The term "software factory" comes from a NATO conference in 1968. The idea of software used to build software with analogies to a factory is more than 60 years old, and every generation of our industry has tried to automate more of the loop of building software. AI agents are just yet one more attempt, although probably the most successful one.

Do check out the show notes below for the related *Pragmatic Engineer* deep dives that go even deeper into AI engineering and other related topics. If you enjoy this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show. Thanks, and see you in the next one.