Hi, my name's Lauren. You might know me as potato on X, and I work on Grok Bot at xAI.

So, last month I did something pretty crazy: I shipped 2,000 pull requests to production. A lot of how I'm able to do this is through trust. And I think a lot about trust in terms of how I can trust my agents to produce high-quality work even when I'm not there.

And my argument and thesis for this talk today is that if you set up your environment for your agents really, really well, you can end up with something that looks more like a personal or even team software factory where you're producing very high-quality code at much greater rates than before.

Uh, but I'm not a fan of the term "software factory." I like the analogy of a Michelin kitchen better where, you know, as technologists, we're not mass-producing a product on an assembly line. But the work that we do looks very creative. It's the act of building a product, and it's art in some sense.

So, you know, even though with agents we're not cooking the individual components that go into the product anymore, we're still responsible for the final outcome and thinking about our kitchen setup. Right? Because depending on how you set up your line cooks, your sous chefs, you know, the kind of equipment they have, the kind of training they have, dishwashers, and the ratio of line cooks to dishwashers—all of these ingredients go into making the final product. And I think this analogy is really apt.

So, I wanted to talk a little bit about a story before we begin. Six months ago, when I first joined Cursor—before we were a part of xAI—I obviously had no agent skills to use, right? I just joined the company. It was a fresh codebase and a fresh product that I was working on.

At the time, Cursor was building the replacement for the Cursor IDE, which is the new agents window. And before I joined, the Cursor agent window had quite a lot of performance issues. My manager at the time asked if I was able to help them. And since I had spent some time on the React team before I joined Cursor, it seemed like a good fit.

But when I first started, I quickly realized how manual this process was. Obviously I've done performance work before, but at the rate at which pull requests were being landed, it felt almost like an insurmountable wall of pull requests that just kept coming in, and I had no idea whether or not the performance of the app would be regressing, right?

So, a lot of my early time on the Cursor team was spent looking at Chrome DevTools, doing performance traces, and taking heap snapshots. And it was extremely, extremely manual. It got to the point where I just got really frustrated and started to think about, "Wait, we have agents, what am I doing?"

And so, I started thinking about verification skills. Where, you know, what if my agent could actually run the application itself and take the traces for me, understand the traces, find the hotspots, and basically hill climb to better performance in our application automatically?

Throughout the last 6 months of being at Cursor and xAI, you can really see that my productivity has skyrocketed. And, you know, I never set out to ship 2,000 pull requests a month. That was not a goal of mine at all.

But I realized that all of the skills, all of the tools, and codebase changes I was making laddered up to this idea of trust. I didn't know it at the time, but I had this thought in the back of my head, which was: I am the bottleneck, and I need to be able to take all of the knowledge that I have as an engineer and impart them into my team of agents so that I didn't need to be the blocker for everything. And you can clearly see that it's paid off.

So, I think it really comes down to trust. But how exactly do you build that trust? And where do you start wherever you are in your journey of using agents?

When I started, obviously, I was in this category—the 1 to 5 range—where you still feel like you have to babysit every chat and conversation, and you're just constantly course-correcting. You're intervening, you're correcting your agent so that it does the right thing. And if you're not there, basically nothing happens and the agents do the wrong thing.

I would actually argue that this phase of using agents is actually the hardest to get out of because it's not always very clear how exactly you get out of it. And again, it just comes down to trust.

The reason you're unable to go from 1 or 1 to 5 agents to something more like 100 is because you don't have trust in your agents' work yet. So, if you don't have trust and you try to spawn 100 sub-agents or cloud agents, you're going to quickly find that you're just going to get a ton of slop pull requests, a bunch of regressions, and a bunch of bugs shipped, and no one's going to be very happy with that.

So that begs the question: how do you trust your agents more?

For me, it really started when I joined the Cursor team and was starting to work on performance, where I realized the need for verification. When I say verification, there are sort of levels to that.

On the lower end of the scale for verification, you have things like the verification skills that I talk about, where you teach your agent how to run your application and use things like the Chrome DevTools Protocol or whatever other protocol that you have for debugging, and teach them how to debug the application, take performance traces, take heap snapshots, and so on.

Then on the opposite end of the spectrum, which is much, much harder and still very much an open question, is more formal verification—where maybe you rely on formal methods or languages like Lean or TLA+ to do verification so that you can check that your business-level or business-logic invariants are always true, and that you can formally verify that your application is always in a correct state.

But I would say that even if you don't have the ability to run formal methods—and very few people really do—with verification skills, you can get very far.

When I joined Cursor and started to work on the Cursor agent window, the first skill that I built was a skill called ControlGlass, which is a verification skill that teaches the agent how to run the application and take traces, like I mentioned, and it does that through the Chrome DevTools Protocol.

Something interesting about that skill—and I kind of iterated my way to this, it didn't start out this way—is that the control or verification skill really has two components to it.

The first part is obviously a CLI. You want your agent to be able to reproducibly run the application and collect traces and collect empirical evidence that the code is working and that your performance bar is being met. Rather than have your agents create scripts every time that can differ between agent sessions, you can actually create a CLI that's within the skill directory, and then your agents will just use that every time. And then, of course, you need to actually invest in it and make it good so that it can handle all sorts of different use cases and be able to run the application correctly.

Another thing that's also really important is this idea of a feature map. A feature map really is something that I kind of coined, I guess. We started using these control skills within Cursor, and then we quickly realized that a Slack report would come in and a user would post a very vague screenshot—like a very small slice of the UI and just three question marks. And the agents using our control skills had just no idea. It could run the application, but it would just be guessing at what exactly the user meant.

And so, I had this idea to create something called a feature map, which is, I guess, kind of inspired by a sitemap. It essentially is a form of materialized memory: how exactly does your application work? What features does it have? How does a user reach it in terms of keyboard shortcuts or what DOM elements to click on, things like that, and what all the different features do.

This feature map is stored in the skill itself, in the codebase as part of the skill directory. And we have an automation that maintains this feature map as well.

When we combined the CLI and the feature map, we quickly realized that this combination was very, very powerful because now agents could not only reproducibly control the application and take traces, but it could also understand requests that came in from internal users as well as external users.

We very quickly realized that the control verification skills were so useful that they've become more or less critical infrastructure for our team, and we constantly maintain it. The ability for an agent to verify its own work is extremely powerful, and we have spent a lot of time on the skill, and it's very, very powerful for building that trust.

In addition to verification, of course, verification is really about correctness. Correctness to me is really about: does the feature or the code do the thing that you want it to do? Right? Like, does the checkout button actually check out the cart? Verification is really important for that because you can get empirical evidence that the feature actually works.

But it doesn't tell you much about the performance or the code quality of that feature. And that's where you start thinking about skills that teach agents to work like real software engineers.

I've built a plugin called Pystack. I'm not going to talk about the plugin too much today, but a lot of the inspiration for that plugin, which is a collection of skills that I've created, are inspired by the kind of workflows that I personally have used in my time doing software engineering for all sorts of different types of tasks: debugging, feature development, prototyping.

There's a whole bunch of different playbooks and skills that ship in Pystack that teach your agents how to write code the way that you want them to. And this is where the more experienced engineers on your team can really contribute to set up a team repository of skills that just make your agents a lot smarter.

When you combine those skills with verification skills, you're able to get to a point where your agents are able to not just verify that the work that they're doing is correct, but that it's also high quality. And again, because of verification, you can collect real performance metrics. You can get real numbers and statistics and telemetry on the performance of the application. So I think that's a really important part to invest in.

Another thing I think is really important is refactoring and rewriting your architecture to be more agent-friendly. I almost want to say that this is one of the most important things you can do as a software engineering team. Because if you really truly believe that agents are going to be writing all the code in the future, then we need to design our codebases so that they do the right thing by default.

Mhm.

And you'll find that there's a scale or continuum between these different pieces of building trust in your agents.

I have five of these points here. The codebase is really like the best form of memory because agents love to extend existing patterns that they see. I think this is just the nature of how LLMs work, where they're more likely to use whatever is in their context window to make changes. And of course, the files that the agents read and open are part of its context window. Therefore, the codebase is a really important part of that, because agents aren't going to just refactor your code in every single PR. They're going to just look at what's already there and extend it.

The next level, I think, is about static analysis, where you have linters, compiler diagnostics, and continuous integration. These are guidelines and constraints that you can enforce in your codebase so that whenever you correct your agent and you find that they just keep making the same mistake, you can add those as lint rules. Or even better, you can refactor your codebase so that the mistake that the agent is making becomes categorically impossible.

Then a step above that—and this is where we're starting to get less into hard constraints and enforcement and more into the realm of guidance—you have things like rules, BugBot, and skills, which your agents will obviously mostly use when they're doing their work. But there's also a chance that it might, for various reasons, forget to read a rule, or maybe the user that is piloting the agent ignores them. So these aren't quite as enforceable, but they are also an important part of setting up your environment so that you can really trust what your agents are doing.

And then finally, you have a style guide, which is really only enforceable by humans in a code review. I guess you could put these in your rules, BugBot, and skills as well, but if you don't, then you have this big glaring hole in your review process where now humans have to look at every single line that's being changed and remember the comment. With the rate of pull requests coming in, it just becomes impossible.

So I definitely wouldn't recommend relying only on the style guide. I think looking at human reviews is a good place to start in terms of what's missing, but you should really invest the time to think about the other four parts: your codebase, making things categorically impossible through better data structures or algorithms, static analysis, and then of course layering that with rules, BugBot, and skills.

On the codebase front, and in the Grok Bot codebase, we actually have invested into setting something up that we call Dune, which is our agent-friendly framework.

The inspiration for Dune really came about from a lot of the performance issues we were seeing in the Cursor agent window. A lot of lessons came out of that exploration, but the key principle that we landed on is really that agents love taking shortcuts. So what if we designed a framework such that the shortcut—the easy path—is the right path for agents? And also one that would be a codebase that is maybe pretty annoying for humans to work in because it's so locked down in terms of what you can do and what you can't do, but it actually creates the perfect environment for agents, especially ones that have very minimal context.

Because not every contributor to your codebase is going to be an engineer anymore. You can have designers, product managers, CEOs going into the codebase and shipping features. So we want to really think a lot about how we invest and set up our codebases so that even agents piloted by busy people with not a lot of context can do a good job by default.

Like I mentioned before, your codebase is really a form of memory for agents because they love to extend the existing patterns that they see. And the reverse is actually also true. You can invest the time to set up your codebase in a way that bad patterns are categorically impossible, or you have lint rules that prevent them. But the reverse is also true in the sense that if you have existing anti-patterns, you'll actually find that these will spread kind of like a virus.

You have one small workaround or a comment that explains a workaround, and you'll quickly find that agents just love to copy that. And then in a matter of a few days or a few weeks, you'll find that the workaround has spread everywhere, and it has become a de facto pattern for all agents. That's a really, really bad place to be in.

That goes back to what I was saying about why it's really important, whenever you're correcting your agents, that you invest the time into thinking about codebase changes and static analysis, and of course layering them with good rules, BugBot, and skills.

Another analogy that I like, in addition to the Michelin kitchen, is this idea that your codebase is kind of like a garden. You have workarounds that seem kind of innocent at first, but then because of the nature of agents, you just copy that pattern over and over again, and you quickly end up with a very vibe-coded codebase that is a pain in the butt to maintain and has a lot of performance issues.

In my opinion, the perfect agent codebase is one that's so locked down that it's really annoying for humans to write code in, but it's so conventional and so standardized that even innocent-looking patterns are just forbidden.

The best example of this I have is actually something that seems very, very innocent when you look at it, but when you think about it, it's actually really bad. And that pattern is agents leaving comments in the code.

When I first saw agents starting to do this, I initially thought—well, I did think that a lot of them were slop, but I also thought that it's not a bad thing if agents are leaving comments in the code. Because as humans, we left comments in the code whenever we saw edge cases, or we needed to make a workaround, or leave a note to ourselves or a colleague on a particularly tricky part of the codebase.

But what I quickly realized when we saw this happening in the Cursor codebase was that agents were just using the comments around the code as justification for why it wasn't going to solve the actual problem, and instead paper over it with a band-aid or a short-term solution.

So in Dune, which is again the framework that powers Grok Bot, we made the choice to actually ban comments for that reason, so that agents would not just copy that pattern and propagate it everywhere in the codebase.

My pitch here is that every team really needs a role that I'm calling a gardener. In the same way that with a real garden, you need someone who is thinking a lot about things that can creep in and grow in ways that you don't want—like weeds or unwanted pests and stuff like that that creep into your codebase—you want to nip them in the bud as soon as possible before they start propagating everywhere.

A lot of the principles behind Dune are really centered around these three things: First of all, we want to delete tech debt that we already have, for reasons I just mentioned. We want to enforce a single paved path for blessed patterns. There should be one conventional way to do things so that agents don't really need to guess, and there should be enough guidance in the codebase, in CI, and in lint rules so that the agents are guided to follow that path. And then finally, whenever you see tech debt or bad patterns, your instinct should be, "I need to write a lint rule against it." You don't always have to clean up immediately, because if you write a lint rule, you can at least stop the bleeding, which at least prevents it from growing.

So I definitely recommend really thinking a lot about how you can guard against anti-patterns so that they don't spread like a virus, and then also spend time to get your agents to clean them up so that your codebase is constantly kept in a state where you would be happy if an agent copied it. That's the kind of mindset that I would recommend having.

I won't actually go through all the details of Dune itself, but I'll just kind of gloss through some interesting parts. Again, as a reminder, Dune is the client framework that we built to power Grok Bot. We've invested a lot into conventions about where code should live and where and how code should be imported between them.

In Dune applications, there are different concepts: features are all co-located in a single folder; you have an entry point in the React part of the code that determines what you can kind of think of like a route; you have transcript cards that show up in the Grok Bot application; you have a host that runs on the Grok Bot virtual machine; and then, of course, you have your client which powers the overall Dune application.

We have a lot of strict boundaries between these things. Just as an example, things that run on the main process or the main thread in Electron aren't allowed to be run on the renderer thread. We keep that separation very intentionally because of lessons we learned from Cursor's agent window, where we would sometimes see slow code accidentally get imported into the renderer thread.

Since on the renderer thread you want your UI to be very smooth and performant, you need to make sure that you don't have any long tasks or things that take longer than 16 milliseconds if you want 60 frames per second, or 8 milliseconds if you want 120 frames per second. Your renderer has to be constantly in a state where you can really chunk up the work and not do it all at once.

We have code within Dune that enforces these boundaries through the import dependency graph. But yeah, this is just an example of a pattern that we saw lead to really bad performance that we categorically eliminated through the architecture of Dune.

All of these other pieces aren't that interesting, but again, the core theme here is not about Dune, but the idea that an agent-friendly framework of your own is actually very, very powerful. You can encode all of the learnings that you and the best engineers on your team have tribal knowledge of.

I think the lesson here is: how do you take that away from what used to be in the style guide process of reviewing code and leaving comments, to extracting that knowledge and encoding that into the framework and into the codebase itself so that the codebase acts as the memory?

Coming back to this idea that the codebase is the materialized snapshot of the state in which you want your agents to extend: you want that codebase to be so pristine, so great, that the next agent that comes along is just very likely to continue that pattern and keep it really, really good.

If you spend enough time on this process, like I mentioned, you can really set up a Michelin kitchen or a software factory where, because you spent so much time on all of these pieces that allow you to trust your agent—whether it's in the codebase, lint rules, diagnostics, rules, BugBot, or skills—these layers come together and provide you a lot of trust.

Just imagine for a moment you're working in the Grok Bot codebase. It's super locked down; it's almost impossible to write bad code. So even an agent with very little context and not a lot of reasoning can come in and actually write code that's good.

Going back to my example about the Michelin kitchen, there's a lot here where we're setting up our bots with skills and tools, we're training them, we're setting up our kitchen in a way that makes sense for the agents and bots to do the right thing by default.

Whenever we notice in the kitchen example that one of our cooks or dishwashers is constantly tripping over something, of course we need to fix that. We need to problem-solve and ensure that others don't trip as well, because a kitchen is a very dangerous place and you don't want to hurt yourself.

It's the same mindset, I think, that we should have with our codebases: how do we set it up so that even agents without a lot of knowledge can do a good job?

I think Grok Bot and Cursor play an interesting role together, where Grok Bot is really great at providing what I call the outer loop. You can connect Grok Bot to lots of different connectors—like Slack, Datadog, Sentry, PlanetScale, whatever services that you use. You can aggregate all of that information together and use that to make really good decisions.

Some people call this a company brain. I personally don't think you need anything that sophisticated here because agents are really good at using tools. If you connect these tools to Grok Bot and you start having your Grok Bots auto-kick off things like cloud agents, you can actually find that you don't really have to invest in a lot of infrastructure to build a software factory. In fact, I'm going to cross out this term because I don't like it.

I think you can set up this personal mission-control kitchen for yourself through Grok Bot—things like Grok Bot routines which let you subscribe to Slack threads and Sentry alerts that let you kick off things automatically.

When you combine all of these things that I've been mentioning—your codebase, your rules, your skills—they all compound. Grok Bot will be able to automatically respond to events that come from the outer loop and then kick off cloud agents. You can also set up Cursor automations and use our SDK to set up additional bots as well that reuse a lot of these pieces of agent infra that you've set up, allowing them to do much more complicated tasks.

If you've done all this, then you can get to a point where—I have some screenshots here of some of our automations and our agents that work on Cursor—we are automatically reproducing bug reports, we're automatically opening pull requests, and we are essentially adding a lot of value to the entire team because all of these things compound.

If we kind of zoom out again and go back to this graph, to close off the talk: if you spend a lot of time thinking about all of the pieces that you need to be able to ascend the trust graph, you start getting to a place where you can really trust your agents more, parallelize your work, and empower your entire team to build on top of these pieces of infrastructure for your agents. You empower every engineer on your team, every builder, to be extremely productive and be able to write high-quality code.

The last thing I want to leave you with is actually this piece: if there's only one thing you take away from my talk, it should be this slide here. These are the activities that will help you build up towards a high-trust environment.

Whenever you find yourself correcting and intervening with your agent, you really want to think about it from these five pieces, and where is the most effective step in this sequence in order to make your agent much more trustworthy.

I definitely recommend thinking about it in this order: you either invest the time to make that pattern categorically impossible through your codebase, architecture, and data structures; or you start looking at things like static analysis; and then you layer that on with rules, BugBot, and skills.

If you do all of that, and you also spend some time thinking about your code quality in terms of skills, you get to a place where you trust the environment so much that your agents can just be free. Personally, I have spent a lot of time on this for the Grok Bot codebase, for example. And this is really the secret—well, it's not really a secret, it's a lot of hard work.

I hope you found this talk useful. Please reach out to me on X, my handle is @potatowithe. And I hope that you'll have a lot of fun and success in your own Michelin kitchen. Thanks for watching.