One of the most intimidating things about using AI tools can be getting started. There’s a question of what to do with this genie in a chat bar. We can tackle that at a later date. Today, I want to talk about giving this genie the context it needs to make magic happen.
My apologies if this ends up being a little too inside baseball or dry. Building in proper context is one of those topics that everyone seems to love to talk about at a high level but rarely gets down to the tactical level that most people need when starting out. I’m going to try to do this, and in order to make it concrete I’ll use personal examples whenever possible. The legwork that goes into context building (as I see it) isn’t one-size-fits-all, but hopefully hearing what I do will give you a good sense of what you’ll want to get out of your own system. I think this is honestly such a big topic that I’ll probably do multiple posts on building a corpus for my agents, so let’s consider this a starting place.
First, the high level concept. LLMs train on vast amounts of data. This means they’re really really good at recognizing patterns (next token prediction). If we think about basic chat, this is why ChatGPT or Claude can shoot the shit with you like a person might. Turns out we’re surprisingly predictable. Harnesses (the software wrapped around the model — the thing that turns a bare LLM into ChatGPT or Claude Code) bring the next level of capability in: things like tool use, skills, and memory. Tools allow agents (I’m going to use this term loosely here to basically describe an LLM + harness package) to do external things like search. Search is really important because it lets an agent pull in information that it otherwise wouldn’t be able to predict from its training (“Who won the Mariners game last night?”). Skills are basically saved instructions that tell agents how to do things (a way to make these non-deterministic systems a little more consistent). And memory is kind of what it sounds like — retaining information from previous interactions in the hopes of making future interactions smarter.
Now that we have a simple model of how agents work, let’s talk context. When I say context I’m talking about the dataset that an agent can pull from in order to respond to your prompt. I think most of us became familiar with this concept when we needed to think of clever (or not-so-clever) workarounds to make the most of the limited “context windows” of earlier models. Easiest way to think about this is how long your chat can be without hitting severe degradation. Back when I was learning what AI could do for financial advisors at Zocks, the two biggest levers I had were what context we gave the model (transcripts, client data, etc.) and prompting (telling the model what sort of output we wanted…had to be very specific back in those early days).
Ok — so why did I call this piece “Building a world”? It’s because that’s how I think about making sure that my agent has enough context to give me answers that I would expect. Some of this lives at the prompt layer (claude.md is an example — a file of standing instructions that the agent reads at the start of every session). That’s where I give the agent context about what I expect across the board. For example, my claude.md file in my codebrain repo has everything from “This is Andrew’s personal thinking + memory vault. You are a thinking partner, not an autocomplete.” to “Read for intent, not literally — input is often speech-to-text with transcription noise. Don’t nitpick the wording back.” (Honestly, reading through it now is a good reminder that I need to clean things up a bit — this file should be evolving with your thinking and model capability.) There’s also the memory file system that holds key decisions (and why) as we work together. I think these two pieces (prompts and memory) are pretty well understood as important by most people if they’re moving beyond basic chat.
The third piece of context is what I see people neglect — not because it’s unintuitive, but because it’s a pain in the ass. And that’s giving your agent an example of what good looks like. Easy example of this is when we’d draft a follow-up email for financial advisors after a client call, we’d have a library of example emails. We had templates that we built in house along with giving advisors an option to make their own templates. Turns out if you want something specific, you should probably give a reference point. I think it’s pretty common startup advice to do a task or job function (think sales, customer support, etc.) before you hire for it. Not only will it give you a sense for what’s needed (are you going to rely on cold emails or in-person events) but it also means that onboarding your first hire in that role will be that much easier. They might be an expert and think what you did was trash, but at least they’ll have a starting point to work from. Agents these days are a lot like these new hires — incredibly smart and “expert” at a lot of things, but also lacking context around what you want/expect.
I promised personal examples earlier, so let’s start with writing. AI slop is here and not going anywhere. I hate it, you probably hate it, but people are still doing it. Why? Because writing kinda sucks (for a lot of people). It’s time consuming. Many of us have been told we’re shitty writers. Does it even matter? I’ve told myself I’m going to publish pieces here regularly…and frankly that’s a decent time commitment. It would be a lot easier to just have Claude and Codex bang out a bunch of drafts and pick the best one. Maybe I will get there, but at least for now I’m making myself write them. There’s the “writing helps you think” side of things, but I also want to show Claude what I sound like.
This isn’t limited to writing though. My VC friend keeps a log of all the deals he sees and why he made whatever decision. This has always been good practice, but it’s even more valuable when you have a superintelligence that can pattern match. We should all be looking for ways to build our own little datasets around not just what we’re doing but why or how. In order to build a corpus of work, you have to…do the work. And I think that’s why it’s such a neglected part of “building a world” for your agent to live in. When Claude can do everything, I would like to do nothing. Maybe Fable 5.1 will get us there. But for now, we have to do our part to guide our jagged little interns.
# Building a world.
> Prompts, memory, and the third piece everyone skips — showing your agent what good looks like. Follow-up emails, deal logs, and jagged little interns.
- written by: Andrew Schroeder
- edited by: Fable 5, GPT-5.6 Sol
- status: published · 2026-08-11
- canonical: https://www.moltolabs.ai/notes/building-a-world/
- raw markdown: https://www.moltolabs.ai/notes/building-a-world.md
---
One of the most intimidating things about using AI tools can be getting started. There’s a question of what to do with this genie in a chat bar. We can tackle that at a later date. Today, I want to talk about giving this genie the context it needs to make magic happen.
My apologies if this ends up being a little too inside baseball or dry. Building in proper context is one of those topics that everyone seems to love to talk about at a high level but rarely gets down to the tactical level that most people need when starting out. I’m going to try to do this, and in order to make it concrete I’ll use personal examples whenever possible. The legwork that goes into context building (as I see it) isn’t one-size-fits-all, but hopefully hearing what I do will give you a good sense of what you’ll want to get out of your own system. I think this is honestly such a big topic that I’ll probably do multiple posts on building a corpus for my agents, so let’s consider this a starting place.
First, the high level concept. LLMs train on vast amounts of data. This means they’re really really good at recognizing patterns (next token prediction). If we think about basic chat, this is why ChatGPT or Claude can shoot the shit with you like a person might. Turns out we’re surprisingly predictable. Harnesses (the software wrapped around the model — the thing that turns a bare LLM into ChatGPT or Claude Code) bring the next level of capability in: things like tool use, skills, and memory. Tools allow agents (I’m going to use this term loosely here to basically describe an LLM + harness package) to do external things like search. Search is really important because it lets an agent pull in information that it otherwise wouldn’t be able to predict from its training (“Who won the Mariners game last night?”). Skills are basically saved instructions that tell agents how to do things (a way to make these non-deterministic systems a little more consistent). And memory is kind of what it sounds like — retaining information from previous interactions in the hopes of making future interactions smarter.
Now that we have a simple model of how agents work, let’s talk context. When I say context I’m talking about the dataset that an agent can pull from in order to respond to your prompt. I think most of us became familiar with this concept when we needed to think of clever (or not-so-clever) workarounds to make the most of the limited “context windows” of earlier models. Easiest way to think about this is how long your chat can be without hitting severe degradation. Back when I was learning what AI could do for financial advisors at Zocks, the two biggest levers I had were what context we gave the model (transcripts, client data, etc.) and prompting (telling the model what sort of output we wanted…had to be very specific back in those early days).
Ok — so why did I call this piece “Building a world”? It’s because that’s how I think about making sure that my agent has enough context to give me answers that I would expect. Some of this lives at the prompt layer (claude.md is an example — a file of standing instructions that the agent reads at the start of every session). That’s where I give the agent context about what I expect across the board. For example, my claude.md file in my codebrain repo has everything from “This is Andrew's personal thinking + memory vault. You are a thinking partner, not an autocomplete.” to “Read for intent, not literally — input is often speech-to-text with transcription noise. Don't nitpick the wording back.” (Honestly, reading through it now is a good reminder that I need to clean things up a bit — this file should be evolving with your thinking and model capability.) There’s also the memory file system that holds key decisions (and why) as we work together. I think these two pieces (prompts and memory) are pretty well understood as important by most people if they’re moving beyond basic chat.
The third piece of context is what I see people neglect — not because it’s unintuitive, but because it’s a pain in the ass. And that’s giving your agent an example of what good looks like. Easy example of this is when we’d draft a follow-up email for financial advisors after a client call, we’d have a library of example emails. We had templates that we built in house along with giving advisors an option to make their own templates. Turns out if you want something specific, you should probably give a reference point. I think it’s pretty common startup advice to do a task or job function (think sales, customer support, etc.) before you hire for it. Not only will it give you a sense for what’s needed (are you going to rely on cold emails or in-person events) but it also means that onboarding your first hire in that role will be that much easier. They might be an expert and think what you did was trash, but at least they’ll have a starting point to work from. Agents these days are a lot like these new hires — incredibly smart and “expert” at a lot of things, but also lacking context around what you want/expect.
I promised personal examples earlier, so let’s start with writing. AI slop is here and not going anywhere. I hate it, you probably hate it, but people are still doing it. Why? Because writing kinda sucks (for a lot of people). It’s time consuming. Many of us have been told we’re shitty writers. Does it even matter? I’ve told myself I’m going to publish pieces here regularly…and frankly that’s a decent time commitment. It would be a lot easier to just have Claude and Codex bang out a bunch of drafts and pick the best one. Maybe I will get there, but at least for now I’m making myself write them. There’s the [“writing helps you think”](/notes/use-it-or-lose-it/) side of things, but I also want to show Claude what I sound like.
This isn’t limited to writing though. My VC friend keeps a log of all the deals he sees and why he made whatever decision. This has always been good practice, but it’s even more valuable when you have a superintelligence that can pattern match. We should all be looking for ways to build our own little datasets around not just what we’re doing but why or how. In order to build a corpus of work, you have to…do the work. And I think that’s why it’s such a neglected part of “building a world” for your agent to live in. When Claude can do everything, I would like to do nothing. Maybe Fable 5.1 will get us there. But for now, we have to do our part to guide our jagged little interns.