Klaviyo co-founder and co-CEO Andrew Bialecki came to SaaStr AI to walk through how a 2,300-person public company builds AI products. Not the vision deck. The actual build system.
Klaviyo won B2B for e-commerce with one insight: instead of showing merchants how many emails they sent and how many got opened, connect to the cart and to Shopify and show them the campaign made $18,372. Merchants still post those screenshots on LinkedIn years later. Changing what the dashboard reported took Klaviyo to roughly 80% share in the Shopify ecosystem, one of the only IPOs in the 2023 cohort, and the most beloved app I’ve come across in years. Talk to e-commerce merchants and they like a lot of tools. They love Klaviyo.
Now all of it has to be rebuilt for agents.
Klaviyo did $370.6M in Q2 ’26, up 26%, with 205,000+ customers and full-year guidance raised to $1.526B to $1.534B. Composer, the marketing agent Andrew describes below, hit 95,000+ users in its first month, about a quarter of them coming back weekly, with credit consumption growing 30% week over week. The first working prototype of that agent was built over a single weekend by other agents.
Andrew’s Top Takeaways
- Every employee has to be “L3” or they don’t survive this era. Klaviyo defined levels of AI autonomy the way self-driving cars are defined. L1 is using AI to search. L2 is spinning up a session and running an agent. L3 is constantly running multiple sessions or a team of agents. Everyone had to be L3 by the end of June, including PMs, designers, sales and marketing. Everyone commits code, from Andrew down to the summer interns.
- Use teams of agents to build your agents. Klaviyo’s internal system, “Dark Factory,” takes a prompt, acts as the PM, writes the specs, decomposes the problem into engineering subsystems, writes contractual API interfaces between them, then runs subagents against each piece. It builds through the weekend and interrupts with questions when requirements are ambiguous.
- The LLM is a great general athlete. The harness is the coaching. Klaviyo treats the base model like an athletic high schooler who could play any sport well. To make Composer great at marketing specifically, they feed it live signal on how consumers across the entire Klaviyo network are responding, and a “coach” agent scores every proposal on predicted engagement and revenue before it ships.
- Agents are power users on day one, and they sit to the right of your best human users. Software has a power law of user sophistication: a few experts, a long tail of novices who have an hour a week. Agents skip the curve. Onboarding matters less. What matters is what the agent asks you to build next.
- Headless is the default now, so your product is infrastructure. Anything a human used to log into should be treated as infrastructure, which means it needs APIs. Klaviyo is building a path to sign up, configure and pay without ever touching the UI, with a dedicated engineer whose only mission is that experience.
1. The L1/L2/L3 Mandate
The framework is borrowed from levels of driving autonomy, and it applies to every function.
- L1: I use it to search.
- L2: I spin up a session and run an agent.
- L3: I’m constantly running multiple sessions or a team of agents, decomposing a problem into pieces, and validating the output.
Klaviyo told 2,300 people to be at L3 by the end of June. Andrew says there was very little pushback, even though the slope is steep. His argument to the team wasn’t about Klaviyo. It was about them: very few people are going to get to L3 in the next year or two, and if you can put a team of agents to work, decompose a problem, and check and validate the output, you’ll be enormously successful in this next era whether you stay at Klaviyo or not.
The average PM at Klaviyo who was writing wireframes and specs pre-AI now has to hit L3. That’s a job redefinition applied to an entire org at once, not a tooling rollout.
2. Dark Factory: Agents That Build Agents
The name comes from lights-out manufacturing. You keep the lights on in a factory because humans are the ones fixing the machines. Automate enough of it and you turn the lights off, because the machines don’t need them.
Klaviyo started building Dark Factory last fall for a specific reason. Their early agent code looked like most agent code looks: prompts built one at a time, stacked on top of each other, an unmaintainable mess.
The loop works like this. You give Dark Factory a prompt, through a standalone repo or through Slack. It acts as the PM and writes out specifications. It decomposes the problem into engineering subsystems. It writes actual contractual API interfaces between those subsystems. Then subagents build against each contract.
For Composer, that decomposition produced an agent for creative and design that pulls from Canva and Figma and your own assets, an agent for orchestration that decides which segment gets what and when, and an agent for analysis that predicts what the campaign will produce.
Andrew’s strongest recommendation from the session: the clear contracts and interfaces between parts of the build are what make it work. They’re also what makes the human code review survivable at the end, because you’re not reading a tangled mess, you’re reading a system that’s laid out.
- It runs for the whole weekend. Klaviyo reviews progress every Friday, and a new idea doesn’t wait for the next sprint. It goes into Dark Factory Friday afternoon and there’s something to look at Monday. The first Composer prototype was one weekend’s run.
- Human-in-the-loop is continuous, not front-loaded. Instead of an all-in-one plan mode where every decision surfaces up front, Dark Factory raises the flag when it hits something uncertain. Across Friday, Saturday and Sunday you get a stream of small questions: your requirements weren’t clear here, specify this harder. Andrew thinks that’s how software gets built going forward, and it maps to how a real product review works with humans.
3. Tom Brady and the Coaching Layer
Andrew’s mental model for the base model: treat it like a very athletic middle schooler or high schooler. Good at a lot of sports. Not yet great at one.
Tom Brady got drafted to play baseball for the Montreal Expos. He might have had a fine career there. What made him great at football was years of coaching and tailoring: film of Joe Montana, and the specific drills you run to be a great quarterback.
The agent equivalent at Klaviyo is two things layered on the harness.
- A proprietary data feed. Composer gets real-time signal on how consumers across all of Klaviyo’s businesses are responding right now. That’s the film room, and a competitor can’t prompt their way to it.
- A coach that scores the work. Every time Composer proposes a campaign, it checks against a coaching agent that returns a numerical score for predicted engagement and revenue, plus feedback on how to tune it. The agent doesn’t ship its first idea.
An agent product built on a general model with no domain-specific feedback loop is a very athletic high schooler with no coaching staff.
4. Your Agents Are Your Most Advanced Users
Normal software has a power law of user sophistication. A few advanced users you put on a customer panel, and a very long tail of novices. Klaviyo has plenty of customers where one person has an hour or two a week for the software and that’s the whole budget, no matter how good the product is.
Agents break that distribution. They start as power users and, in Andrew’s framing, sit to the right of your best humans.
- Onboarding matters less. Agents onboard themselves off good documentation, plus whatever hinting you provide. They go 0 to 60 much faster than any human user.
- Ask your agent what functionality is holding it back. This is the most specific idea in the session. Klaviyo has been running Composer and asking it what it can’t do.
The example: about a decade ago there was a push to make email interactive, and Google shipped the AMP for email spec. Almost nobody used it, because building for it means building a web app inside an email payload. Klaviyo could only offer a raw HTML editor, and very few customers could figure it out. Composer figured it out on roughly day one. It came back saying it could improve conversion with carousels, dynamic product imagery and JavaScript calls, and that Klaviyo was missing the APIs it needed to specify content to the AMP spec. Can you fix that for me.
Same pattern in experimentation. Klaviyo dramatically leveled up its experimentation infrastructure because agents treat a test as nearly free. Throw it at 1% of the audience and see. A human user says “yes, I’d love to do that” and then never does it, because it’s a hassle.
Andrew’s rule: whatever you do to gather customer feedback, do the same thing with your agents. They’ll ask for what’s hard, and for what they want to do thousands of times instead of ten.
5. Agents Training Agents on the Customer Side
I pushed Andrew on this one, because it’s where a lot of agentic GTM products fall apart. Zendesk, which is adjacent, has essentially said their self-service support agent gets about 20% of the way there on its own, and getting to 80%+ takes forward-deployed engineers and humans. Klaviyo’s base is mostly SMB. Those customers can’t afford an FDE, and neither can you at 205,000 accounts.
Klaviyo’s answer is that agents train the agent:
- Take a feed of real conversations a business’s customers might have with it, or generate a synthetic one.
- Classify that feed into use cases, and pull representative examples.
- Hand it to an agent with full API access to the Klaviyo platform, and tell it to train up the customer-facing agent and not stop until the case works.
- The training agent interrupts when it hits a business rule it can’t infer. Refunds are the clean example: below or above a dollar threshold, some merchants don’t want the item back at all. So the agent asks, and it asks with context: businesses like yours usually have a rule here, what’s yours? Or it says there’s a backend system that prints the return label and I need to know where it is.
What gets delivered to the customer isn’t a blank agent. It’s an agent already trained on five or ten of their own use cases, already at a 50% to 70% resolution rate, before they’ve done anything.
Andrew’s sharpest GTM point follows from that: if an agent product can’t be tried out of the gate and requires a big implementation, it’s dead on arrival. The demo-then-decide-then-implement motion is going away. The wow has to be day one, then you tune.
He also thinks framing these as customer service agents is far too narrow. His view is that the outside agent becomes a kind of proto web server: you hit a URL, provide some context, and it renders output based on who you are and what you want to see. He expects most businesses to have their own agent deployed on a website, behind a phone number, over email, or reachable through an assistant like Claude by the end of this year. Klaviyo can’t get 205,000 customers there by teaching them all to build agents, and those customers can’t afford to hire someone to do it. So agents build the agents.
6. Great APIs Can Rescue Dated Software
This is the point I’d most want a B2B founder with an aging codebase to hear.
My own epiphany was going headless on Salesforce. I hadn’t logged into Salesforce since two companies ago, and it turned out I didn’t even have a seat. Amelia had to tell me. Now that we run it headless, I query Salesforce in real time constantly. Twilio is the public version of the same story: close to left for dead 18 months ago, growing 4%, now growing over 20%, because the API works with agents.
Andrew’s framing: if you previously built software a human logs into and uses, the right mental model now is infrastructure. Infrastructure is what developers use, and therefore it has APIs. Think that way and you get to the right questions about interfaces, actions and exposed surface, and everything downstream gets easier. You also stop needing your users to level up, because PhD-level agents will figure your product out and prefer it if it’s accessible and good at what it does.
So the simplest, highest-leverage improvement most B2B products can make right now, internal and external, is state-of-the-art agent-friendly APIs. Your product team can then build things they couldn’t have built a year ago, on the same codebase.
Klaviyo is building toward signup with your email or phone number and full use of the product without ever touching the interface. Someone says they’re starting a business this weekend and needs a place to store everything plus an email and SMS list, and it just gets done. There’s an engineer at Klaviyo whose only mission is making that experience great. Andrew expects a version where you never really know Klaviyo is there beyond the bill.
7. Codified Taste, and the Slop Problem
My worry with all of this, which I put to Andrew directly: if every PM can build a working feature in an hour or two, and you have 50 people on the product team, you’re not shipping a feature a quarter anymore. You could be looking at 2,000 half-baked, intellectually interesting features a month. The volume itself becomes the problem. Taste still matters enormously.
Klaviyo’s answer is to codify the taste. They took every product crit and every piece of feedback given over the last couple of years, Zoom meetings and raw notes, and dumped it into a database for an agent. Now, before anything reaches a review, there’s a stated bar: here’s what has worked, here’s what’s good, here’s what won’t pass.
Two things keep that from calcifying. People can push back on the rule set, and the team now argues from first principles about what makes a great product. And it’s a living document with real input from Andrew, not tribal knowledge that only exists if you were in the room. His point: a taste standard that works only when “you know it or I know it” is silly, and an interface to that knowledge is a big plus-up because more people can reach it.
Andrew thinks taste itself gets codified further: our product should work this way, feel this way, produce these outcomes. The hard part today is decomposition, having the agent explain its work so you can find where the logic failed.
On whether Dark Factory stays in-house: his belief is that the gains go to companies that stay on the bleeding edge, and the bleeding edge is expensive because you have to invent the patterns yourself. Klaviyo keeps investing until something open source or commercial does it clearly better. He isn’t attached to owning it.
The 5 Mistakes Andrew Says Teams Are Making
Andrew didn’t present these as confessions. They’re the failure patterns he flagged repeatedly, and every one is common at companies that think they’re doing this well.
1. Letting agents touch anything that matters. Everyone has seen the “it dropped my database” stories. Klaviyo architected the agent environment first: tight access to specific tools, hard limits on what it can build against their infrastructure, real sandboxing and staging so prototypes can go live somewhere you can look at them without risking internal systems. Getting the agent running is easy. Figuring out where it gets stuck is the hard part, and you can’t do that debugging in production.
2. Stopping at the demo. Draw two circles, then draw the rest of the owl. Andrew says the prototype-to-production gap is what they obsess about. At 205,000 customers across many languages, with businesses that have 100 customers and businesses that have 100 million, the same agent has to work and stay performant for all of them. Klaviyo wrote agents that validate the software and run load tests to find where things break at scale. An agent that works for the use case you demoed has finished the easy 20%.
3. Shipping the raw model as your product. No coaching layer, no proprietary feedback signal, no scoring before output. That’s a great athlete with no drills, and it’s what most agent products currently are.
4. Keeping fuzzy interfaces between teams and systems. Humans tolerate ambiguity and schedule a meeting to work it out. Andrew’s view is that’s a bad way to work for humans and it flatly doesn’t work with agents, because agents want hard rules. Klaviyo applied the software contract pattern to job design itself: the people team mapped out what a marketer or a sales engineer actually does in contractual terms, then asked how you scale that function 2x, 5x, 10x with agents instead of headcount.
5. Building for a human to log in. If your mental model is still “screens a person uses,” you’ll underinvest in the APIs, and the agents that could have been your best users can’t reach you. Headless is the default now, software is the infrastructure, and the intelligence layer on top is where the value moves.
If You’re Not Klaviyo
Most founders don’t have 2,300 people or a real-time feed of global consumer behavior. Most of this still transfers.
The Dark Factory pattern is decomposition plus contracts plus subagents plus interrupt-driven questions. That works at three people. Asking your agent what functionality is blocking it costs one prompt. Dumping your accumulated product feedback into a database and making it the pre-review bar is a weekend of work. Deciding that a customer’s first experience with your agent has to be a working agent trained on their own data, not a blank one, is a product decision, not a scale advantage.
The API point costs the least and returns the most. If your product was hopelessly dated a year ago, great internal and external APIs may be the fastest route back to parity, because that’s the only surface the agents care about.
