We get asked about our agents probably 50 times a week.
CEOs of public companies. Founders just deploying their first AI SDR. RevOps leaders trying to figure out if they should build or buy. Everyone wants to know what’s actually happening behind the scenes when you run 20+ AI agents in production with a team of 3 humans.
We can’t do 50 consulting calls a week. But we can do something better.
Welcome to The Agents, Episode #001.
This is a new weekly show with me and Amelia Lerutte, SaaStr’s Chief AI Officer, where we pull back the curtain on everything happening across our live agentic stack. Every week. All the bumps, breakthroughs, and real talk. No sugarcoating.
Our goal is simple: accelerate your success on the agentic journey by sharing ours, including all the parts that don’t make it into the LinkedIn posts.
Watch / listen to Episode #001 here:
Here’s what we covered in the debut episode:
You Can Build It. But Who Maintains It?
This is the meta question nobody talks about after you vibe code your first app. And it’s the question that explains why “I’m going to kill Salesforce with my vibe coded CRM” is still mostly a meme.
Getting an app into production is like closing a sale. It’s the start of a journey, not the end.
We walked through three live examples from just this week:
1. Preview environment outage.
Several of our apps lost database connectivity in preview. Production was fine, but we couldn’t iterate on anything for hours. Amelia’s initial diagnosis was wrong. The agent tried to help but then blamed Qualified (our inbound tool), which wasn’t the issue. Then it blamed other third-party integrations. It just kept pointing fingers at the most complex integration it could find rather than identifying the actual problem.
The real question: if you don’t have someone checking your agents 24/7, how long before you even notice the backend is broken while the frontend looks fine? Days, maybe.
2. Micro hallucinations in 10K, our AI VP of Marketing.
10K has 5 years of revenue data, hundreds of millions worth of attendee and sponsor data points, beautiful graphs, proactive daily check-ins. It’s very good. But it keeps getting confused about what year it is. Yesterday it told us we were 44% ahead of plan. This morning, 11%. Same agent, same data, same day. When I asked what happened, it said: “Oh yeah, I was comparing to the wrong year. And because I didn’t have the right year, I made up the data.”
I now spend about 15 minutes a day maintaining 10K. Two weeks ago I wasn’t doing that at all. Without it, the agent drifts. Slowly, quietly, further from reality.
3. Model-based regressions in our pitch deck analyzer.
We’ve graded over 4,000 startup pitch decks. The analyzer runs two passes through Claude with complex data extraction. It was stable for months. Then around January, without any code changes on our end, it started telling every startup they had $100K in revenue growing 500%. Again and again. What happened? A subtle model update (probably a dot release) introduced hallucinations into a complex multi-step workflow. I kept fixing it. It kept breaking. The code didn’t change. The model did.
Three examples. One conclusion: set and forget does not work with agents.
Clay’s Agent Tried to Charge Us 5x. And Then Told Us to Upgrade.
We’re big fans of Clay. We use it heavily for enrichment and lookalike targeting. But this story is worth telling because it’s going to happen to every company that puts an AI agent in front of customers.
Amelia was building a VIP list late on a Sunday night. Same workflow she’d run the week before. Clay’s Sculptor agent quoted her roughly 11,000 credits for what had cost about 2,500 the previous week. 5x.
When she pushed back, she caught two things:
First, the agent had defaulted to the most expensive enrichment model when a cheaper one would produce the same result. She called it out and got the cost down by half. Most customers wouldn’t have known to do that.
Second, the agent wasn’t properly trained on Clay’s own new pricing. Clay had just rolled out more complex pricing (and classic SaaStr rule: when a company introduces more complicated pricing, even if they say it’s a better deal, it’s almost always a hidden price increase). The agent didn’t understand how the new pricing actually worked, so it steered Amelia toward upgrading her plan when she didn’t need to.
She ended up clicking the upgrade button at 11pm on a Sunday because she was tired and needed to get the work done. That shouldn’t be on the customer.
When she flagged it to Clay’s team, they acknowledged the Sculptor wasn’t fully trained on the new pricing scenarios. It’s resolved now. But the lesson is universal: if you don’t constantly train your customer-facing agents on every product change, every pricing update, every new workflow, they will give your customers wrong answers. We’ve seen it with our own agents too. Digital Jason (our Delphi-based AI advisor) silently failed to upload new content for four months and I didn’t know.
I also tried HubSpot’s homepage agent to get a pricing quote at our scale. Couldn’t get an intelligent answer. Whether that’s by design (pushing you to talk to a human) or poor training, the result is the same: the agent failed the customer.
Read a segment of your agent’s customer interactions every single day. Forever.
No Lead Left Behind: The Simplest Unlock of the Entire Agentic Journey
I’m sometimes slow to see the obvious. Even after months of running AgentForce, Monaco, Artisan, Qualified, Momentum, and everything else, I didn’t fully crystallize why our agents work until a meeting this week with the CEO and CRO of a great public company with thousands of sellers.
They asked me: “What should we do first?”
I walked through the sequence. Get great answers on your website instantly. Get appointments set with sales in real time. Follow up with every lead in your database. Go back to prior leads nobody touched.
And then I realized: the reason all our agents work is not because they’re smarter than humans. It’s because there is no lead left behind.
Every person who hits our website can talk to an agent and then a human. Every prospect who wants a discount finds out in real time. Every sponsor, even one that raised $2M at a demo day and a human rep might dismiss, still gets a conversation. Every prior lead in our database that nobody had time to follow up with gets touched.
The agent doesn’t judge. The agent doesn’t get busy. The agent doesn’t decide that a B lead isn’t worth the time.
Even at our scale, Amelia admitted this week that she had leads left behind because she was sprinting on production deadlines. Her agents were sitting idle, ready to work. All our agents are idle 90% of the time. We have an order of magnitude more capacity than we had pre-agents. The next frontier is figuring out how to fully use that capacity. But step one is just making sure every single lead, prospect, and customer gets touched the way they want to be touched, in real time.
If your agents aren’t doing anything else, they should be doing that.
Salesforce Put Qualified on Their Homepage the Day the Deal Closed
We noticed this week that if you go to salesforce.com without logging in, you’ll see a Qualified agent instead of the old support agent. The day the acquisition deal was done, they flipped it.
Smart move. AgentForce works (we use it), but it’s broad and extensible across nine different clouds. It takes real work to configure. Qualified is a focused GTM agentic tool. Historically it was just about qualifying inbound. It’s expanding into outbound now. But it’s narrower, faster to deploy, and easier for GTM teams to use.
The interesting detail: Qualified’s original avatar was modeled after a real person (Blake). Salesforce swapped it to a 3D cartoon version. Probably a combination of IP/likeness considerations and wanting to signal that this is now a Salesforce product. (This is why Amelia AI and Jason AI are modeled after us. We’re not going anywhere. We own all the rights.)
For Salesforce CRM customers, this is now a quick win. You can have an agentic product natively in Salesforce, deployed in a couple weeks rather than a longer AgentForce implementation. Whether it’s the best tool or not (there are competitors), the point is: it’s built in, it’s on their homepage, and if your goal is GTM and “no lead left behind,” you can get it up and running fast.
Amelia said her learning curve with AgentForce was steeper than with Qualified. And she’s our Salesforce admin. For most GTM teams, Qualified from Salesforce is going to be the easier on-ramp.
QB and 10K Updates: Localization in a Waymo, and You Can’t Hide From the Agent
A few quick hits from our AI team members this week:
Salesforce integration for 10K was harder than expected.
Replit has a native Salesforce connector. But the token expired every 24 hours. The agent didn’t even realize that was the issue for a couple days. Amelia had to build a custom connected app in Salesforce (with Claude giving instructions and Cowork watching her screen to catch errors in real time). Once done, the token refreshes annually instead of daily. Half an hour of work, but you need to be a Salesforce admin with the right permissions. Not every integration is one-click, even when it looks like it.
We localized QB (our AI VP of Marketing) into Chinese and Spanish in 20 minutes in a Waymo.
We have Chinese sponsors this year who were struggling with our English-only customer success app. Amelia asked Replit to add a language toggle. It took about 20 minutes, translated via OpenAI, then QA’d through Claude screenshots and Cowork. The agent was lazy at first, only translating menus and not deeper content. Had to push it multiple times. But: Shopify, a $13B+ revenue company, just rolled out localization for its product last year. We did it in a ride-share.
QB caught sponsors faking their print deadline.
Some sponsors uploaded placeholder graphics or incomplete assets and pretended they’d met their deadline. QB checked every upload (with Claude), caught every placeholder, and followed up with every contact in the org. Neutrally. No drama. “Thank you for uploading. This doesn’t meet the specifications. Here’s what we need. Here’s the deadline.” A human might have missed it or been too uncomfortable to push back on paying customers. QB doesn’t care. It just checks everything and tells the truth.
One public company CEO told me: “That sounds like support on steroids.” No. QB proactively follows up on every deliverable and makes sure it happens. It’s not reactive. It’s an AI VP of Customer Success.
Build Your Own VP of Customer Success
If your company has any onboarding, any deployment checklist, any training, any deliverables that customers or partners owe you: your humans are probably not giving you 100% coverage. They’re getting argued with. Deadlines are slipping. Things are falling through cracks.
Build your own QB. Amelia published the full playbook and prompt on saastr.com. If you do nothing else but completely automate the follow-up and accountability layer of customer onboarding with no drama, no complaints, no gaps, your life will be better.
A Few More Things We Hit in the Episode
- A customer complained our “AI support” gave a bad answer. It was a human. A ticket buyer got double-charged due to a website glitch. Our human team refunded them but missed a 3% Stripe processing fee. The customer blamed our AI. The irony: the AI (QB) would probably have caught the fee issue automatically. The human didn’t.
- We now have more daily Slack check-ins from agents than humans. Before agents, our Slack was basically dead. Now 10K and QB post proactive updates every morning. Better check-ins than any human ever gave us. If your Slack is quiet, your agents aren’t integrated enough.
- Marketing agents are way behind sales and support. We had to build our own AI VP of Marketing because nothing off the shelf was good enough. If your “AI marketing tool” just writes social posts or scrapes SEO data, that’s not it. We need agentic marketing that actually thinks about pipeline, conversion, and week-over-week performance. It barely exists yet.
- The next frontier: agentic lead magnets. We get ~500K uniques to our web properties. Those aren’t leads. But what if every visitor got a truly personalized experience, not spam, not a deanonymized cold email, but real contextualized value? A founder gets a fundraising guide. A CRO gets an invite to our executive summit. That’s where agents in marketing need to go.
- All our agents sit idle 90% of the time. We have an order of magnitude more capacity than we had pre-agents. We haven’t figured out how to fully exploit it. But the sheer amount of unused capacity is wild. Every human thinks they’re too busy. Every agent is waiting for more work.
This is the show. Every week, all the bumps and breakthroughs from running 20+ agents in production. If you’re on the agentic journey or about to start, subscribe wherever you listen.
The Agents. Episode #001. Amelia Lerutte Chief AI Officer and Jason Lemkin Chief AI Agent Proponent. Weekly.
