After a long week of coding, you might think that San Francisco builders would retreat to the mountains, beaches, or vibrant clubs of the Bay Area. But in reality, the week ends with an AI hackathon.
In recent years, San Francisco has seen an explosion of AI hackathons. On Saturdays and Sundays, techies talk about the latest advances in AI, networks, and most importantly, build their ideas into working demos. Hackathons sometimes offer prizes in the form of cash or cloud credits, but the real winners walk away with a sign of a startup.
“There’s no better place in the world to build your most ambitious project than San Francisco,” says Agency co-founder Alex Reidman. “You see a lot of competitions like hackathons, but they’re not competing against each other. They’re as collaborative as they are competitive.”
At a San Francisco hackathon last summer, Reibman decided to try his hand at building AI agents that could scrape the web. Agents are a hot topic in Silicon Valley as the AI boom is at its peak. The term is not precisely defined, but it generally describes AI-powered bots that can perform tasks automatically using interfaces and services that were not originally designed to be automated. It’s a kind of replacement for routine tasks that used to require human intervention.
But Reibman immediately ran into problems. “They were a mess,” Reibman said in an interview. “Agents failed 30 to 40 percent of the time, and often in unexpected ways.”
To solve this, Reibman’s team built an internal debugging tool to see where the agent was going wrong. They ended up getting the agent to work a little better, but the debugging tool itself stole the show and won the hackathon.
“I started showing the tool at a number of hackathons and events around San Francisco, and people started asking me to make it available,” Reibman said. “That was basically the validation that I needed: Instead of building agents ourselves, we need to build a tool that makes it easier to build agents.”
So Reibman, along with co-founders Adam Silverman and Shawn Qiu, started Agency to provide tools to observe what AI agents are actually doing and figure out what’s wrong. A year later, those tools ultimately became Agency’s core product, the AgentOps platform, which is now used by thousands of teams every month, Reibman told TechCrunch. The startup has now raised $2.6 million in pre-seed funding led by 645 Ventures and Afore Capital.
Adam Silverman, COO, tells TechCrunch that AgentOps is like “multi-device management for agents,” analyzing everything agents do to ensure they’re not doing anything nefarious.
“You have to figure out if the agent is going to be a bad guy and what kind of constraints you can put in place,” Silverman said in an interview. “A lot of the work is to visually see where the guardrails are and make sure the agent is following them before you put it into production.”
The startup is working with Cohere and Mistral, AI model developers that provide agent generation services, to enable customers to use the AgentOps dashboard to see how agents interact with the world and the cost of each agent. Agency is model-agnostic, meaning it works with a variety of different AI agent frameworks, but integrates with popular tools like Microsoft’s AutoGen, CrewAI, and AutoGPT.
In addition to the AgentOps dashboard, the Agency also offers consulting services (Reibman was previously at consulting firm EY) to help companies get started building their agents. The Agency wouldn’t disclose the names of its clients, but did share that hedge funds, consultants, and marketing firms are using its tools.
For example, Reibman says the agency helped clients build AI agents that write blog posts about the companies they work with. Now, those same clients use the AgentOps dashboard to track the performance and costs of their agents.
Major players like OpenAI and Google are likely to build their own agent products in the coming months, and AI startups like Agency must figure out how to work with these advances, not against them.
“There are so many layers in the stack that it’s unlikely that an LLM provider will try to capture all of them,” Reibman said. “OpenAI and Anthropic are building agent builders, but there are a lot of layers surrounding it to make sure you have a production-ready code base.”
