Of all the debate about AI’s potential downsides, there’s one worry that’s causing the most worry among AI enthusiasts in Silicon Valley. Their fear is that giant AI labs selling proprietary models are somehow acting like Trojan horses.
The problem is that as startups and enterprises use AI models from labs like OpenAI and Anthropic, those labs will increasingly have access to those companies’ most sensitive business information. The modeler can then leverage that knowledge on his own, potentially becoming a competitor for the customer. Those issuing such warnings range from VCs like Jason Calacanis to Palantir CEO Alex Karp.
Now, in a surprising blog post published on Sunday, Microsoft CEO Satya Nadella has joined this crowd. Nadella warns that AI users (“buyers,” as he calls them) are paying twice as much. They intentionally spend money to use AI tokens, but they also unknowingly hand over valuable data in the process.
“You are essentially paying for intelligence twice: once in money and once in something much more valuable: proprietary knowledge that must be revealed to make that intelligence useful. The better you want your model to perform, the more knowledge you have to provide!” he writes
The most dangerous thing is that companies are literally teaching their models about the nuances of their business.
“Models learn from ‘burnout,’ the prompts people write, the tools agents use, and especially the corrections people make when the model goes wrong. All corrections are refined by institutional know-how,” he wrote.
This is “the kind of knowledge that competitors will never be able to buy,” and yet companies are handing it over.
Nadella argues that if AI companies are free to scrape the Internet to train their models, it’s only fair that the companies study or “refine” those models in return. “Distillation” is the practice of using a model’s own output to learn how it works and training new, less expensive models based on these insights. In February, Anthropic accused the Chinese open source model of sending millions of messages to Claude as a way to improve its own model and called on the U.S. government to crack down on export controls.
Nadella’s point is that modellers can’t have it both ways. It’s hypocritical to freely train on the world’s data while restricting others from doing the same with their own models.
“We need great innovation coming from model providers with fair use rights to train models on public data, but I think it’s ironic that the status quo is being reversed and imposing restrictive conditions on distillation,” Nadella wrote.
Nadella is particularly concerned when modelers “reserve the right to learn from customer usage and interaction data.”
Nadella’s solution is the kind of solution proposed by the CEO of a large cloud provider. He wants companies to “retain ownership” of their data, including prompts, feedback and more. He therefore urges you to build your own “proprietary learning environment” in the cloud (where your data is likely already stored, which for convenience could mean Azure, Microsoft’s cloud). He also wants companies to build what he calls an “orchestration layer.” This is essentially a way to easily switch between AI models from different providers without being tied to one. Tools like AI “gateways” that allow companies to do exactly this are becoming increasingly popular.
Nadella never uses the word “open source” as a way to maintain ownership, but that’s a clear subtext. But there is another subtext.
In addition to using the cloud, large enterprises that still have their own data centers are already moving to an open source model installed on their own premises (“on-premises” in industry jargon). Idit Levine, founder and CEO of Solo.io, which makes networking and security software that helps businesses manage AI systems, says he’s seeing exactly this shift with his customers. After experimenting with proprietary model builders, they start asking themselves: “Can I take an open source model and run it on-premises? It will do almost 90% of what the big companies do, and it will cost a lot less,” she tells TechCrunch. “They understand it and they can control it.”
Solo.io’s technology was selected last year to support the Linux Foundation’s Agentgateway project. Her company counts companies like T-Mobile, ADP, and SAP among its customers. She sees companies increasingly installing on-premise open source models and sees this as the next big wave of enterprise AI usage.
She is not alone. Both Vercel (best known as a website building and hosting platform that recently added an AI model conversion tool) and OpenRouter (a company that helps developers route requests between different AI models) are seeing a surge in traffic to their open source models. In fact, the open model accounted for 29% of all traffic routed through Vercel’s gateways last month.
This trend is likely to continue to grow, with the CEO of Microsoft, a company that has invested in both OpenAI and Anthropic, now publicly urging companies to be careful about using proprietary models. “Consuming intelligence is like creating intelligence, and what you create must be yours,” Nadella wrote.
If you purchase through links in our articles, we may receive a small commission. This does not affect our editorial independence.
