Your Corrections Are the Product
Nadella's argument turns on a mechanism most enterprise security teams never audit. Models improve from what he calls exhaust - the prompts employees write, the tools an agent reaches for, and above all the corrections people make when the model gets something wrong [1]. Each of those corrections quietly encodes something proprietary: what your organization measures, where its processes break, and how it defines good. That is knowledge a competitor could never buy off a shelf, and Nadella's point is that it leaks trace by trace, correction by correction, eval by eval [1].
The asymmetry is the sharp part. As The Register summarized it, the seller learns more and more about you as you use what you purchased, while you learn very little about what the seller is learning in return [2]. Because the leakage happens at the level of model exhaust rather than a database export, it sits below the layer most legal and security reviews are built to inspect - which is why Nadella frames it as a risk hiding in plain sight rather than a headline breach.


