Let me begin with an important disclaimer: my accountant is an honest person.
I feel obliged to say this because, should this article ever end up in the wrong hands (I’m looking at my friends over at the tax office—warm regards), I want the record to be perfectly clear. She’s honest, highly competent, and probably far more reliable than I am when it comes to tax matters.
And yet.
There’s a part of me that, whenever Google Maps tells me to turn left in 200 meters, obeys without the slightest hesitation. I never stop to wonder whether the GPS has hidden motives, whether it’s trying to ruin my day, or whether it’s simply having a bad one itself.
While writing NOISE:77, I found myself thinking about this strange contradiction.
Ivan, my protagonist, lives in a world where trust is both rare and dangerous. Trust the wrong colleague, and you risk the Gulag. Trust the wrong superior, and your career is over. Trust the system, and you become complicit in its lies. So Ivan trusts the machine.
The BESM-6. Because a machine has no political agenda. It doesn’t fear consequences. It has no reputation to protect. Here’s the interesting—and slightly unsettling—part.
That logic only works as long as machines are simple tools.
The machines we use today are no longer tools.
They’re systems. And that’s an enormous difference. A tool does what you tell it to do. A system does what it has been optimized to do—which can be, and often is, very different from what you actually want.
Instagram’s feed has been optimized to keep you scrolling. Not to make you happier. Not to give you accurate information. Not to help you make better decisions. To keep you scrolling. Everything else is a side effect.
A credit-scoring system has been optimized to estimate financial risk. But “financial risk” is itself a human construct, carrying all the assumptions and biases of the people who defined it.
The truth is, we almost never know what the systems we rely on have actually been optimized for.
And when we do find out, we often don’t like the answer.
So yes, I trust certain algorithms more than certain human beings. But only the algorithms whose motivations I understand. The others I’ve learned to distrust just as much as I distrust the most opaque people.
Which brings me back to my accountant.
At least I can look her in the eye when she’s explaining something I don’t understand.
