Unsurprisingly, (Cohn 2009) shows that people who have more resilience are actually more happy, and those who endure hardship with greater inner elasticity often find themselves more capable of joy. Happiness, then, is not only the absence of adversity, but a way of inhabiting it differently. The happier person is often the one who can still discover play within difficulty, who can cast a gentler light over misfortune, and who, without denying suffering, learns to transform it into meaning, beauty, and so. #Modus Vivendi

I've been turning the "human-in-the-loop" essay over in my head all evening, and I think the reason it irritated me earlier without my being able to name why is that the piece is half right and frames the wrong half. The author is right that something is being eroded. What is actually being eroded, the thing that matters and that nobody in those essays seems willing to name plainly, is the transmission of tacit knowledge.

For seventy years, since roughly the moment a non-trivial number of people sat down to write software for a living, the field has run as an apprenticeship culture. Not formally, there are no guilds, he universities did their best but most of what they taught was either wrong or out of date by the time the student arrived at his/her first standup, but functionally. You learned why you don't roll your own crypto because someone older than you, on a code review at two in the morning, pulled up your diff and asked, with that specific tone, whether you were perhaps unaware that the entire history of academic cryptanalysis existed. You learned about idempotency because the first time you wrote a webhook handler that wasn't idempotent it duplicated forty thousand transactions and a senior named [name redacted] walked you through the postmortem the next day with the kind of weary patience that is itself a pedagogy. You learned that a UNIQUE index on a nullable column does not do what you think it does because someone showed you. You learned where the bodies were buried because the people who buried them were still in the room.

The model now contains a lossy compression of all of that yelling. Every code review that ever made it onto a public mailing list, every postmortem Cloudflare or Stripe or GitLab ever wrote, every Stack Overflow answer where someone in 2011 explained patiently to a stranger why their migration would deadlock, all of it is in there, averaged out, smoothed, retrievable in a polite paragraph on demand. This is, in a narrow sense, an extraordinary achievement, and I will not pretend otherwise.

But the model cannot yell back. It cannot watch you make the mistake, register that you have made it, decide that this particular junior needs to feel the embarrassment of having made it in front of three of his colleagues, and apply that embarrassment at the precise moment that will inscribe the lesson permanently. It can only answer. And juniors who only ever get answered will not become the seniors who can tell when the answer is wrong, which means that in fifteen years the only people on earth capable of auditing what the model produces will be the cohort that grew up before it, the cohort that already has the whiteboards in their heads. After that cohort retires, nobody.

I will be honest about it because the alternative is the kind of selective autobiography that makes these essays unreadable. Last week I used an LLM to help write a Postgres migration. The migration was syntactically perfect. It added a non-null column with a default to a table with about forty million rows, in a single statement, inside a transaction. The model did not warn me, because the model's training data is mostly people writing migrations in tutorial blog posts against empty databases, and in that distribution the statement is fine. In production, on a table that is being written to concurrently, that statement takes an ACCESS EXCLUSIVE lock and holds it until the default has been backfilled across every row, which in our case would have been somewhere between four and nine minutes of every write to that table queueing up behind it, and by minute two the connection pool would have been exhausted and the application would have been down. I caught it because, back in the day, a senior at a previous project I worked on, a man named [name redacted], stood at a whiteboard and drew me the lock graph for exactly this scenario and made me explain it back to him until I got it right. The model did not have that whiteboard.

Nobody currently being trained has that whiteboard either, unless they happen to work somewhere that still has a [name redacted], and the number of places that still have a [name redacted] is going down, not up, because the economic argument for keeping a [name redacted] around when you have an API that produces [name redacted]-shaped output for a fraction of the salary is one that the people who sign the cheques find very persuasive.

I want to be careful here because the obvious move is to escalate this into a civilisational alarm, and I don't think that is honest. The bargain on offer is more output now in exchange for less competence in 2040, and this is a perfectly ordinary kind of trade. We made the same one with handwriting, and arguably my generation cannot write a paragraph in cursive that a doctor from 1955 would not laugh at. We made it with mental arithmetic, and I cannot multiply three-digit numbers in my head the way my father can, and my father cannot do it the way his father could probably. We made it with map-reading, and I have watched friends in Cairo become unable to find a street that is two turns from their flat without Google Maps open on the dashboard. The people who say "but those didn't matter, software is different" are not wrong, exactly, software does interact with electricity grids and hospital records and the money supply in ways that cursive does not, but they are also begging the question, because every generation has thought its own surrendered skill was the load-bearing one and most of them were mistaken.

Maybe the people saying it this time will be mistaken too. Maybe the model will get good enough fast enough that the audit problem solves itself before the cohort of auditors ages out. Maybe in 2040 we will all be doing something else entirely and this will read like a complaint about the loss of telegraph operators. I don't know. I would like to pretend that I know, and I notice in myself a strong pull toward writing a final paragraph that closes the question, and I am refusing it on principle. The honest position is that the bargain has been struck, that it is being struck a thousand times a day in offices like the one I sat in this evening, and that none of the people striking it (me included, last week, with the migration) are in any position to evaluate what they are giving up.

#LLM #Modus Vivendi #I'm not consulting an LLM