In Greek myth, Odysseus faced a challenge that couldn't be met by strength of arms. The Sirens sang, and sailors who heard them steered their ships onto the rocks. Most crews stopped their ears with wax and rowed past. Odysseus wanted to hear the song for himself, so he had his crew plug their ears and tie him to the mast, with one standing order: no matter how he begged to be released, they were to bind him tighter.
I can completely relate. My Sirens go by many names. Claude. Copilot. DeepSeek. Qwen. It's a huge pack, and the song they sing stirs something in me that takes a great deal of restraint to resist. They sing of laziness.
I didn't get into computing to work hard. It's a deep skill set to do well, and I learned it to avoid work. There's truth to the old joke that "anything worth doing once is worth spending weeks automating," and the irony isn't lost on me. But all of it, in the end, was to cut the Gordian knot, to bypass the heavy lifting that I or others had to do to reach a goal. My lazy little heart sings in resonance with the notion of not having to go through quite so many steps to get where I'm going.
So you flirt with those Sirens on a small goal, and thwock, you hear that meaty sound a bat makes when you knock one out of the park. You watch the ball sail away and think, "Cool, this works," so you run your bases, and you step up to the plate to take another swing with your new friends. And again, thwock. Rinse, repeat a few times, until finally, as you're rounding second, you hear the shattering of glass. Yeah, you hit the ball, but you weren't paying attention to where it went.
Enough analogies. The core problem with using generative AI to solve problems is twofold:
- The ease with which humans stop checking.
- The Dunning-Kruger effect.
Complacency
If your tests pass every time, you stop running them. They feel like wasted time, and then the one-in-a-hundred case arrives. Nobody who skips a check wants anyone hurt. It's a simple facet of human nature: a task that never seems to produce results is a task we quit doing.
Casner, Geven, and their colleagues put 16 airline pilots in a Boeing 747-400 simulator to see what heavy automation does to flying skills. Instrument scanning and aircraft control held up reasonably well, but the cognitive skills needed for manual flying depended on how actively the pilots stayed engaged in supervising the automation. Performance on those cognitive tasks was associated with how often pilots drifted into task-unrelated thought while automation was flying. The hands stayed sharp. The thinking slipped.
That's the part that worries me, because verification is thinking. I'll go further than the study does, and this is belief, not proof: the judgment behind a validation is as much a skill as the mechanics of performing it, and it fades the same way when we stop exercising it. Microsoft and Carnegie Mellon researchers surveyed 319 knowledge workers and found that those with more confidence in the AI applied less critical thinking to its output, while those with more confidence in their own skills applied more. That's self-reported and correlational, but it points the same direction.
Ignorance
A friend recently shared his concern with generative AI, and he said it this way: "Never before have we been able to build so many bad ideas so quickly."
He's got a point, and I've been party to it. If you don't have the skills to do a thing well, you haven't built up the scars and calluses that grant you solid judgment. Pair that with how confidently wrong AI can be, and you get the "AI Slop" everyone hates so much.
- People with no musical ability who use AI to make songs make bad songs.
- People with no graphical ability who create images make ugly images. (I fall into this category.)
- People with no coding experience create bad code bases.
You can't avoid this. You can backstop your code with all the "grill me" skills, "as a professional developer" reviews, and "challenge me" prompts you want. It takes the ability to recognize a good implementation to catch a bad one. The skill it takes to build a thing is the same skill it takes to judge it, and AI magnifies whatever gap the operator brings.
What to do?
That's where the rope and mast come in.
AI is here to stay. Even if the bubble bursts, even if legislation changes, even if data centers burn, it's part of our lives now, and turning your back on a tool is foolish. It's exactly against the grain of why I took an interest in any of this. So let's not bother trying to get the toothpaste back in the tube.
My commitment is "Not on my watch." Doesn't matter who wrote it; that code carries my name, and I bear the responsibility to look closely and to keep improving my judgment. But a vow is only as good as my attention that day, and attention is the thing I've just argued fails. Odysseus didn't trust himself once the song started. He built the constraint while he was still thinking clearly.
So the rope is structure that works on the days I'm not paying attention:
- Make it reversible. Git history, backups, a VM I can revert, a network that can't reach production. Never let AI do anything I can't undo.
- Decide what "right" looks like before I run it. I write down what I expect first, so I'm checking against my own understanding and not against whatever the tool produced.
- Get a second set of eyes that isn't mine. A different context window helps, but it shares the generator's blind spots. People help more.
- Keep the crew. Cultivate relationships with people whose judgment I trust, and learn to think as they do. They're the ones who tighten the ropes when I'm begging to be let loose.
Odysseus didn't skip the song. He heard it, and he lived, because the safeguards were in place before it started. I'm not turning my back on the Sirens. I'm just tying myself to the mast first.
-- LATER
A colleague shared this with me, and it's a far more elegant discussion on the point than mine.