When I first started working with AI, I was deeply skeptical, and so I followed a rule that I'd heard others set who seemed wiser than I was.
Never let AI do anything you can't do for yourself.
I followed this for quite a while, and I was working on the assessment of AI as being .. well, not very useful.
Maybe that was on me. The analogy that I've been using about AI is that it's very similar to working with a table saw. On the one hand, it's an extremely powerful tool, and used well, can allow you to perform actions that you could not achieve yourself. On the other hand, you can slice that hand clean off if you allow yourself to stop thinking, to stop sitting in judgement of the work, the method, the steps involved in the use of that tool.
Not to beat the analogy to death, but I can't chew through boards as fast as a table saw. That doesn't stop me from using one, provided I'm careful and I plan my steps in advance.
My new relationship with AI is the same, and my guideline has changed. "Never allow AI to do anything that you can't undo."
- No git history or database backup to pull back to this moment? That's on you. As Ken and Roberta Williams taught us, Save Early, Save Often.
- No test environment to allow the changes so that you can validate them? Then don't do it.. yet. Stand the env up.
- Are you working with network capable software? Do you know what it's going to contact when? You want to make absolutely certain you are allowing the minimum necessary connections, and that there's no network path to production end points. There are real world consequences to things like test message emails that go to real people, or API calls that should've never hit production and did.
I've spent decades learning how to build software badly, and then what to look for in poor software, and that gives me the confidence to work outside of my comfort zone on the construction side; I know how to build the safety nets that I need to keep from facing regrets.
Let's use an example, just experienced this. My mom is writing gospel music, and wants a means to throw a still image into an MP3 to generate a video file for Facebook. (For those of you curious who don't do Facebook, check out https://dianeblevins.com for the output.) Now I know the ffmpeg syntax to do that; it's trivial, but in no world is my mom ever going to fire that up. So, from my M5 Macbook, I asked Claude to build & compile a little app that lets her drag and drop the two files into a window to construct the third, setting sane defaults. Claude thought for a minute, and kicked out a standalone compiled EXE.
I reviewed the code. It was short, no sign of anything troubling. So then I fired it up on a VM with Process Monitor running to make sure that it wasn't doing anything that I didn't expect. I also used a different context window to review it while I experimented. I used copies of the artifacts that my code would touch, and ensured the originals weren't altered.
Worked like a champ. I moved the app to Mom's machine, and she used it successfully first time, and every time after.
But I've never used the x64 compile process for a dotNet app on a Mac before. I barely know the language, but I've got enough familiarity with other languages that the code reads sensibly to me. My old heuristic would never have allowed me to get her up and running.
I was able to do this safely because I had made certain that my first run of the tool was in an environment that I could completely destroy without any consequence. I could just revert the VM to undo any changes it made, and I had the network within the VM completely gapped so that it couldn't reach out and alter anything. The code from the first run was in a git repo already so that if there were anything to change, I could iterate over it and recover back to where I started.
And that last, the idea of iterating and refining quickly and safely, is the heart of everything that I've learned so far about working with AI. There's a lot of focus on the mistakes that AI makes, but a safety net covers a great many such problems, and it's not as if we mere mortals don't have the same failures. Further, most of the kinds of problems that I have to iterate over when working with AI stem from a poor job on my part of articulating what my goal was in a way that doesn't either allow for misinterpretation, or an approach that I couldn't live with. As much as I am teaching my agents to work with me, I'm also learning how better to work with them.
Mom has a tool that works for her that I couldn't have made without a great deal more time, trial, and error, than the process that I used here.
