I hired a new developer. I gave him a one-pager describing what I wanted built. Then I disappeared for a couple of weeks and came back to check in. The project wasn’t finished, the code didn’t compile, and it didn’t look anything like what I had in mind. So I fired him.
Sounds ridiculous, right? No decent manager would evaluate a developer that way.
We’re Evaluating AI Wrong
That’s exactly how I see people evaluating AI coding tools. They throw in a one-time prompt, check back later, and when the result isn’t great, they declare, “AI development doesn’t work.”
The problem isn’t the tool. It’s the expectation.
We want AI to be a silver bullet. You think it, and it comes to life. But that’s not how effective teams work, and it’s not how AI works either.
Treat AI Like a (Speedy) Junior Teammate
If you want to get usable code from an AI tool, treat it like a junior teammate. Not a senior developer. Not a seasoned architect. Someone who can help if you’re willing to work with them.
What Good Practice Looks Like
- Give clear, structured input: Don’t just say “build a dashboard.” Tell it what data you have, who is going to use the dashboard, and what decisions the dashboard supports.
- Review early, review often: Just like code reviews, check progress in small chunks. Don’t wait until the end. In fact, check the plans before it even starts writing code.
- Iterate: The first draft is just that. Ask it to revise. Build on it. Add your own expertise. Treat it like a demo. Provide feedback and let it process it and come back for another demo.
- Own the outcome: You’re still responsible for the result. AI is a tool, nothing more.
- Check in often: Use source control, branches, and pull requests. Commit frequently so you can track changes, review diffs, and catch mistakes early.
- Don’t give blind access to production: The same security you’d expect with any developer applies here. Use a dev environment for the day-to-day development.
Reset Your Expectations
AI isn’t failing. It’s just not doing well when we evaluate it under unrealistic expectations and then write it off. And remember, this is the worst that AI development will ever be. It’ll get better. Maybe someday, you will be able to think it, and it will come to life.
For now, though, if you want better output, reset your expectations. The way you approach the tool matters as much as the tool itself.