Written by Nick Sawinyh

I’ve hired engineers and product people across several companies, including a globally distributed team I built from nothing. For most of that time I ran the standard process: a technical exercise, a system design conversation, some behavioral questions. It worked reasonably well.

It works considerably less well now, and the reason is specific rather than general anxiety about AI.

The problem in one line

Nearly every technical interview measures whether a candidate can produce a solution. Producing a solution has become dramatically cheaper. Evaluating whether a solution is correct hasn’t.

So the interview is now weighted toward the part of the job that got easy, and it barely tests the part that got hard. That’s a measurement problem, and the usual responses to it are both bad: lock candidates in a room with no tools, which tests a job nobody does anymore, or let them use everything and watch the signal collapse because everyone produces something plausible.

What I do instead

I give candidates a working solution and ask them to tell me whether it is right.

Concretely: a piece of code, or a plan, or an analysis, that looks finished. It’s well-structured and confident, and it contains a real defect. Sometimes an edge case, sometimes a wrong assumption baked into the middle, sometimes a subtle mismatch between what it claims to do and what it does. They may use any tool they want, and I say so explicitly.

Then I watch how they approach it. That is the whole test.

The strong candidates do something recognizable within a few minutes. They decide what “correct” would mean before examining the artifact, rather than reading top to bottom hoping something looks wrong. They pick the load-bearing assumption and check that first. They say “I don’t know yet” and describe what would resolve it. They find the defect or they clearly bound where it must be.

The weak ones read it, find it plausible, and start suggesting improvements. Naming, structure, a nicer approach. All reasonable, all missing that the thing doesn’t work.

That difference has predicted on-the-job performance better than anything else I have used. It also reads as fair to candidates, which matters more than people admit: nobody feels tricked, because the task is obviously the job.

Why this correlates with something real

I found this in my own work before I turned it into an interview.

When code started arriving from a model, fully formed and plausible, my review got worse rather than better. Handwritten code announces its own rough edges. Generated code reads as though it already passed review, so I skimmed it, and I approved things I would have questioned from a colleague. The skill I needed wasn’t reading more carefully. It was checking against a definition rather than against my impression.

So I stopped assessing whether code looked right and built a measure of whether it was still standing weeks later, with evidence attached to every failure it counted. Reading is subjective. Survival isn’t.

The interview is a compressed version of that shift. I am not asking whether someone can write the thing. I’m asking whether they can tell me it works and defend the claim.

Two other changes worth making

Ask about a time they were confidently wrong. Not the sanitized failure story. A specific instance where they believed something, acted on it, and it was not so. What made them believe it, and what would have caught it earlier. People who verify well have vivid answers to this because they have thought about their own failure modes. People who do not will produce a generic answer about a project that ran late.

Stop treating tool use as cheating and start treating it as the exercise. The interesting question is not whether a candidate used a model. It’s what they did with the output. Did they check it? Did they notice the part that was wrong? Did they know which part to check first? That’s observable, and it’s the actual skill.

The part I’m still unsure about

I don’t have this fully solved and I want to say so.

Verification-focused interviews may favor experienced people more strongly than production-focused ones did, because knowing where things typically break is largely pattern accumulation. That’s a real risk for junior hiring and I haven’t found a clean answer. For now I run a different, more teaching-oriented process for early-career roles, and I am not confident it’s right.

I’m also aware that any interview format becomes gamed once it is common enough. This one will too. The underlying point survives that, though: whatever you test, make sure it’s a thing that is still scarce. Production isn’t scarce anymore. Judgment about whether the production is any good very much is.

Author Bio:
Nick Sawinyh is Head of Product and GTM at Veodyn. He previously co-founded and led a venture-backed platform with a globally distributed team, and has spent over a decade taking technically complex products to market.