Start with the decision
The single biggest predictor of interview quality.
Every good interview starts with a decision the output will inform. Not a topic, not a curiosity, not a "let's understand our users" - a specific decision that someone on your team is going to make, and that the answers from this interview are going to move. If you cannot name that decision before you open create_agent, the interview will be unfocused, the analysis will be hard to act on, and the time you and your respondents spend will mostly be wasted. Picking the decision is the first design step. Everything else - persona, techniques, graph structure, probing depth - is downstream.
One decision, one interview
A single interview should serve a single decision. Two decisions means two interviews - different respondents, different questions, different graph. Asking both in one sitting is rarely worth what you save in calendar time, because the question set ends up fighting itself.
Three common ways this goes wrong:
- Bundling. "Should we build feature X?" and "are users satisfied overall?" get crammed into one blueprint. The first wants people who experienced a specific gap; the second wants a representative sample. The graph cannot serve both, and the analysis cannot pull them apart afterwards.
- Discovery without a target. "Discovery" interviews run without naming what the discovery is for produce broad transcripts that no one acts on. Discovery is a mode, not a purpose - it still has to feed a decision, even if that decision is "which problem to investigate next".
- Asking respondents to weigh in on decisions only the team can make. Whether to deprecate a feature, set a price, or hire a role - respondents can give signal on the inputs, but turning the interview into a vote turns it into theatre. Their job is to describe their world; the team's job is to decide.
The test, before configuring an agent: finish the sentence "the output of this interview will help us decide ___". If you cannot, stop and figure that out first. Everything below assumes you can.
Exploratory vs confirmatory
Once you have the decision, the next thing to know is whether you are running an exploratory or a confirmatory interview. The two modes look superficially similar - both end up as conversations between an agent and a respondent - but they ask for almost opposite design choices.
| Attribute | Exploratory | Confirmatory |
|---|---|---|
| What you don't know | The shape of the problem | The likelihood of a hypothesis |
Recommended probing_depth | deep | light or moderate |
| Node mix | Mostly open-ended question nodes; few form nodes | Higher proportion of form and structured question nodes |
Use of required: true | Sparing - let respondents skip | Liberal - a fixed dataset matters |
| Branching style | Loose, narrative arcs | Tight, deterministic transitions on structured fields |
| Run length tolerance | Longer runs acceptable (15–25 min) | Shorter runs (5–10 min) for response rate |
| Sample size | Smaller (5–15) | Larger (50+ for any statistical claim) |
Most real interviews lean one way but include a few elements of the other. A confirmatory pricing study still benefits from one or two open nodes near the end where unexpected objections can surface; an exploratory churn interview can ask the respondent to confirm their plan tier in a form node up front. The lean is the design decision; the borrowing is fine in moderation.
The cost of misdiagnosis goes both ways. A confirmatory interview that is actually exploratory produces brittle data - a tight, structured graph run against a problem you did not understand, and answers that tell you nothing you didn't already assume. An exploratory interview that is actually confirmatory produces narrative without rigour - long, rich transcripts that cannot be aggregated, against a question that needed a count. Naming the mode up front is what stops both failures.
The four-question pre-interview worksheet
Before you open create_agent, answer these. In order.
- What decision will this interview inform? Write it as a sentence that begins "We will use this to decide whether…" or "We will use this to decide which…". If multiple decisions surface, split into multiple interviews - see the section above.
- What would change in the decision based on the answers? If the honest answer is "nothing - we already know what we'll do", the interview is theatre, not research. Either skip it or pick a different decision the team genuinely has not made yet.
- Who experienced what we're asking about? The respondent has to have first-hand experience of the situation the decision concerns. If they're being asked to speculate about other people's behaviour or imagine a scenario they have not lived, the data is weak - and no interview technique recovers from a sample that doesn't have the experience to draw on.
- What do we already know, and what would surprise us? Naming the prior makes it easier to spot two things later: when the data is just confirming what you walked in with, and when it is actually moving you. Without the prior written down, every transcript reads as confirmation.
The worksheet is short on purpose. If any answer is hard to write, that is the work - push on it before you configure an agent, not after the first batch of runs comes back.