Running Sessions Without Bias

Interview tools & tips

• Tools — pen & paper, recording (always ask for permission)
• Have an outline ready for the researcher
• Use non-verbal communication — nod, raise eyebrows
• Give verbal encouragement — repeat what they said, follow up, "mhm", "why"
• Give space to everyone involved
• Provide stimuli material, e.g. images for the participant
• Show empathy

Biases to watch for

• Sample bias — make sure your sample represents your early adopters/customers
• Data consistency — use a uniform collection method across participants and researchers
• Leading questions — avoid questions that influence answers
• Observer effect — your presence can change participant behavior
• Recall bias — participants may not remember past events accurately
• Non-verbal cues — pay attention, they add insight
• Social desirability — people answer what's acceptable, not what's true
• Selective perception — your own biases can affect what you choose to observe
• Confirmation bias — watch for seeking only confirming information
• Overgeneralization — don't draw broad conclusions from a small/non-representative sample
• Subjectivity — qualitative interpretation is inherently subjective

Pilot testing

Prepare and review a research plan, then collect data from the first 3 participants (qualitative) or 30 respondents (quantitative).

• After each pilot session, run an evaluation debrief on how it went
• Evaluate the data collection technique itself
• Evaluate what data/answers you're getting toward what you want to know — and what you're not getting
• Consider the effect of biases
• Propose changes to technique, questions or metrics before scaling up