Usability Testing: Assessing Contact Experience (Ease of Use)

Once you have selected a final version of your flow, a usability test helps ensure that it provides a clear, comfortable, and effective experience for your contacts.

Usability testing shifts the focus from whether the flow works to how it feels to use. It evaluates ease of use, learnability, clarity, and satisfaction before your flow is launched at scale.

Important: A flow that works technically may still fail if contacts find it confusing, frustrating, or difficult to complete.

Before conducting a usability test, make sure you have:

• A large group of test contacts representative of your target population
• A clear testing protocol, which may include:
  • A pre-test questionnaire
  • The test itself
  • A post-test questionnaire
• A facilitator who can answer questions and provide instructions
• Observers who track contact behavior and interactions

Tip: Unlike a pilot test, facilitators should take a supporting role during usability testing, allowing contacts to lead the experience.

Subjective metrics capture how contacts feel about the experience. A usability test plan should include questions asked before, during, and after the test.

Common subjective metrics

• Background questions: asked before the test to understand context and expectations
• Ease of use, comfort, and satisfaction: asked after each task to evaluate immediate reactions
• Overall experience: asked after the test to assess overall ease of use, comfort, and satisfaction
• Likes, dislikes, and recommendations: what contacts liked most, liked least, and how the service could improve
• Continued use: how likely contacts are to continue using the service
• Net Promoter Score (NPS): how likely contacts are to recommend the service to a family member, friend, or colleague

Quantitative metrics measure performance and efficiency. Track these consistently so results can be compared across iterations.

Core quantitative metrics

• Flow completion: a flow is complete when a contact successfully passes through all steps
• Critical errors: issues that prevent completion (e.g., consistently incorrect responses, unsupported formats, opt-outs)
• Non-critical errors: recoverable issues (e.g., “other” responses, redirects that slow completion)
• Error-free rate: percentage of contacts who complete flows without any errors
• Time on task: completion time minus start time

Warning: Critical errors indicate serious usability issues and must be addressed before launch.

During usability testing, participants should lead the experience while the facilitator supports and observers document behavior and outcomes.

Use your protocol to:

• Collect pre-test context (background questions)
• Run the flow tasks end-to-end
• Capture task-level reactions (ease, comfort, satisfaction)
• Collect post-test feedback (overall experience, likes/dislikes, recommendations, continued use, NPS)

After testing, analyze what you observed and summarize both what worked well and what needs improvement.

You may want to include:

• Completion rates by contact, step, and flow
• Average completion times
• Satisfaction and ease-of-use scores
• Illustrative comments from contacts

Important: Always identify both what worked well and what needs improvement.

Use all collected data to list findings by flow and provide recommendations grounded in observed behavior. Balance negatives with positives—maintaining what works is just as important as fixing what doesn’t.

You may not be able to implement every recommendation due to:

• Budget constraints
• Timelines
• External dependencies

Prioritize changes based on:

• Frequency of issues
• Severity of impact
• Contact feedback

Tip: After implementing changes, retest to confirm improvements before scaling.