Why conduct a usability test?

Why Perform Usability Testing?

Now that you have selected the ideal flow, conducting a usability test can help ensure it delivers a positive experience for your contacts. Usability testing encourages a focus on the user experience by evaluating ease of use and learnability. This step is crucial—it allows you to create the most favorable impression when deploying the final version of your flow.

Usability Testing Requirements

• A substantial group of test contacts that accurately represents your target audience.

• A well-defined protocol for test contacts to follow, which may include a pre-test questionnaire, the test itself, and a post-test questionnaire.

• A trained facilitator who can provide guidance and address questions. During the pilot test, the facilitator played a more active role; in usability testing, they will take a supportive role while test contacts assume the lead.

• Observers who will monitor and document test contact behaviors.

Components of a Usability Test Plan

Subjective Metrics

• Background Questions: Administered before the test begins.

• Ease of Use, Comfort, and Satisfaction: Assessed after each task is completed.

• Overall Impressions: Collected once the test is finished.

• Likes, Dislikes, and Recommendations: Ask contacts what they enjoyed most, what they liked least, and how the service could be improved.

• Continued Use: Measure the likelihood that test contacts will use the service after testing concludes.

• Net Promoter Score: On a scale from extremely unlikely to extremely likely, gauge how likely contacts are to recommend your service to family, friends, or colleagues.

Quantitative Metrics

• Flow Completion: A flow is considered complete when a contact successfully passes through every step, whether an action or a split step.

• Critical Errors: Errors that prevent contacts from finishing the flow, such as consistent incorrect responses, improperly formatted answers, unanticipated inputs, or opt-outs.

• Non-Critical Errors: Mistakes that contacts can recover from, such as responses categorized as “other” that redirect them within the flow. Track the frequency of these errors per contact, per flow, and overall.

• Error-Free Rate: The percentage of contacts who complete the flow(s) without any errors.

• Time on Task: The duration taken to complete the flow(s), calculated from start to finish.

Test Results

Compile an analysis based on your observations. Identify flows with the highest and lowest completion rates. If applicable, include a summary of completion rates by contact, step, and flow. Depending on your metrics, you may wish to present:

• The number and percentage of contacts who completed each flow or series of flows.

• The average completion time per flow or series for those who finished.

• Results related to ease of use, comfort, and satisfaction.

• Illustrative comments from test contacts.

Findings and Recommendations

List findings and recommendations supported by all collected data. Each finding should reflect actual test events. Organize findings and recommendations by flow. Remember, highlighting positives is just as important as identifying negatives—what works well should be retained.

Implementation and Retesting

To derive value from usability testing, apply what you’ve learned to enhance your service. It may not be feasible to implement all recommendations—development often involves trade-offs considering factors like budget, timelines, and external constraints. Prioritize changes based on the most frequent and critical issues. Advocate for the modifications your contacts need most.