Overview

This standard is about devising user tests and analysing, evaluating and reporting the data obtained from them. This is likely to be an iterative process during development of interactive media projects.

Interactive media projects can involve any type of interactive media content, products or services for multi-platform or multi-channel use, including, but not restricted to, games, websites, applications or online marketing campaigns and can also include the use of immersive technology. (Immersive technology can include, but is not restricted to, Augmented Reality (AR), Virtual Reality (VR) and Mixed Reality (MR).)

This standard is for anyone involved in user testing of interactive media projects.

 

Performance criteria

You must be able to:

  1. Obtain current project information required for user testing
  2. Carry out a thorough analysis of project information to identify which aspects of projects are to be tested
  3. Specify assessment criteria which will meet project requirements
  4. Select testing methods, tasks and activities which can generate valid data and which meet project parameters
  5. Decide how test data is to be collected and design appropriate data collection instruments
  6. Devise ways to mitigate factors which may affect test results
  7. Specify end-user samples which are representative of target audiences and will provide sufficient valid test data
  8. Select and recruit appropriate test observers or facilitators to meet testing requirements
  9. Prepare clear instructions with relevant information for all the people involved
  10. Collate test results
  11. Analyse and evaluate test results against assessment criteria to identify issues and problems
  12. Rate issues and problems for severity, ease of fixing and impact on projects
  13. Document your findings in an appropriate format for circulation to relevant colleagues

 

Knowledge and understanding

You need to know and understand:

  1. the range of data that can be tested including useability, quality, fitness for purpose, overall design, integration of design elements
  2. how to obtain information about projects and parameters including design objectives, client requirements, design constraints, functional specifications, functionality, features, target audience
  3. the principles of interaction design and issues of usability and accessibility including the use of real-time animation
  4. how to obtain and analyse project information to identify end-user characteristics, requirements, expectations, tasks and goals
  5. ethical issues relating to user testing including privacy, confidentiality, diversity, inclusivity, accessibility, emotional intelligence and behavioural psychology
  6. the features, benefits and disadvantages of low-tech techniques for conducting usability tests including questionnaires, interviews, focus groups, surveys, user observation
  7. how to prepare test scripts, interview questions and survey forms
  8. the features, benefits and disadvantages of high-tech automated techniques for conducting usability tests including eye-tracking, server logs, analytics and other scripted processes
  9. the role of test observers and facilitators and when it is appropriate to use them
  10. information required by test observers and facilitators including test objectives, assessment criteria, and their role in the process and why it is important to brief them
  11. how to produce instructions for end users which contain sufficient information for them to carry out tests without unnecessarily influencing results
  12. what factors may affect test results including equipment, environment, clarity of task instructions, leading questions, awareness of being observed
  13. collation, analysis and evaluation techniques for qualitative and quantitative data
  14. project reporting formats and who needs to receive them
  15. how to write a well-researched and clearly worded report