As the international organization responsible for promoting, coordinating and monitoring the fight against doping in sport, the World Anti-Doping Agency (WADA) is committed to educating athletes about the risks associated with doping and their rights and responsibilities in the doping control process. As a proponent of encouraging stakeholder involvement and the formal, formative evaluation of educational tools, I have been impressed with WADA’s approach to evaluation. (Full disclosure: my company is one of WADA’s educational tool development vendors.) WADA is creating an online game to reach elite youth athletes and, at various stages during the development process, WADA tested concepts and the final version with students. The following is an interview with Jen Sclater, WADA Education Manager, who speaks on the topic of user testing, A.K.A Formative Evaluation of educational technology tools. Listen here:
Educating Elite Youth Athletes
WADA has high hopes for the premiere of the Play True Challenge Game at the Youth Olympic Games to be held this August in Singapore. The game allows the player to experience the consequences of his/her decision of whether or not to dope. And, yes, doping will improve the player’s performance in the game, but not without consequences to the athlete’s life and sports career. Listen here:
Approach to Evaluation
WADA used focus group user testing to help determine whether developing a serious game would “speak to young people.” Once in development, Jen went so far as to share concept art with the focus groups. Listen here:
Formal or Informal Approaches
Many clients and developers simply ask their own children or friends to test their games or simulations. Sometimes no more than two people are classified as a user testing group. To encourage more validity, I like to recommend a more formal approach, specifically the use of electronic surveys for collection and stakeholder reflection. Listen as Jen discusses WADA’s approach. Listen here:
What is WADA’s Rationale for Conducting Evaluations?
Jen discusses the two reasons WADA evaluates their educational tools during development. First there are the business reasons: Is this the best resource investment? Will it accomplish strategic educational goals? Acquiring stakeholder “buy-in” is an important strategic goal especially with adults who will ultimately be responsible to see that the educational tool gets used. Secondly, on a tactical level, it is important to know if the game or eLearning is effective in delivering anti-doping education and that it ultimately helps to change behavior. Listen here:
Does Formal Formative Evaluation Improve the eLearning Product?
Jen believes that maintaining a formal approach to formative evaluation helps improve the final product. Testing provides confidence that youth will perceive the game as fun and that stakeholders will perceive it as a good investment. Listen here:
When does Evaluation become Summative?
Play True Challenge debuts August, 2010 in Singapore. I asked Jen when the evaluation becomes summative and when is a project done? Here she discusses how WADA gets qualitative feedback on-site through informal interviews and when WADA involves outside evaluators. Listen here: