Project Overview

DePaul University lab “Play for Change” created four desktop video games. Each game was intended to increase understanding of a different mental health issue by providing an experiential understanding of an individual’s lived experience with the disorder.  The games were not intended to necessarily provide players with pleasing experiences, but rather experiences that comported with aspects of Attention Deficit Disorder (ADD), Bi-polar Disorder, Eating Disorder (Anorexia Nervosa), or Obsessive Compulsive Disorder (OCD).

Objective

The producer was interested in whether a player’s experience matched the experience intended by the games’ designers.  Lead faculty member focused inquiry to a narrow group of potential game players/users, mental health therapists.  Thus, we were tasked with evaluating game usability and evaluating the therapist players user experience to determine whether playing the games impacted their experiential understanding of the modeled disorder.

Approach and Research Methods

Recruited 16 therapists (N = 16) via email using a recruitment flyer. Conducted three part study: 1) Pre-gameplay survey and interview; 2) Gameplay observation study; 3) Post-gameplay interview. Audio was recorded for each study pursuant to university IRB approval and participant informed consent.

Study Protocol Checklist

Pre-gameplay Survey and Interview

• Captured demographic information, gameplay experience, familiarity and expertise with mental health disorders.
• Evaluated whether demographic, prior gaming experience, or therapy specialization correlates with observed gameplay performance or player’s reported experience.

Survey

Pre-gameplay interview discussion guide

Gameplay Observation Study

• Randomly assigned order of games to be played by therapist players to control for ordering effect.
• Players were instructed to “think aloud” as they played each game. Researcher prompts given to therapists only when they indicated they could not continue or game usability issue prevented them from moving forward.
• Evaluate each game’s usability and identify any unintended pain points during gameplay.

Post-Gameplay Interview

• Open ended-themed questions structured with a discussion guide.
• Confirm/better understand any observed usability issues through discussion.
• Evaluate the therapist players’ gameplay experience.
• Evaluate whether game provided new or reinforced existing experiential insight into modeled disorder.

Post-Gameplay discussion guide

Audio recordings of gameplay observation and post-gameplay interview were transcribed, coded by theme in atlas ti, and analyzed for patterns.

Limitations

Coding and analysis was conducted by one researcher and thus did not benefit from verification of objectivity via coder agreement.

Tools

tableau, saturate app, atlas ti

Results

Results Presented at Games for Change Conference 2015

Download Slide deck for presentation

Well Played Journal Article

View: “Barriers To Learning About Mental Illness Through Empathy Games –
Results Of A User Study On Perfection”

Summary of Results

Usability issues were identified during gameplay observation in 3/4 of the games and confirmed during the post-gameplay interview.

These usability issues greatly impacted the therapists’ experience playing the games and their ability to understand the games’ metaphors.

• Therapist players group was a non-game savvy audience and more literal in their expectations of how game elements function.
• Majority of subjects had no gaming language, UI conventions, control scheme knowledge.
• Most therapists reported and displayed signs of insecurity in their ability to play the video games prior to even playing them.
• When usability issues were encountered, some players assume it was their fault. They displayed a lower tolerance for the “failure” necessary to learn game system in inexperienced gamers
• Although the games were designed to specifically create discomfort and the perception of failure so to model mental health disorders, other therapists attributed in-game frustration to a usability issue or a system problem.  They misattributed in-game frustration to at-game frustration. This was especially evident in therapist’s observed behavior in the game Perfection when compared to their description about their experience and how it might relate to the modeled disorder of anorexia.

Overall the therapists focused on learning game system exclusively and not critically thinking about in-game metaphor.

Emotions reported by therapists during gameplay

Back to UX Portfolio