Dr. Somang Nam, Dr. Karam, Tatyana Kumarasamy, Margot Whitfield, Dr. Fels (Toronto Metropolitan University)
Live captioning in fast-paced sports tends to exceed 200 words per minute, which is beyond many people’s reading comfort level. Commentators fluidly transition between commenting on the game and describing the gameplay action (play-by-play), which is visually redundant for hearing and Deaf (D)/Hard of Hearing (HoH) viewers alike, and commenting on other information unrelated to the gameplay such as injuries, refereeing, and other teams performances (colour commentary). The fast pace and large quantity of text generated by captioners when gameplay descriptions (voiced by commentators) are included can pose a visual distraction to caption viewers, which can in turn influence their enjoyment and comprehension of the game, so much so that some D/HoH prefer to watch the game without the captions altogether. Rather than an all-or-nothing approach, we introduced a colour commentary only style (CO) of captions, where gameplay captions were removed. We compared D/HoH viewers’ eye gaze behaviour, comprehension, and subjective preferences for game clips containing CO captions versus the conventional condition (PbP) where all speech was captioned.
A 2×2 research study with 17 D and 11 HoH participants was designed and carried out. The between-subjects variable was hearing status (D or HoH) and the within-subjects variable was caption condition (CO or PbP). Participants watched two segments of a 20-minute sporting event of their choice, hockey or basketball. Following consent and completion of a pre-study demographic questionnaire, each participant watched one clip that contained CO captions and another one with PbP captions in random order. The Tobii Eye Tracker was used to collect participants’ eye gaze fixations and durations. It was recalibrated before each viewing session to ensure tracking accuracy. Participants also completed questionnaires after each segment clip viewing, as well as a final questionnaire after viewing both clips. This was followed by a one-on-one informal discussion of the game, with standardized topics and expected responses from the CO and PbP captions in order to assess comprehension in a more realistic and informal way compared with existing comprehension assessment methods.
Results showed that there were differences between the PbP and CO conditions for D/HoH. In the CO condition, D and HoH participants spent significantly more time and fixations watching the game than reading the captions. HoH participants also spent significantly more time watching the game than reading the captions in the PbP condition. Caption condition preference and reading comprehension were related to the amount of experience in viewing fast-paced sports; experienced caption users preferred the CO condition, and less experienced, the PbP condition. However, there was no significant difference in comprehension scores between the two conditions. Participants noted that the caption condition influenced the level of distraction they experienced watching the gameplay; in the CO condition there was less distraction reported. Synchronization delay, spelling mistakes, and position of captions on-screen were also reported as negative influences on participants’ caption experiences.
Dr. Somang Nam is a human factors researcher. He obtained a Ph.D. in Industrial Engineering from the University of Toronto. He is currently a postdoctoral research fellow at Toronto Metropolitan University.
Dr. Karam‘s academic profile is grounded in human-computer interactions, with a focus on sensory computing for entertainment and accessibility applications. Karam is also an inventor, entrepreneur, and the owner of a historic theatre in Toronto’s East End.
Tatyana Kumarasamy is a student at Toronto Metropolitan University completing her bachelors degree in Psychology. She is currently a research assistant and undergraduate thesis student at Toronto Metropolitan University.
Margot Whitfield earned her B.F.A. with a minor in philosophy from York University and her M.A. from the University of Toronto. Her research interests include access to arts and culture, critical disability studies, and accessible design.
Dr. Fels received a Ph.D. degree in human factors from industrial engineering from the University of Toronto. She is a Professor at the Ted Rogers School of Information Technology Management, and the Director of the Inclusive Media and Design Center, at Toronto Metropolitan University. She is also a professional engineer. Her research interests involve inclusive design, access to media and technology for people with disabilities and older adults, inclusive video game design, and inclusive business.