Matteo Manzoni

For people with upper extremity motor impairments, interaction with mobile devices is challenging because it relies on the use of the touchscreen. Existing assistive solutions replace inaccessible touchscreen interactions with sequences of simpler and accessible ones. However, the resulting sequence takes longer to perform than the original interaction, and therefore it is unsuitable for mobile video games. In this paper, we expand our prior work on accessible interaction substitutions for video games with a new interaction modality: using facial gestures. Our approach allows users to play existing mobile video games using custom facial gestures. The gestures are defined by each user according to their own needs, and the system is trained with a small number of face gesture samples collected from the user. The recorded gestures are then mapped to the touchscreen interactions required to play a target game. Each interaction corresponds to a single face gesture, making this approach suitable for the interaction with video games. We describe the facial gesture recognition pipeline, motivating the implementation choices through preliminary experiments conducted on example videos of face gestures collected by one user without impairments. Preliminary results show that an accurate classification of facial gestures (97%) is possible even with as few as 5 samples of the user.

This paper describes the development of PRACTICE, a distributed healthcare technological platform that supports various research initiatives by the University of Milan and the Angelo Bianchi Bonomi Hemophilia and Thrombosis Center, Fondazione IRCCS Ca’ Granda, Ospedale Maggiore Policlinico. PRACTICE includes three main components: a mobile app that patients can use to self-acquire ultrasound images at home, a computer-aided diagnosis web application that supports the practitioners through a set of machine learning models, and a set of web tools for image annotation, a prerequisite for training the machine learning models. Although PRACTICE was designed in the specific context of supporting the detection of joint recess blood effusions in hemophilic patients, this paper describes the main design and implementation challenges that apply to other applications of a research-oriented health platform.

Shared control is a form of video gaming accessibility support that allows players with disabilities to delegate inaccessible controls to another person. Through interviews involving 14 individuals with lived experience of accessible gaming in shared control, we explore the ways in which shared control technologies are adopted in practice, the accessibility challenges they address, and how the support currently provided in shared control can be automated to remove the need for a human assistant. Findings indicate that shared control is essential for enabling access to otherwise inaccessible games, but its reliance on human support is a key limitation. Participants welcomed the idea of automating the support with software agents, while also identifying limitations and design requirements. Accordingly, this work contributes insights into current practices and proposes guidelines for developing automated support systems.