Abstract: To compare and select machine learning models, relying on performance measures alone may not always be sufficient. This is particularly the case where different subsets, features, and predicted results may vary in importance relative to the task at hand. Explanation and visualization techniques are required to support model sensemaking and informed decision-making. However, a review shows that existing systems are mostly designed for model developers and are not evaluated with target users in terms of their effectiveness. To address this issue, this research proposes an interactive visualization, VMS (Visualization for Model Sensemaking and Selection), for users of the model to compare and select predictive models. VMS integrates performance-, instance-, and feature-level analysis to evaluate models from multiple angles. Particularly, a feature view integrating the value and contribution of hundreds of features supports model comparison on local and global scales. We exemplified VMS for comparing models predicting patients’ hospital length of stay through time-series health records and evaluated the prototype with 16 participants from the medical field. Results reveal evidence that VMS supports users to rationalize models in multiple ways and enables users to select the optimal models with a small sample size. User feedback suggests future directions on incorporating domain knowledge in model training, such as for different patient groups considering different sets of features as important.

Reference: Chen He, Vishnu Raj, Hans Moen, Tommi Gröhn, Chen Wang, Laura-Maria Peltonen, Saila Koivusalo, Pekka Marttinen, Giulio Jacucci. VMS: Interactive Visualization to Support the Sensemaking and Selection of Predictive Models. In: ACM IUI 2024.

Speaker:  Chen He is a postdoctoral researcher at the Ubiquitous Interaction Group, Department of Computer Science, University of Helsinki. Her research interests include information visualization and human-centred AI. She collaborates with machine learning and bioinformatics experts to explore how visualization could make AI more accessible to domain experts, such as biologists. On the other hand, her current research also investigates how people discover insights during visual data exploration by evaluating interactive visualization prototypes. 

Affiliation: University of Helsinki

Place of Seminar:  Kumpula Exactum CK111 (in person) & zoom ( Meeting ID: 640 5738 7231 ; Passcode: 825217)