Engineering Trustworthy Automation: Design Principles and Evaluation for AutoML Tools for Novices

AutoML systems targeting novices often prioritize algorithmic automation over usability, leaving gaps in users’ understanding, trust, and end-to-end workflow support. To address these issues, we propose an abstract pipeline that covers data intake, guided configuration, training, evaluation, and inference. To examine the abstract pipeline, we report a user study where we assess trust, understandability, and UX of a prototype implementation. In a 24-participant study, all participants successfully built their own models, UEQ ratings were positive, yet experienced users reported higher trust and understanding than novices. Based on this study, we propose four design principles to improve the design of AutoML systems targeting novices: (P1) support first-model success to enhance user self-efficacy, (P2) provide explanations to help users form correct mental models and develop appropriate levels of reliance, (P3) provide abstractions and context-aware assistance to keep users in their zone of proximal development, and (P4) ensure predictability and safeguards to strengthen users’ sense of control.

Recommended citation: Thys, J., Vanacken, D., & Rovelo Ruiz, G. (2025). Engineering Trustworthy Automation: Design Principles and Evaluation for AutoML Tools for Novices. arXiv preprint arXiv:2511.22352.
Download Paper