Engineering Trustworthy Automation: Design Principles for AutoML Tools for Novices

Related Publication: Engineering Trustworthy Automation: Design Principles and Evaluation for AutoML Tools for Novices

arXiv Paper

This blog post describes research presented at the EICS 2025 International Workshops and Doctoral Consortium.

What We Did

This work focused on a practical problem: many people want to use machine learning for their own domain tasks, but most AutoML tools are still not truly designed for beginners. Existing systems often automate model search and optimization, yet they leave major gaps in usability, understanding, and end-to-end workflow support. As a result, novices may be able to click through a pipeline, but still not understand what the system is doing, when to trust it, or how to use the trained model safely afterward.

The goal of this research was to design and evaluate a more trustworthy form of automation for novice users. Instead of focusing on peak algorithmic performance, we focused on building an AutoML workflow that helps users successfully create a working model, understand the main steps, and maintain a sense of control throughout the process.

More specifically, the project had three core contributions:

1. An abstract AutoML pipeline for novices We proposed an end-to-end pipeline covering the full process from data intake to inference, rather than only automating isolated parts of model training.

2. A prototype implementation called NovaClass To examine that pipeline in practice, we built a novice-oriented system for Transformer-based text classification that makes advanced model training more accessible through a guided interface.

3. A user study and resulting design principles We evaluated the prototype with 24 participants and used the results to derive four design principles for future AutoML tools aimed at novices: first-model success, explanations, context-aware abstractions, and predictability through safeguards.

This problem matters because domain experts increasingly want to apply modern AI to their own work—whether that means grading open-ended answers, analyzing text, or building task-specific classifiers—but they are often blocked by programming requirements, framework complexity, and fragile configuration steps. If AutoML tools are going to make AI genuinely more accessible, they need to support not just automation, but also trust, usability, and learning.

How We Did It

1. Designing an Abstract Pipeline

We first designed an abstract end-to-end AutoML pipeline specifically aimed at novice users. The paper frames this pipeline as six connected stages:

1. Data Intake and Upfront Safeguards The system begins by ingesting data and validating it early. This includes identifying issues like missing values, checking labels, and catching common problems before training starts. The idea is to reduce avoidable failures and make the system more robust from the start.

2. Key Model Parameter Configuration Rather than exposing every technical setting, the system only shows the parameters that matter most for novice decision-making—such as selecting which columns should be used as input and which column should be predicted. Other settings can be handled automatically using safe defaults.

3. One-Click Training Once the essential choices are made, the rest of the pipeline is automated. This includes preprocessing, train/validation/test splitting, label conversion, and other steps that usually require technical knowledge.

4. Simplified Results After training, the system presents performance in a simplified way so users can understand outcomes without needing deep ML expertise. The emphasis is on interpretability and clarity rather than only raw metrics.

5. Save Model and Metadata In addition to saving the trained model, the system stores metadata about the training configuration. This creates an inference “contract” that helps the system remember how the model expects data later.

6. Auto-Configured Inference Finally, the saved metadata is used to automatically configure the prediction interface so that users can apply the model consistently without manually rebuilding the setup.

This pipeline was intentionally designed around reliability and novice comprehension, not maximum benchmark performance. The core idea is that a good beginner tool should make it easy to reach a safe and usable baseline first.

2. Building a Prototype: NovaClass

To put the abstract pipeline into practice, we implemented NovaClass, a prototype system for novice-friendly automation of Transformer-based text classification. We chose text classification because it is widely useful—for example in grading, spam detection, and sentiment or emotion analysis—while still being complex enough that current Transformer workflows are difficult for non-experts to configure manually.

NovaClass includes several key features:

• Dataset inspection at upload time Users upload a CSV file, and the system highlights column types, missing rows, label balance, and basic statistics. Users can also inspect class distributions and preview the first rows of the dataset. This helps users understand their data before training begins.

• A simplified configuration interface The interface only exposes decisions novices are expected to understand, especially which columns contain the input text and which column contains the target labels. On page 6 of the paper, the interface is shown with configuration options on the left and an integrated contextual assistant on the right, designed to explain choices in plain language.

• Safe defaults and automated training The system runs a reproducible pipeline with sensible defaults, aiming to help users produce a working classifier on their first attempt. Metadata is saved automatically to reduce mismatches later during inference.

• One-toggle cascade classification NovaClass also includes a simplified way to use a more advanced strategy: cascade classification. Instead of training a single multi-class model, the system can automatically decompose the task into a hierarchy of binary classifiers. On page 7, the paper shows how this is presented in the interface as a one-toggle option, making a more advanced technique accessible without additional configuration burden.

• Metadata-driven inference The inference interface uses saved metadata such as label order, encoders, and strategy so that users can apply the model without manually reconstructing the training setup. The system also shows both the predicted label confidence and full class-probability distribution.

• A conversational assistant NovaClass includes a context-aware assistant that explains metrics like accuracy, recall, and F1-score, and suggests next steps based on what the user is currently doing. The prototype uses IBM Granite 3.3 8B because it followed instructions well in testing, has a large context window, and is small enough to run locally for privacy-sensitive use cases.

3. Conducting a User Study

We evaluated NovaClass through a 24-participant study designed to assess usability, trust, and understanding. The study included users with different levels of prior ML experience and compared novices against participants who had previously trained models.

Participants completed three tasks of increasing complexity:

1. Task 1: Binary text classification Participants configured and trained a classifier for fake news detection. This tested whether the guided configuration helped them choose the right input and label columns.

2. Task 2: Cascade classification Participants trained a cascaded classifier for e-commerce description categorization. This tested whether the interface could make a more advanced classification strategy approachable.

3. Task 3: Diagnosis task Participants analyzed results to identify a class imbalance issue in a dataset. This tested whether the available tools helped users interpret model performance and diagnose problems.

To measure outcomes, we collected:

• demographic and background information,
• PAILQ-6 scores for perceived AI literacy,
• a questionnaire on trust and understandability based on prior AutoML trust research,
• and the User Experience Questionnaire (UEQ).

Results

The study showed that the system worked well as a usable end-to-end workflow, but it also revealed an important gap between task completion and genuine understanding.

Success Metrics

The strongest result was that all 24 participants successfully trained a working binary classifier and a working cascaded classifier. That means the core workflow succeeded in making advanced Transformer-based classification accessible enough for everyone in the study to complete the main tasks.

More detailed task outcomes were:

• Task 1: All participants trained a functioning fake news classifier. Also, 21 of 24 participants (87.5%) reported at least some confidence in training the binary model afterward.
• Task 2: All participants correctly trained a cascade model, and 22 of 24 participants (91.7%) correctly identified the weakest-performing stage in the cascade.
• Task 3: 17 of 24 participants (70.8%) correctly identified label imbalance as the main issue in the dataset.

These results suggest that the combination of guided configuration, automation, and metadata-driven inference successfully reduced common setup errors that would normally block beginners.

User Experience

Participants rated the system positively across all six UEQ dimensions. The highest scores were:

• Efficiency: 1.917
• Attractiveness: 1.778
• Perspicuity: 1.635

The remaining dimensions were also positive:

• Dependability: 1.375
• Stimulation: 1.458
• Novelty: 1.052

The figure on page 10 shows these ratings against the UEQ benchmark, where the strongest results are in efficiency, attractiveness, and perspicuity, with more moderate but still positive scores for dependability and stimulation.

So from a pure UX perspective, NovaClass was received as effective, appealing, and relatively easy to understand.

Trust and Understanding

The more nuanced finding was that experienced users reported significantly higher trust and understanding than novices. This was not just a minor trend; the difference between the groups was statistically significant for the overall trust/understandability score. The median average rating was 4.05 for experienced users versus 3.64 for novices.

Significant differences also appeared on individual questions, including:

• understanding the tool,
• understanding the overall process,
• understanding the data,
• and understanding model evaluation metrics.

The biggest gap appeared in understanding evaluation metrics: experienced users had a median rating of 5, while novices had a median of 2.5. This is especially important because metrics are central to deciding whether a model is actually good enough to use.

Deployment Confidence

When asked whether they would deploy models trained with NovaClass, 17 of 24 participants (70.8%) said yes. Most justified this with statements about “high accuracy” or “high F1-score.” Meanwhile, the participants who said no often cited transparency concerns, such as not knowing what the model based its decisions on or not understanding how the data had been processed.

This result shows a key tension:

• some users may be too willing to trust strong headline metrics,
• while others hesitate because they lack insight into the model’s reasoning and the system’s hidden steps.

In other words, automation can help users finish tasks, but without better explanations, it does not automatically produce appropriate reliance.

Role of the Conversational Assistant

The conversational assistant played a major role across all tasks. Participants frequently used it:

• to select inputs in Task 1,
• to identify the weak stage in Task 2,
• and to diagnose imbalance in Task 3.

In some cases, it was used as often as or more often than static tools like the classification report or confusion matrix. This suggests that context-aware, plain-language guidance is especially valuable for novice users trying to make sense of ML workflows. At the same time, the paper notes that this also raises the cost of occasional hallucinations or imprecise explanations.

Four Design Principles

Based on the study and relevant theory, we derived four design principles for future AutoML systems targeting novices.

P1: Support First-Model Success The system should make it easy for users to produce a working baseline on their first attempt. Early success raises self-efficacy, builds momentum, and reduces the chance that beginners quit before they see value. In practice, this means safe defaults, aggressive upfront validation, automatic preprocessing, one-click training, and advanced functionality that can still be enabled with minimal friction.

P2: Provide Explanations to Build Mental Models and Appropriate Reliance Metrics alone are not enough. Users need simplified, interpretable explanations of what a score means, how to read a visualization, and what limitations remain. The goal is not just to inform, but to help users avoid both over-trust and under-trust. The study’s deployment results showed exactly why this matters.

P3: Use Abstractions and Context-Aware Assistance to Keep Users in Their Zone of Proximal Development Interfaces should hide unnecessary complexity, but still provide targeted support that helps users perform just beyond what they could do alone. This includes adaptive interfaces, stage-aware assistance, and advanced features that stay easy to access. NovaClass’s assistant is one example of this principle in practice.

P4: Ensure Predictability and Safeguards to Strengthen Perceived Control Trust also depends on consistency and control. Systems should be predictable, validate data and settings before training, generate inference interfaces directly from saved metadata, and prevent train-inference mismatches. These safeguards make the workflow feel safer and reduce invisible failure points.

Conclusion

This work shows that making AutoML accessible to novices is not just a technical automation problem. It is equally a human-computer interaction problem. If a tool only automates the hard parts without helping users understand what is happening, then it may reduce effort, but it will not necessarily build trust, correct mental models, or appropriate confidence.

NovaClass demonstrated that a carefully designed workflow can make advanced tasks like Transformer fine-tuning and cascade classification usable even for beginners: every participant completed the main training tasks, and overall UX ratings were positive. But the study also made clear that successful task completion does not automatically mean deep understanding. Experienced users still understood the workflow, the data, and the metrics better than novices.

The main lesson, then, is that good AutoML design for novices should balance:

• automation so people can get started,
• explanations so they understand what happened,
• assistance so they can keep progressing,
• and safeguards so they remain in control.

If we extend this work further, the most important next steps would be:

• building expertise-adaptive interfaces that reveal more detail as users grow,
• improving explainability so metrics and model behavior are easier to interpret,
• and studying how these systems affect learning and confidence over time through longitudinal evaluations.

Ultimately, the project argues for a more human-centered vision of AutoML: not tools that simply automate model building, but tools that help people become more capable, more informed, and more appropriately confident while using AI.

This blog post is based on research presented at EICS 2025. For complete technical details and evaluation results, see the full paper on arXiv.