Developing an AI-Based Prototype to Support Students and Teachers with Exercises

Related Publication: Ontwikkeling van een AI-gebaseerd prototype dat studenten en docenten ondersteunt met oefeningen

Full Thesis

This blog post describes my master’s thesis work on using AI in education, supervised by Davy Vanacken and Gustavo Rovelo Ruiz at UHasselt.

What I Did

My thesis explored how AI—especially Large Language Models (LLMs)—can be integrated into education in a way that is useful, transparent, and responsible.

With the emergence of LLMs, students now have the opportunity to ask AI questions. However, teachers have concerns about the impact on teaching quality—particularly regarding the accuracy of AI-generated answers, the consistency of those answers, privacy, and the potential reduction in student-teacher interaction.

The goal of this thesis was to:

• Explore applications and challenges of AI (especially LLMs) in education
• Create a prototype demonstrating how these applications can be implemented
• Ensure that AI enhances rather than replaces student-teacher interaction

More specifically, the project was structured around three objectives:

1. Identify useful AI tools and their challenges in education through a literature review and exploratory experiments with LLM-generated answers discussed with teachers.
2. Develop a student-facing prototype that gives responsible access to an LLM, offers a dynamic FAQ, and recommends exercises based on earlier difficulties.
3. Develop a teacher-facing prototype that helps teachers manage the LLM, review frequently asked questions, inspect learning difficulties, and use data visualisations to improve course materials.

The broader research question was not whether AI should replace teachers, but how it can be integrated in a way that supports both students and teaching teams while preserving human oversight.

How I Did It

The research followed a structured process inspired by the ADDIE model: analysis, design, development, implementation, and evaluation.

1. Literature Review I conducted a broad review of existing AI applications in education, including machine learning, personalised learning systems, chatbots, expert systems, intelligent tutors, and virtual learning environments. I also focused on the main challenges of LLM use in education: unreliable answers, inconsistent outputs, issues around assessment, privacy, intellectual property, and the risk of weakening teacher-student interaction.

2. Experiments with LLMs Before designing the prototype, I explored how LLMs behaved when solving exercises and discussed those outputs with teachers. This helped clarify where LLMs were useful, where they failed, and what kinds of controls teachers would need to trust them in an educational setting. This phase also informed the role of prompt engineering in steering LLM responses toward course-specific expectations.

3. Concept Development Based on the literature and experiments, I defined four core design goals for the prototype:

   • Give students responsible access to an LLM
   • Build a dynamic FAQ that evolves from student questions
   • Recommend exercises based on earlier difficulties
   • Make the whole application transparent about which parts use AI and why

4. Prototype Implementation I built two applications using Windows Presentation Foundation (WPF) with C#:

   • one for teachers
   • one for students

   Both applications communicate with a central Python Flask server, which acts as the hub between the interfaces and the AI/ML components.

   The system combines several components:

   • An open-source LLM for student questions
   • KeyBERT for keyword extraction from exercises and questions
   • Sentence Transformers for grouping similar questions using sentence similarity
   • A collaborative filtering recommender for exercise suggestions
   • A database for courses, questions, ratings, and related interaction data

5. Design Decisions A key design choice was to rely on locally manageable, open-source models rather than a fully external commercial AI provider. In the prototype, the default model was GPT4All Falcon, chosen because it can run on relatively modest hardware and gives teachers more control. Teachers can also configure course-specific model URLs through Hugging Face-compatible APIs and provide custom instructions such as an opening prompt and instructions that are appended to every student message.

Results

The prototype demonstrated how multiple AI-assisted features can be combined into a practical educational workflow.

Teacher-side results

The teacher application gives control over:

• Which LLM students use
• How the LLM behaves, through course-specific instructions
• Which keywords are associated with each exercise
• Which FAQ items are added
• Which trends and difficulties appear in student data

Teachers can:

• Add a course and configure the LLM
• Input exercises and curate automatically extracted keywords
• Review grouped student questions
• Receive suggestions for new FAQ items
• Inspect exercise completion rates
• View perceived exercise difficulty
• Use visualisations to better understand where students struggle

This makes the AI layer adjustable rather than opaque.

Student-side results

The student application focuses on guided support:

• Students can chat with the LLM in a familiar interface
• They can select a course and optionally a specific exercise, which links questions to the right context
• They can browse a dynamic FAQ with previously answered questions and teacher-added hints
• They can receive exercise recommendations based on earlier ratings and difficulty patterns

An important addition is the option to use the system in an anonymous mode. This allows students to contribute data about difficult topics and common questions without attaching that data to their identity, while clearly warning them that anonymous use limits the teacher’s ability to give individual feedback.

Core technical outcomes

The prototype successfully demonstrated several innovations:

• Teacher-Customizable LLMs: Teachers can configure AI responses to align with their teaching style and course requirements
• Exercise Recommender: A collaborative filtering system predicts which exercises may help a student practise weak areas
• Dynamic FAQ: Student questions are grouped through sentence similarity, and commonly recurring questions can be proposed to teachers as FAQ additions
• Data Collection for Insights: Questions, ratings, completion data, and detected keywords help teachers understand learning bottlenecks

Key Insight: Rather than diminishing interaction between teachers and students, the collected data can help teachers maintain and even improve that interaction by making student difficulties more visible and by supporting more targeted feedback.

At the same time, the thesis also identified important limitations:

• LLM reliability is still a real issue
• The chosen LLM was efficient, but limited (for example, it only responded in English)
• The recommender suffers from the classic cold start problem, since recommendations are weaker when little prior data is available
• Some desirable features, such as stronger emotional intelligence or deeper reflection, would require far more computational resources or further model adaptation

Because this thesis focused on concept validation and implementation, it did not yet include a full user study with classroom participants. That evaluation is the most important next step.

Conclusion

This work shows that AI in education does not have to mean replacing teachers. Instead, it can be used to create a more supportive and data-informed learning environment—provided that the system is transparent, locally manageable, and designed around teacher control.

The thesis started with a broad analysis of AI opportunities and risks in education, then translated those findings into a working prototype for both students and teachers. The result is a concrete demonstration of how LLMs, keyword extraction, sentence similarity, and recommender systems can be combined in an educational setting while addressing important concerns such as transparency, privacy, and meaningful human oversight.

Several lessons stood out during the project:

• Control matters: teachers need ways to shape and constrain AI behaviour
• AI works best as a complement: not as a replacement for teacher-student interaction
• Transparency increases trust: users should understand which parts are AI-driven
• Local deployment helps privacy: avoiding unnecessary external data sharing is a major advantage

Future Work: The most important next step is a real user study comparing a control group and a test group over a longer period. Such a study should evaluate learning outcomes, retention, usability, transparency, student autonomy, and whether teachers still feel in control of the learning process. There is also room to improve the recommender, add more prompt-engineering support for teachers, and explore additional AI applications such as AI-assisted course design or tutoring.

If I were to extend this work, I would focus first on evaluating the prototype with real users, then on refining the recommendation logic and making the LLM support more accessible for teachers with little AI experience.

This blog post is based on my master’s thesis completed at UHasselt in 2024. For the complete technical details, please refer to the full thesis.