INSIGHT: Bridging the Student-Teacher Gap in Times of Large Language Models

Related Publication: INSIGHT: Bridging the Student-Teacher Gap in Times of Large Language Models

Full Paper

This blog post describes research presented at the D-SAIL Workshop on Transformative Curriculum Design in Palermo, Italy (2025). Co-authored with S. Vanbrabant, D. Vanacken, and G. Rovelo Ruiz.

What We Did

This work focused on a problem many teaching staff now face: if students increasingly ask their questions to general-purpose LLMs instead of to instructors or teaching assistants, teachers lose an important source of feedback about where students are struggling. That makes it harder to adapt course materials, prepare targeted support, and maintain meaningful student-teacher interaction. INSIGHT was designed as a response to that problem.

The core goal was not to block AI use, but to create a privacy-aware, human-centered environment where students can use an LLM while their interactions still generate useful teaching insights. In other words, we wanted to explore how AI could help bridge the student-teacher gap instead of widening it.

This research builds directly on the ideas from my master’s thesis. In that earlier work, we explored how AI could support students and teaching staff with exercises through features such as LLM access, a dynamic FAQ, and teacher-side analytics. INSIGHT takes that direction further in a more focused proof of concept: it emphasizes modular deployment, explicit privacy choices, and the use of student questions as a signal for improving face-to-face support.

The paper identifies a clear tension in educational AI:

Opportunities:

• AI can help personalize learning and teaching support
• Students can get quick answers while solving exercises
• Teaching staff can reduce repetitive question load
• Interaction data can reveal patterns in student difficulties

Challenges:

• Student-teacher interaction may degrade if too many questions move to AI
• Teaching staff may miss cues about knowledge gaps
• Privacy becomes a major concern when educational data is collected
• Students may become over-reliant on AI tools if they are used uncritically

So the main research contribution was to design and present a modular prototype that gives students access to an LLM in a monitored, privacy-aware setting while equipping teaching staff with data-driven insights they can use to improve in-person teaching.

How We Did It

The project started with interviews and focus-group style discussions with teaching staff at Hasselt University. This happened in two phases: first through semi-structured interviews across different courses, and later by presenting an initial prototype during an internal university workshop focused on AI. These conversations shaped the design priorities of INSIGHT. The strongest recurring concern was that staff feared losing contact with students and, with that, losing insight into students’ understanding of the course material.

System Design: INSIGHT

We developed INSIGHT as a modular proof-of-concept system centered around an INSIGHT Core. As shown in the architecture diagram on page 3, this core combines a keyword extraction component, a sentence similarity component, and a course database, and connects them to both a local LLM and a user interface that can be adapted to different course contexts.

Core Components:

1. AI-Assisted Exercise Support

   • Students can ask questions to an LLM while working on exercises
   • The LLM is the main interaction method in the prototype
   • In this implementation, we used Llama 3.2 3B, mainly because it offers relatively fast responses and is feasible to run on consumer hardware
   • The modular setup makes it possible to swap in newer or more capable models later, depending on the needs of the course

A key design choice is that INSIGHT does not interfere with the LLM’s behavior through prompt engineering or artificial restrictions. The idea is that students should interact with it naturally, rather than being nudged into a constrained workflow. The system then analyzes those interactions afterward to generate useful signals for teaching staff.

1. Keyword Extraction

   • INSIGHT first needs to know which topics belong to each exercise
   • To do that, it uses KeyBERT to extract an initial keyword list from the exercise text
   • Teaching staff then review and refine those keywords manually in a mixed-initiative process
   • These reviewed keywords become the course’s topic vocabulary

The example shown in the keyword selection interface on page 4 illustrates why human review matters: automatic extraction is helpful, but imperfect. The paper explicitly notes odd outputs such as “tree tree” in the extracted keywords for one exercise, which demonstrates why final control remains with the teaching staff.

1. Dynamic FAQ Generation

   • Student questions are embedded in the same vector space as the topic vocabulary
   • The nearest keyword is used to assign a topic label to a question
   • Similar questions are grouped using the all-MiniLM-L6-v2 sentence embedding model and cosine similarity
   • Recurrent questions can then be turned into FAQ items
   • INSIGHT caches the LLM’s answer for grouped questions, and teaching staff can edit that answer before reuse

This makes the FAQ both dynamic and supervised. It grows from real student questions, but the teaching staff still verify and adjust the answers, which reduces variability and keeps the system human-in-the-loop.

1. Teaching Staff Insights Dashboard

   • INSIGHT collects and analyzes student questions to the LLM
   • It also tracks FAQ usage and exercise difficulty ratings
   • The teaching staff interface includes a dynamic FAQ and two visualizations

   • According to the interface shown on page 5, these graphs display:

     • how often each FAQ item is viewed
     • how frequently different course topics appear in student questions

These visualizations give teaching staff a fast way to spot recurring confusion, identify weak points in course materials, and prepare more targeted support during office hours or interactive sessions.

Modular Design

INSIGHT is designed to be:

• Flexible: The core system is separate from the user interface and LLM, so it can be deployed in different higher education courses without changing the core logic. The proof of concept uses data from UHasselt’s Algorithms and Data Structures course, but the architecture is intentionally broader than that one course.
• Privacy-aware: Students can choose an anonymous mode, allowing staff to see the data without attached names. In addition, the system uses Ollama to run the LLM locally, so queries can remain private and are not shared with third parties.
• Extensible: Because the components are modular, new models, interfaces, and analytics features can be added later without redesigning the full system.

This privacy design is especially important because the system deliberately collects interaction data. INSIGHT’s approach is to make that collection transparent and consent-based, rather than hidden or mandatory.

Implementation

Technically, the system consists of:

• a central INSIGHT Core
• a database storing course information and interaction data
• a keyword extraction module
• a sentence similarity module
• a local LLM
• separate teacher and student user interfaces

The student interface, shown on page 6, combines three functions:

• a chat window for asking the LLM questions
• access to the FAQ
• an exercise difficulty rating mechanism that gives explicit feedback to the teaching staff

The teacher interface, shown on page 5, allows staff to:

• review and edit FAQ items
• add FAQ entries manually
• inspect topic frequencies
• inspect FAQ view counts

Results

As a proof of concept, INSIGHT demonstrates that it is possible to combine LLM access, keyword analysis, dynamic FAQs, and teacher-facing analytics into a single educational support system that strengthens visibility into student difficulties instead of hiding them.

System Capabilities

INSIGHT successfully demonstrated:

1. Better Student-Teacher Connection The system is explicitly designed so that student questions to AI still become useful signals for the teaching staff. Rather than replacing face-to-face support, it helps staff understand where students need help before or during in-person interactions.

2. Data-Driven Personalization By analyzing question topics, FAQ usage, and difficulty ratings, teaching staff can:

   • identify which topics cause recurring confusion
   • adapt course materials over time
   • prepare more targeted explanations for students or groups
   • provide more personalized face-to-face support

3. Dynamic FAQ with Human Oversight Frequently recurring questions can be grouped, answered once, and then reviewed by staff. This reduces repetitive question load while still allowing teachers to verify accuracy and refine explanations.

4. Privacy-Conscious Monitoring The system shows that monitoring does not have to mean intrusive surveillance: students can opt into identified sharing or choose anonymous use, and local deployment reduces third-party data exposure.

Insights from Deployment

This paper presents INSIGHT primarily as a proof of concept, not a full classroom validation study. The demonstrated value lies in the kinds of insights the system can surface for teaching staff, especially in interactive course settings.

Based on the prototype, staff can derive insights such as:

• which exercise topics are producing the most questions,
• which questions recur often enough to justify an FAQ item,
• which FAQ items students revisit frequently,
• and which exercises students rate as more difficult.

Those signals can help answer practical teaching questions like:

• Which concept needs clearer explanation in the next session?
• Which exercise caused unexpected confusion?
• Which FAQ answers should be strengthened or expanded?

The paper also discusses important limitations. Students may avoid monitored systems and ask questions to outside LLMs instead, which could bias the teacher’s view of student understanding. That means the usefulness of INSIGHT depends not only on the tool itself, but also on whether students perceive value in using it and whether teaching staff actively act on the insights it provides.

Conclusion

INSIGHT shows that AI in education does not have to reduce human contact by default. If designed carefully, it can create a shared space where students get quick AI support and teaching staff still retain visibility into learning difficulties, common misconceptions, and emerging needs.

Key Contributions:

1. A modular architecture for integrating AI support into higher education courses
2. A mixed-initiative keyword analysis pipeline to map student questions to course topics
3. A dynamic FAQ system built from real student questions and supervised by teaching staff
4. A teacher-facing analytics layer that supports more personalized face-to-face teaching
5. A privacy-aware deployment model with anonymous mode and local LLM execution

What stands out most in this work is the design philosophy: INSIGHT does not try to stop students from using LLMs, nor does it assume AI should replace teachers. Instead, it accepts that these tools are already part of students’ reality and tries to make their use safer, more transparent, and more useful for actual teaching practice.

The paper also makes clear that technical design alone is not enough. Responsible AI use in education depends on dialogue, trust, and shared expectations between students and staff. INSIGHT supports that conversation, but it cannot replace it.

Future Work:

The next steps described in the paper are:

• conducting a cognitive walkthrough with new teaching staff interested in piloting the system,
• deploying INSIGHT in real courses at Hasselt University,
• empirically validating whether it improves support and learning,
• exploring knowledge tracing for more advanced personalization,
• and using the collected data to support adaptive learning and more inclusive learning experiences over time.

If we were to extend this line of work, the most important next step would be a real classroom study: not just to test whether the system works technically, but to understand whether students actually use it, whether teachers trust it, and whether it genuinely improves student-teacher interaction in practice.

This blog post is based on research presented at the D-SAIL Workshop, 2025. For complete details, see the full paper.