AI Feedback That Works Without Internet
A published IEEE study on how small AI models can run directly on a student's own device and support coding practice even when internet access is limited or unavailable.
Overview
Access to intelligent feedback during programming practice should not depend on a reliable internet connection — yet most AI-powered tutoring systems assume exactly that. SHIELD (Secure Handling of Information in EducationaL Devices) was designed to test whether that assumption can be eliminated. For this, we used our previous work on Prompt Literacy, and deviced a system prompt stratigically designed to act as pedagogical guardrails for open models.
The project deploys a small, quantized language model directly on the learner’s device using LM Studio, eliminating any network dependency during active use. The model responds to student code submissions in real time, offering natural-language feedback calibrated to introductory computing concepts — all without transmitting student data to an external server.
Why On-Device?
The equity argument for on-device AI in education is straightforward: connectivity gaps track socioeconomic fault lines. Students in under-resourced schools, rural districts, or developing-world contexts routinely face intermittent or absent internet access. If intelligent tutoring systems require cloud inference, they will systematically exclude the students most likely to benefit from additional instructional support.
SHIELD addresses this by treating connectivity as a design constraint, not a design assumption. The system is built to work fully offline, with no degradation in core functionality when the network is unavailable.
The Research Design
The SHIELD study was conducted with students learning introductory Python programming. Participants submitted code to a locally-running instance of Llama 3.2 (3B), which generated formative feedback in natural language. The study examined:
- Feedback quality: Whether device-local model responses were educationally appropriate, accurate, and non-misleading.
- Perceived helpfulness: How students rated the AI feedback relative to no feedback or generic error messages.
- System viability: Whether the inference latency and resource requirements of on-device deployment were acceptable in a real classroom context.
Findings
The SHIELD study found that device-local models at the 3-billion-parameter scale can produce feedback that students rate as helpful and that does not significantly mislead them about the correctness of their code. Inference on consumer-grade hardware was fast enough to be usable in practice. These findings support the argument that the threshold for educationally viable on-device AI has already been crossed — the barrier is now deployment and institutional will, not technical capability.
The research was presented at the IEEE International Conference on Advanced Learning Technologies (ICALT) 2025.
Implications
SHIELD contributes to a growing body of work arguing that equitable AI in education requires infrastructure-aware design. Systems optimized for high-bandwidth, cloud-connected classrooms are not neutral — they embed assumptions about who has access. On-device deployment is one pathway toward AI tutoring that does not replicate the connectivity divide it should be helping to close.