Providing feedback on student writing is time-intensive, but using online generative-AI tools like ChatGPT raises concerns about student privacy and teacher accountability. This session reports on an open-source, offline feedback assistant that runs on teachers’ desktops/laptops using small, local large language models (LLMs). The goal is to reduce time spent on writing feedback without putting student work online while maintaining pedagogically appropriate feedback for different proficiency levels. Methodologically, the development of the assistant required a purpose-built benchmark to compare candidate LLMs on one-shot judgement tasks, such as judging the effectiveness of topic sentences and aligning claims and evidence accurately. Results from the one-shot tests on controlled, synthetic learner texts show that tiny models like the one-billion parameter TinyLlama model frequently hallucinate and make logical errors. Even larger local models, such as the 20-billion parameter GPT-OSS-20B model, prefer overly academic registers in their feedback, reducing suitability for lower-proficiency learners. For CALL practice, the findings highlight a key trade-off: privacy-focused offline feedback demands nuanced GenAI engineering but cannot substitute for teacher oversight. Teachers who wish to preserve student privacy will need new skills to review, correct and contextualize model output and to have an explicit understanding of model limitations.