Schools across Europe have made real progress on student online safety — stronger policies, better-configured filters, and clearer guidance for staff. But there's a category of AI risk that filtering alone was never designed to address.
What happens once a student is on a permitted platform? What does the conversation actually contain? Is a student using an AI chatbot to explore something they'd never say to a teacher — a worry about their mental health, a question about self-harm, a request for content that would trigger alarm in any other context?
Digital safeguarding software exists to answer exactly those questions. And as AI tools become more emotionally accessible — and more emotionally risky — for young people, the schools that have this visibility are the ones best placed to intervene early.
Safeguarding software for schools works by monitoring activity on school devices — analysing what students type, search for, and share — and generating alerts when the content suggests a student may be at risk. It operates in the background, without interrupting lessons or flagging normal school use.
The core mechanism: when a student's digital activity matches a risk indicator — specific terms, phrases, or patterns associated with safeguarding concerns — the system captures context and routes an alert to the relevant staff member. In most schools that's a Designated Safeguarding Lead (DSL), whose job is then to assess the alert, determine whether it warrants a direct conversation, and log the outcome.
This is quite different from filtering. Filtering says: you can't go there. Monitoring says: here's what someone did, and here's what it might mean.
Before the proliferation of AI chatbots, most digital safeguarding concerns centred on browsing behaviour — the sites students tried to access, the searches they ran. That's still relevant, but AI has opened a new and arguably more significant channel of risk.
Students now have access to conversational AI tools that feel personal, private, and non-judgmental. Research consistently shows that young people sometimes disclose things to AI that they won't tell adults — including distress signals, disclosures of abuse, and expressions of suicidal ideation. Some are using AI tools as informal emotional support, without recognising that an AI has no duty of care, no professional training, and no ability to escalate a crisis appropriately.
Student wellbeing software that monitors these interactions gives safeguarding staff a window they didn't have before, not to spy on students, but to catch early warning signs that a student may need support before a situation becomes an emergency.
The emerging risks are also broader: students using AI tools to produce or request inappropriate content, engaging with unmoderated AI platforms that have no age-related safeguards, or forming unhealthy patterns of AI dependency that may indicate underlying social or emotional difficulties.
Automated monitoring systems flag potential risks — but automated systems can also misread context. A student researching historical atrocities for a school project looks very different from a student who is genuinely in distress, even if some of the language overlaps.
This is why the most effective digital safeguarding schools are investing in human-moderated monitoring: a model where expert reviewers assess flagged content before alerts are escalated to DSLs. Human moderators bring interpretive judgment that no algorithm fully replicates — reading tone, context, and language in a way that significantly reduces both false positives (which waste DSL time) and false negatives (which miss real risks).
In the context of AI-related safeguarding, human moderation is particularly valuable because:
For DSL safeguarding software to genuinely support the work of a Designated Safeguarding Lead, it needs to deliver context, not just data. Human moderation is how that context gets built.
GDPR compliant school software is a non-negotiable starting point for any monitoring solution used with students in Europe. But compliance extends beyond data protection: EMEA education regulators increasingly expect schools to demonstrate active, documented safeguarding practices — including evidence of how digital risks are identified and responded to.
DSL reporting software that produces clear logs of alerts, assessments, and outcomes provides exactly that evidence. It turns monitoring from a passive technical measure into an active part of your safeguarding framework — and it gives your DSL the documentation trail that matters most during inspections or serious case reviews.
| Qoria Monitor combines automated detection with human-moderated review, giving DSLs across Europe the contextual insight they need to act quickly and confidently on AI-related safeguarding risks. Learn more → |
Neither filtering nor monitoring works as well in isolation. Schools that treat them as complementary, a filter controlling access, monitoring revealing behaviour, have the most complete picture of their students' digital lives, and the best foundation for early intervention.
Add a clear AI usage policy, regular staff training, and documented risk assessments, and you have a safeguarding framework that keeps pace with the way AI is actually evolving.