AppDev UK All articles
Trust & Safety

Before You Ship the Reply Button: Moderation Design for UK Apps in 2026

AppDev UK
Before You Ship the Reply Button: Moderation Design for UK Apps in 2026

Photo: moderation dashboard on laptop screen with smartphone showing comment thread, via images.pexels.com

A community board is a feature request. A direct message is a growth lever. Then a lawyer reads your release notes, points out that both make you a user-to-user service under the Online Safety Act 2023, and suddenly the sprint has a legal annex.

This piece is for the team that's just had that conversation. Not the panic — the plan.

The scope creep nobody scopes

User-to-user means content one person uploads that another person can encounter. That's broader than most product teams assume. It covers:

The plugin point matters. Outsourcing the screen doesn't outsource the duty. If your users are meeting each other inside your app, the obligations follow the product.

What the regulator actually asks for

The heavy lifting for user-to-user services has been live in stages: illegal content duties from March 2025, children's safety duties from July 2025, and Ofcom's codes sitting underneath both. Enforcement is not theoretical — the fines run to £18m or 10% of qualifying worldwide revenue, whichever is higher, and senior managers can be named.

The practical asks, stripped of jargon:

  1. A risk assessment that's suitable and sufficient, in writing, done before launch. Not a paragraph in a Notion page. A document that names the harms relevant to your service, who could be exposed, and what you're doing about each.
  2. A children's access assessment. Can under-18s get in? If they can reach primary priority content, you're in age assurance territory — see our earlier piece on picking a route you can defend.
  3. Terms that describe what you actually do. Copied terms are worse than vague ones, because you'll be measured against them.
  4. Reporting and complaints routes that work. A report button that fires an email into a shared inbox is not a route.
  5. Consistent enforcement of your own terms. The duty isn't 'have rules'. It's 'apply them'.
  6. Records. Decisions, timestamps, reasoning, and evidence that you reviewed the risk assessment when the product changed.

Moderation is a product surface, not a policy PDF

The teams that struggle are rarely the ones that didn't care. They're the ones that treated moderation as an admin job for whoever was free on a Tuesday.

Design it like a screen. That means:

A queue with states. New, triaged, actioned, appealed, closed. Each state has an owner and a clock. If a report can't be actioned without a developer deploying code, you don't have a moderation system, you have a suggestion box.

Harm categories mapped to your risk assessment. The categories in your internal tooling should be the same list as the harms in the document. If they diverge, you can't evidence anything.

A decision log. Who removed what, when, under which rule, with what note. Retention policy agreed with legal, not guessed by engineering.

An appeal path. Users who lose access to a community they've built will write to you. Give them a route and a response time.

An escalation ladder. Child sexual abuse material, credible threats, self-harm content and terrorism have no place in a general queue. They need a named person, an out-of-hours contact, and a reporting route to the Internet Watch Foundation or police as appropriate. Staff who review this content need support too — that's a rota and a wellbeing plan, not a bullet point.

The three documents that save you later

If a client, an investor or a regulator asks for evidence next year, you want three things in a folder with dates on them.

The risk assessment. Living, versioned, re-run when you add a feature that changes who can see what. Adding photo attachments to a previously text-only board is a change worth recording.

The moderation playbook. Internal and blunt: what gets removed, what gets labelled, what gets escalated, what gets reported. And a public-facing summary in plain English so users know the rules and how to challenge a decision.

The decision log export. A sample of recent actions, the queue volumes, the time-to-action, the appeals. Numbers, not vibes.

Failure modes we keep seeing

A sensible launch gate

Before the reply button ships: the risk assessment is signed off and dated; the children's access question has a written answer; the admin tooling can hide, remove and suspend without a deploy; the reporting route has been tested by someone outside the team; the escalation list has names and phone numbers; the terms have been read by someone who didn't write them; and the decision log records its first entry.

None of that is glamorous. But social features are the part of the app where the product team's best intentions meet the worst of the internet — and where 'we'll sort moderation after launch' has stopped being an option in the UK.

All Articles

Related Articles

Borrowing the State's Front Door: GOV.UK One Login and the Commercial App

Borrowing the State's Front Door: GOV.UK One Login and the Commercial App

Five Stars, Thin Evidence: Auditing App Store Ratings Before the CMA Comes Calling

Five Stars, Thin Evidence: Auditing App Store Ratings Before the CMA Comes Calling

The Browser and the Borrowed Phone: Demoing Your App to a UK Buying Panel

The Browser and the Borrowed Phone: Demoing Your App to a UK Buying Panel