Moderation and safety
How does AI moderation work?
AI moderation uses language models to read messages and flag abuse, spam or rule breaks. Learn how it works, where it fails, and how to use it safely.
AI moderation uses a machine-learning model to read content and judge whether it breaks a rule, then flags it for a human or takes a limited action such as deleting it. Unlike a fixed word filter, it considers meaning and context, so it can catch abuse that avoids banned words. It is also fallible, which is why good setups keep humans in charge of serious decisions.
How does AI moderation decide what to flag?
A typical flow has four steps:
- Collect: the system gathers recent messages from the channels it is allowed to watch.
- Evaluate: a language model is given the messages plus instructions describing what counts as a violation, such as clear harassment or disruptive spam.
- Score or label: the model returns a decision for each message, often with a confidence level.
- Act or escalate: the system either logs the result, asks a human to review it, or performs an action that the server owner has permitted.
Because the model reads language rather than matching a list, it can understand that a message is a threat even when no single word in it is banned. The same strength creates the main risk: it is making a probabilistic judgment, not a lookup.
AI moderation vs keyword filters
| Keyword or regex filter | AI moderation | |
|---|---|---|
| Understands context | No | Partly |
| Catches misspellings and workarounds | Only if you list them | Often |
| Predictable | Very | Less so |
| Explains itself | Always (it matched X) | Only if designed to |
| Cost of mistakes | Over-blocking harmless words | Misjudging tone or sarcasm |
The two work well together. Cheap, exact rules handle known bad strings and links, and AI handles the fuzzy middle. See how to stop spam and scam links on Discord for the rule-based side.
What are the limits of AI moderation?
- False positives: jokes, quotes, song lyrics, and reclaimed language can look abusive without being so.
- False negatives: coded language, new slang, and slow-burn harassment spread across many messages can slip through.
- Missing context: the model usually sees a short window, not your server's history or inside jokes.
- Language and culture gaps: performance is uneven across languages and dialects.
- Overconfidence: a model can sound sure and still be wrong, a behavior related to AI hallucinations.
These limits are the reason to start conservative, review the logs, and widen the settings gradually rather than switching everything to automatic punishment on day one.
How should you deploy AI moderation safely?
- Begin in a review-only mode where the AI flags but changes nothing.
- Send results to a private staff channel and read them for a week or two.
- Note what it gets wrong and adjust your policy or channel selection.
- Allow the mildest action first, such as deleting a message, before considering timeouts or bans.
- Keep a human appeal path and tell members what is automated.
Pair this with clear written rules, which give both the AI and your moderators something concrete to enforce. See how to write Discord server rules and the broader content moderation overview.
Doing this with Based
Based's automatic moderation module is off by default. When enabled, it reviews messages in the channels you select in batches (30 messages every 600 seconds by default) and looks for clear abuse. The default review policy is conservative, and an optional stricter policy also covers disruptive spam. The maximum action defaults to "Review only", which never changes messages or members; you can raise it to deleting messages, warn and delete, timeout, kick, or ban. A private log channel records what happened, and message excerpts in the log are off by default. Image review is optional.
There is also an Apps menu action called "Review message" that gives an advisory opinion on one message. Commands also work with the "." prefix, for example .help. See the automod docs and the AI automod feature page.
Frequently asked questions
Is AI moderation accurate?
It is often good at obvious abuse and weaker at sarcasm, in-jokes, and coded language. Treat its output as a strong signal that a person should be able to review, not as a final verdict.
Can AI moderation ban people automatically?
Only if the tool allows it and the server owner turns that on. A safer practice is to start with review-only or message deletion and escalate only after checking the logs.
Does AI moderation read every message?
It only reads what it can access. Good tools let you limit it to selected channels and exclude private ones, and they should say how much they keep.
Do I still need human moderators if I use AI?
Yes. People handle context, appeals, and fairness, and they are accountable to the community in a way software is not.
Keep learning
- Moderation and safety
What is content moderation?
Content moderation is reviewing and acting on user posts to keep a community safe and on-topic. Learn how it works on Discord and why it matters.
- Moderation and safety
Discord moderation best practices
Practical Discord moderation best practices: clear rules, tiered responses, least-privilege roles, logging, verification and a staff process that scales.
- AI basics
What are AI hallucinations?
AI hallucinations are confident but false or invented answers from a model. Learn why they happen, how to spot them, and how to reduce them.
- Moderation and safety
How to stop spam and scam links on Discord
How to stop spam and scam links on Discord: link filters, verification levels, role permissions and member habits that block phishing and invite spam.
- Moderation and safety
How do Discord bots handle your data?
What data Discord bots can see and store, how permissions limit access, what to check in a privacy policy, and how to keep your server safe.
Try it in your server
Based is an AI Discord bot in 300+ servers. Every slash command also works with the "." prefix.