Flagging Bad Comments and Messages with Local Decision Models
Since this blog runs on its own code, the comments and the contact form are mine, and so is the moderation. That means somebody has to look at every message that comes in, and most of that work is reading things that were never meant for a person.
I wanted a model that reads them first and says: this is real, this is spam, this is a scam, this is nothing. It had to run on my own machine, it had to be fast, and it had to be careful. This is the story of how I got there, including the parts where my own test was wrong.
What I Wanted#
The rules were short:
- It only informs. A badge, a score and a reason in the admin, a line in the Mattermost message. Nothing is hidden and nothing is deleted.
- A real person wrongly flagged is worse than spam let through. If a reader asks me for a day rate and the model calls it spam, that costs me more than one casino message in the inbox.
- Four labels:
legit,spam,scamandjunk, with a score from 0 to 5. - Local and cheap. Ollama on my own machine, no API bill per comment.
I put this in a small package called message-rater, so the blog only talks to an interface and the model behind it can change.
Starting with Gemma#
The first version was the obvious one: ask a chat model. I used Gemma 4 (gemma4:e4b-it-qat) through the OpenAI-compatible endpoint of Ollama, with a prompt that defines the four labels and asks for JSON only:
{
"label": "spam",
"score": 4,
"reason": "generic price probe from a generated-looking sender"
}
It worked, with one trap. Gemma 4 thinks before it answers on that endpoint, and the same verdict took 45 seconds with thinking on and about a second with reasoning_effort: none. That is a one-line config option, but it is easy to miss when the first calls are that slow.
On a first set of 16 messages in English, Dutch and French, three small models all got 16 out of 16: gemma4:e4b-it-qat in 0.65 seconds per message, gemma4:e2b-it-qat in 0.39 and granite4.1:3b in 0.24. The weak spot was generic bait like "Hello, is this still available?" from a sender that looks generated. Three examples in the prompt fixed most of that: on a set of 28 hard cases the model went from 21 right to 27. I also tried a blunt rule against price questions and dropped it again, because it flagged real people asking for a day rate.
So Gemma was good enough to ship, and it is what the blog is wired to. But it writes a verdict, it takes about 0.6 seconds, and it never says how sure it is beyond the score it writes.
Decision Models#
Then I found out that Ollama has a different kind of model now. A decision model does not chat. You give it a text and a question with the possible answers, and it returns a probability for every answer in one forward pass. There is no text to parse and no thinking to switch off.
The first one I looked at was d3 from the vLLM Semantic Router project, which comes in 2B, 4B, 9B and 27B. You call it on /v1/systemone, here shortened:
{
"model": "d3:4b",
"state": "<sender>\nname: Kimberlyqwe\nemail: xqvbn8812@gmail.com\n</sender>\n<text>\nHello, is this still available?\n</text>",
"questions": {
"kind": {
"type": "choice",
"instructions": "The text comes from a website's contact or comment form. Which kind is it? ...",
"criteria": {
"legit": "A real person writing something the owner would want to read: ...",
"spam": "Advertising, SEO or backlink offers, casino, pills, loans, ...",
"scam": "Fraud and phishing: inheritance or lottery, romance or investment schemes, ...",
"junk": "Nothing a person meant to say: gibberish, keyboard mashing, test text, ..."
}
}
}
}
and it answers with numbers:
{
"type": "choice",
"choice": "scam",
"probabilities": {
"legit": 0.0001,
"spam": 0.003,
"scam": 0.9966,
"junk": 0.0003
},
"confidence": 0.98
}
A probability is what I was missing. At the time of writing there are 14 of these in the decision category of the Ollama library, from several publishers, so d3 was the start and not the end of the list.
A First Test That Told Me Nothing#
I put together 38 messages, spam, scams, junk and real ones in four languages, and ran d3 and Gemma on them:
| Model | Right | Speed per message |
|---|---|---|
| d3:4b | 36 of 38 | 165 ms |
| d3:2b | 35 of 38 | 77 ms |
| gemma4:e4b | 38 of 38 | 631 ms |
Gemma looked perfect, and d3 looked four times faster with two or three misses. Both conclusions were wrong. My Gemma prompt contains three examples, and three of my test messages were those examples. Of course it knew them. I also had only 38 messages, so a single miss moved the percentage by 2.6 points.
A Bigger Test#
I needed more messages than I could write, so I generated them. A local Qwen3.6 (35B) wrote messages for every combination of a category, a style and a language: English, Dutch, French, German and Spanish; real questions, bug reports, blunt criticism, SEO offers, casino spam, romance scams, fake invoices, keyboard mashing and so on. After removing duplicates that gave 583 messages: 161 legit, 167 spam, 159 scam and 96 junk.
| Model | Exact label | Real messages flagged | Unwanted passed as legit | Speed |
|---|---|---|---|---|
| d3:4b | 78.0% | 11.8% | 18.5% | 5.7 msg/s |
| d3:2b | 71.7% | 19.3% | 19.0% | 12.1 msg/s |
| gemma4:e4b | 74.3% | 1.9% | 26.3% | 1.5 msg/s |
Now there was a trade-off and not a winner. d3:4b was more accurate and four times faster, but it flagged one real message in eight, and Gemma almost never did. Given my first rule, that made Gemma the safer one.
Parallel Execution#
The blog does not rate messages one by one. It rates them in the background, so I also measured what happens with several requests at once, on a 150-message subset:
| Clients | 1 | 2 | 4 | 6 | 12 |
|---|---|---|---|---|---|
| d3:4b | 5.7 | 6.0 | 5.9 | 5.8 | 5.8 |
| d3:2b | 12.7 | 13.2 | 13.2 | 13.2 | 13.2 |
| gemma4:e4b | 1.5 | 2.3 | 3.3 | 3.9 | 3.9 |
(messages per second)
Gemma scales up to the six parallel slots my Ollama allows (OLLAMA_NUM_PARALLEL=6) and then stays flat. d3 does not scale at all: one client or twelve, the speed is the same. On Ollama 0.40.16 it looks like the decision endpoint handles one request at a time. Even so, d3:4b alone is faster than Gemma with six clients, and the 2B is more than three times faster. For a blog, none of this matters much, but it is good to know before you size a queue.
My Test Data Was Wrong#
The generated labels were my next suspicion. The generator was asked for "a spam message in Dutch", and sometimes it wrote something perfectly normal. A German "tolles Tutorial! Kannst du mir sagen wie ich das Volume mounten muss?" had come out labeled as a scam. If the model calls that message legit, my test says the model is wrong.
So I had every message where any model disagreed with its label read again, 208 of them, and fixed the label when the text clearly said something else. 104 labels changed. The biggest group, 38, were real praise or questions that had been generated as spam, and 14 "junk" messages were really real remarks. Another 30 messages I could not call either way ("Hey", "Goede artikelen!", a thumbs-up emoji), so I took them out. That left 553 messages: 216 legit, 118 spam, 147 scam and 72 junk. To see whether I had only cleaned up the messages that happened to disagree, I checked 45 random ones that all models agreed on, and none of those had a wrong label.
The same three models on the cleaned set:
| Model | Exact label | Real messages flagged | Unwanted passed as legit |
|---|---|---|---|
| d3:4b | 91.7% | 7.9% | 4.5% |
| d3:2b | 85.9% | 14.4% | 4.5% |
| gemma4:e4b | 90.2% | 0.5% | 9.8% |
Everyone got better, and the difference between d3:4b and Gemma shrank to a trade: d3 catches more unwanted messages, Gemma flags fewer real ones.
Asking Better, and Admitting When Unsure#
My next idea was the prompt. I tried five ways of asking the same question with d3:4b: the baseline, a shorter one, one that says "choose legit unless it is clearly unwanted", one with example messages inside the criteria, and a two-answer version (wanted or unwanted).
The more interesting idea came from the output. d3 gives a probability, so why does a message have to be decided at all? If it is 95% sure a message is unwanted, flag it. If it is 60% sure, say so and leave it to me. I call that third state unsure:
- the probability that a message is unwanted is
1 - P(legit) - at or above a high threshold it is spam, scam or junk (whichever of the three is likeliest)
- below 0.5 it is legit
- in between it is unsure, and a person decides
To keep myself honest I picked the threshold on one half of the messages and measured on the other half, which had 95 real messages and 184 unwanted ones.
| Prompt | Threshold | Real messages: flagged / unsure / ok | Unwanted: flagged / unsure / passed as legit |
|---|---|---|---|
| Baseline | 0.9 | 0 / 11 / 84 | 88% / 8% / 4% |
| Short | 0.5 | 3 / 0 / 92 | 93% / 0% / 7% |
| Legit by default | 0.5 | 2 / 0 / 93 | 92% / 0% / 8% |
| With examples | 0.8 | 1 / 5 / 89 | 91% / 4% / 5% |
| Wanted or unwanted | 0.8 | 0 / 5 / 90 | 93% / 2% / 5% |
| Gemma (same half) | 1 flagged | passes 9% |
Two things stand out. The wording of the prompt matters much less than I expected: all five are within a few points of each other. The threshold matters a lot. With a plain cutoff at 0.5, the baseline prompt flags 11 of the 95 real messages. With 0.9 and an unsure band it flags none, and the 11 become unsure. The share of unwanted messages that passes as legit stays at 4%, against 9% for Gemma.
Putting It in the Rater#
I added this to message-rater as a second driver, so Gemma stays the default and nothing changes for anyone who does not ask for it:
message_rater:
driver: systemone
base_url: 'http://localhost:11434/v1'
model: 'd3:4b'
unwanted_threshold: 0.9
unsure_threshold: 0.5
The rating gets a fifth label, unsure. It is never unwanted, whatever its score, so no code can delete a message because of it. The "reason" is no longer a sentence but the probabilities (spam 88%, scam 4%, legit 7%), because a decision model does not write sentences. The rater refuses to start if the unsure threshold is above the unwanted one, and the cache key contains the model and both thresholds, so changing them never serves old verdicts.
To check the real rater and not only my Python script, I ran it on the same held-out half, 279 messages. None of the 95 real messages was flagged, 7 went to unsure and 88 were fine. Of the unwanted ones 88% were flagged, 9% were unsure and 3% passed as legit, at 5.6 messages per second. That is 0.1.0-rc.5.
Other Decision Models#
After d3 I tried the others in the library that fit on my machine, with the same prompt and the same cleaned messages. The prompt was written for d3, so this favors d3 a little. AUC is the chance that a random unwanted message gets a higher score than a random real one, so 1.0 is perfect and it needs no threshold.
| Model | Size | AUC | Speed | Real flagged / unsure / ok (of 95) | Unwanted flagged / unsure / passed |
|---|---|---|---|---|---|
| kev (4B, q4) | 2.7 GB | 0.991 | 4.7/s | 1 / 4 / 90 | 92% / 4% / 4% |
| d3:4b | 5.8 GB | 0.990 | 5.8/s | 0 / 11 / 84 | 88% / 8% / 4% |
| tev1:4b | 4.6 GB | 0.986 | 5.3/s | 3 / 6 / 86 | 86% / 10% / 4% |
| d1 (3B) | 3.7 GB | 0.983 | 10.8/s | 2 / 0 / 93 | 88% / 0% / 12% |
| d1-omni (600M) | 0.85 GB | 0.842 | 43/s | almost everything unsure | |
| laya (multilingual) | 0.66 GB | 0.812 | 47/s | almost everything unsure | |
| tev1:0.8b | 0.83 GB | 0.770 | 22/s | passes 39% |
The top three are a tie. The differences between them are smaller than what I can measure with 95 and 184 messages. d1 is twice as fast and a little worse at ranking, and it has no good middle ground: at its best threshold it lets 12% of the unwanted messages through. The three small ones, under one gigabyte, are not usable for this. They are almost never sure, in either direction.
A few models did not make it. pplx-decider-flash only comes in MLX formats, which need Apple hardware, and so does strands-decider. nimble-v3 has a non-commercial license. I also did not check the licenses of the ones that did run, and for a project of mine that is not a detail: d3 is Apache 2.0 on its Ollama page, while I could not find one for kev and d1 has its own license.
What I Have Not Done#
- The 9B models (
nimble,clef-flashand others) I left for later. The 4B models already pass about 4% of the unwanted messages, so I do not expect a lot more from them. - Real messages. Everything above is generated, then relabeled. The real test is a dry run on the actual inbox, with me as the judge.
- The blog itself is still wired to Gemma. Switching means showing "unsure" in the admin first, since an unsure message is a state the admin did not have.
Final Thoughts#
If you want to flag spam locally, the thing I would take from this is not a model name. A decision model gives you a probability, and a probability lets you say "I don't know", which a chat model writing a verdict does not. That saved me more than any prompt change did. And whatever you test with, read your own test data first: half of what I first called a model mistake was a wrong label.
I will report back when I have run it on the real inbox.
Comments
There are no comments yet. Be the first to write one.
Write a comment