Gemma-4-31B-LuneLight

The five terms are multiplied, so the model is encourage to not over optimize for any one aspect of the reward. The highest rewards can only be met in a response that satisfies every reward.

Out-of-character notes are removed along with stat blocks, message headers etc. For the length and language checks, HTML, markdown, zero-width characters and Zalgo marks are stripped and look-alike Cyrillic or Greek letters are mapped back to Latin.

A reply scores 0 if it:

• was truncated due to any reason
• has a missing, broken or leaked thinking block, or any thinking when thinking is off,
• is a short refusal or stub, or is abnormally short,
• switches into another script or has too few common English words, beyond what the prompt itself uses.

With thinking on, the first 600 thinking words are free. After that the score falls smoothly, reaching 0 at 1,200 words.

Tense normalisation. To avoid *most* of the problems with comparing different types of AI and human written text, it's normalized where possible ("drops" and "dropped" both become "drop").

Mining the dictionary. Every pair of content words that appear within five words of each other in the model's replies is summed. A pair has to appear across at least 5% of scenarios and then pass two tests:

• Dunning's log-likelihood test (Dunning 1993), which checks whether the two words show up together more often than chance. This is a pass/fail filter.
• Delta P (Gries 2013), which measures how much seeing one word raises the chance of seeing the other and take the stronger of the two directions. It filters out pairs of common words that only meet because both are frequent, and it sets the base penalty for each pair that is kept.

The dictionary ends up with four kinds of entries:

• Hub words: words where at least half of all uses sit inside one of their overused pairs or frames, along with those partner pairs. These are template phrases that the model has learned to vary and easily avoid detection via ngrams (they just swap to the second most common option).
• Standalone pairs: strongly linked pairs that are widespread on their own.
• Frames: a hub word joined to the grammar word next to it ("scent of", "lets out"). Penalising content pairs alone lets the model keep the habit and swap the noun ("scent of cedar", "scent of rain"), and frame allows that to be punished. A frame only counts if the model uses it at least 7× as often as humans, so ordinary grammar doesn't get caught.
• Excess words: single words used at least 8× as often as in human writing, across at least 20% of scenarios ("shimmering", "ozone", "predatory"). Traditional slop words the model grossly overuses.

Comparing against human writing. Everything above is weighted by how much more the model uses it than humans do. Anything the model uses at twice the human rate or less is dropped, which leaves normal phrases, templates and words like "eyes closed" or "head shakes" alone. Above that, the penalty grows with the gap.

Scoring a reply. Each reply is scanned for pairs, frames, stacked adjectives ("a thick, suffocating cloud") and excess words. A longer cliché like "heart hammering against ribs" counts once instead of once for every pair inside it. When a reply hits the same hub word several times, the strongest hit counts in full and each extra hit counts for less, using BM25 saturation (Robertson & Walker 1994). Importantly, because the reward looks primarily at pairs / frames / hubs, the model is not punished when it uses a slop word in a new way (i.e. The scent was strong is allowed vs the scent of ozone which gets punished). The total is measured per 1,000 words.

Negation. This catches "not X, but Y" emphasis: a negation such as "not", "never" or "n't" followed within 12 words by a contrast such as "but", "rather", "instead", a dash or semicolon, or a next sentence starting with "Instead". Guards skip interrupted speech and asides. The patterns the model uses 12× to 82× more than humans add to the slop score.

Calibration. Phrase and negation hits are added into one slop density and a single curve is applied. Up to the human 75th percentile, slop scores 1. Above it the score decays exponentially, at a rate set so a typical base-model reply scores 0.4 while the densest human writing still scores at least 0.8.

A bonus for everyday expressions like "at least", "one of" and "as soon as" that humans use and the model avoids. Just looks for 2 and 3 word phrases that are widespread across forum RP threads and sources, also common in other human writing, and used at least 2.5× more by humans than by the model. Phrases with past tense verbs, personal pronouns or scene-specific nouns were removed so the list works in any scene and tense, and overlapping phrases were merged.

Each phrase counts once per reply and is weighted by how rarely the model being trained uses it, so phrases it learns give rewards can't be abused. Credit is measured per 1,000 words and reaches full marks at the human 25th percentile. A reply with none scores 0.70, and replies under 40 words are left at 1.

Dialogue is removed and the first word of each narration sentence is taken as its opener. One leading word like "Then" or "Slowly," is skipped first so it can't hide a "He… He… He…" chain. Two measures are taken:

• The normalised Herfindahl–Hirschman index, equivalent to Simpson's index for a finite sample (Simpson 1949), for how concentrated openers are across the whole reply. It is 0 when every opener differs and 1 when they are all the same.
• I also manually count how often an opener comes back in the next sentence or the one after, since a long reply can cheat the index.

Both are free up to the human 75th percentile and reach the full penalty at the base model's 90th percentile. The worse of the two sets the score, down to 0.75. Replies with fewer than 5 narration sentences get the human average of 0.96, so dropping narration isn't a way around the check.

Same idea as negation, regex checks whether the reply's dialogue or short italic thoughts copy the user's last message. Before matching, stutters and repeated letters are collapsed, contractions are expanded and "I" and "you" are treated as the same, so "You're crazy" echoed as "I'm crazy?" still counts. Character names, titles, lines the user asked the character to say and replies like "I love you too" are ignored.

Then echoing is split into a few types depending on what check caught it. Models tend to only overuse a few types of echoing, so only those types are punished