🫧⋆｡˚ 💎 Gemsicle-31B 💎 🫧⋆｡˚

A fun, experimental model merge that aims for more creative prose and less stereotypical portrayal of characters, without losing smartness and prompt adherence, compared to its base model google/gemma-4-31b-it.

🐀 About the Model

This model was put together as a fun project between Ateron and Nimbz, the trope heavily inspired by the visual of a humming, glowing and colorful "Gemsicle" crafted for everyone in our little Blazed Forge.

We put our heads together because we both liked the EQ and overall intelligence of the base Gemma 4 31B it model, but we had to acknowledge that it is full of slop phrases, stereotypical character display and narrow paths with almost no swipe variety. The released fine-tunes seemed promising, but they made evident that there is a trade-off between training the base model for more creativity and better prose and preserving the overall intelligence of the model.

We tried to bring both of these worlds together - Preserve the strengths of the base and enhance it with the creative capabilities of our selection of sophisticated fine-tunes.

• Our Primary Focus: Creative Writing, Less Slop, Authentic Character Portrayal, Preserving Prompt Adherence

🗣️ Prompt Format

Please refer to the original google/gemma-4-31b-it for the correct chat template.

Let your frontend handle the chat template if possible (e.g., Chat Completion in SillyTavern).

• For Reasoning: Add <|think|> at the very beginning of the system prompt. Thinking happens between <|channel>thought\n and <channel|> tags.

<|turn>system
<|think|>
You are a helpful assistant<turn|>
<|turn>user
Hello<turn|>
<|turn>model
Hi there<turn|>
<|turn>user
How are you?<turn|>
<|turn>model

⚙️ Recommended Samplers

These are the settings we found work best to keep the model creative without it getting too repetitive or losing the plot. Feel free to experiment, as a different behavior might be desireable for others.

ST Chat Completion Parameters:

temperature: 1.0
top_k: 0
top_p: 0.97
min_p: 0.03
adaptive_target: 0.6
adaptive_decay: 0.5
samplers: ["top_k", "top_p", "min_p", "temperature", "adaptive_p"]
chat_template_kwargs: {"thinking": true, "enable_thinking": true}

🐈‍⬛ Merge Details

We used mergekit to combine the strengths of several different models in a precise, two-phase metallurgical process:

Phase 1: The Creative Sauce (SLERP)

First, we took AuriAetherwiing/G4-31B-Musica-v1 and ConicCat/Gemma4-GarnetV2-31B - two highly creative fine-tunes - and merged them together in an even 50:50 approach with Spherical Linear Interpolation (SLERP). This served as the creative portion for the better wording and swipe variety we were seeking for, creating the Symphony-v2 merge.

models:
  - model: ./Gemma4-GarnetV2-31B
  - model: ./G4-31B-Musica-v1
merge_method: slerp
base_model: ./Gemma4-GarnetV2-31B
parameters:
  t:
    - value: 0.5
dtype: bfloat16

Phase 2: Pressing The Gem (DARE-TIES)

In the second step, we injected specific weights from a randomly found, but somehow very capable fine-tune called p-e-r-e-g-r-i-n-e/Sprinkle-Gemma-4-31B, which inherits enough of the intelligence and prompt ahderence known from the base model, while bringing in enough variable but realistic prose and character portrayal to be of use.

Using the DARE-TIES method, we merged the Sprinkle-Gemma-4-31B with the Symphony-v1 and created our gem with the qualities we sought for.

models:
  - model: ./gemma-4-31B-it

  - model: ./Sprinkle-Gemma-4-31B
    parameters:
      density: [0.90, 0.80, 0.60, 0.65]
      weight: [0.25, 0.30, 0.70, 0.60]

  - model: ./Symphony-v2
    parameters:
      density: [0.05, 0.10, 0.20, 0.35]
      weight: [0.0, 0.0, 0.30, 0.50]

merge_method: dare_ties
base_model: ./gemma-4-31B-it
parameters:
  int8_mask: true
dtype: bfloat16

🔗 GGUF Quants

🤝 Special Thanks

A huge thanks to the creators of the community tools and open-source models that made this possible!

• Google DeepMind: For providing the base model.
• The Open-Source Community: For providing the fine-tunes we used for this merge.
• Mergekit: First, arcee-ai for making model merging incredibly accessible and easy.
• Mergekit Fork: Secondly, zerofata for the Gemma 4 fork of arcee-ai's Mergekit. Without it we wouldn't be able to create such a gem!
• Gemini 3.1 Pro: For providing us with support to achieve our goal and the image prompt ideas for our model card.