Hourly Usage
Performance Metrics
Avg. Total Time
75.83s
Avg. TTFT
22.08s
Avg. Prefill TPS
1753.28
Avg. Gen TPS
18.41
Model Information
Context Size
262144
Quantization
r64
Engine
vllm
Creation Method
LoRA
Model Type
Qwen35
Chat Template
Qwen3.5
Reasoning
Yes
Vision
Yes
Parameters
27B
Added At
4/4/2026
license: mit datasets:
- zerofata/Instruct-Anime
- zerofata/Gemini-3.1-Pro-SmallWiki
- zerofata/Gemini-3.1-Pro-GLM5-Characters
- zerofata/Roleplay-Anime-Characters base_model:
- Qwen/Qwen3.5-27B
BlueStar v1
Qwen3.5 27BAn experimental tune on Qwen 3.5 27B.
Designed for conversational assistant tasks and RP.
Model feels pretty creative and has some nice moments. Couple brainfarts and bits of repetition occasionally, but nothing out of the normal. (The qwen team themselves recommend a presence penalty of 1.5. Yikes)
Non thinking and thinking are both supported. Thinking has reduced censorship as the original thinking refusals didn't seem to generalize well to the new format I gave it.
Creation Process: SFT
SFT on approx 23 million tokens (12 million trainable). New is some Gemini Synth data which replaces some of my lower quality datasets.
About 10% of the dataset included reasoning for creative assistant tasks. This reasoning seems to have generalized quite well to other parts of the model and heavily reduces the token usage of thinking.
I think this model still needs a pass over with DPO to try and tackle the repetition and some of the weird oddities of the original instruct model, but that'll need to wait. I've been overspending training these models recently.
Trained using MS-Swift.
MS-Swift Config
PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True' \
USE_HF=True \
WANDB_PROJECT=Qwen3.5-27B-SFT \
CUDA_VISIBLE_DEVICES=0,1,2,3 \
NPROC_PER_NODE=4 \
swift sft \
--model Qwen/Qwen3.5-27B \
--tuner_type lora \
--dataset '/workspace/think_dataset.jsonl' \
'/workspace/nothink_dataset.jsonl' \
--torch_dtype bfloat16 \
--bf16 true \
--use_liger_kernel true \
--lora_rank 128 \
--lora_alpha 16 \
--use_rslora true \
--target_modules all-linear \
--freeze_llm false \
--freeze_vit true \
--freeze_aligner true \
--per_device_train_batch_size 1 \
--gradient_accumulation_steps 16 \
--num_train_epochs 2 \
--learning_rate 2e-5 \
--warmup_ratio 0.05 \
--max_length 10752 \
--split_dataset_ratio 0.01 \
--add_non_thinking_prefix true \
--load_from_cache_file true \
--group_by_length true \
--eval_steps 200 \
--save_steps 200 \
--save_total_limit 10 \
--logging_steps 1 \
--dataloader_num_workers 8 \
--output_dir output/Qwen3.5-27B-SFT-Model \
--report_to wandb