Avg. Total Time
6.35s
Avg. TTFT
4.21s
Avg. Prefill TPS
332.44
Avg. Gen TPS
24.77
Context Size
32768
Quantization
r64
Engine
aphrodite
Creation Method
LoRA Finetune
Model Type
Llama70B
Chat Template
Llama 3
Reasoning
No
Vision
No
Parameters
70B
Added At
12/22/2024
license: llama3.1 datasets:
art by spindlehorse toons. i do not own this image. all credit goes to them!
compute sponsored by ShuttleAI
this is an experimental l3.1 70b finetuning run... that crashed midway through. however, the results are still interesting, so i wanted to publish them :3
use l3 instruct. not writing an example, almost everything supports it by now and i don't feel like remembering the stupid tags
a mix of publicly available claude synth data in various domains, mostly. also systemchat but i refuse to credit cognitive computations for anything decent so forget i said it