Avg. Total Time
27.43s
Avg. TTFT
5.07s
Avg. Prefill TPS
5505.30
Avg. Gen TPS
17.56
Context Size
32768
Quantization
INT8
Engine
aphrodite
Creation Method
FFT
Model Type
Llama70B
Chat Template
Llama 3
Reasoning
No
Vision
No
Parameters
70B
Added At
12/22/2024
No Model Read Me file available.