Hourly Usage
Performance Metrics
Avg. Total Time
59.02s
Avg. TTFT
4.48s
Avg. Prefill TPS
437.28
Avg. Gen TPS
25.20
Model Information
Context Size
32768
Quantization
r64
Engine
aphrodite
Creation Method
Merge
Model Type
Llama70B
Chat Template
Llama 3
Reasoning
No
Vision
No
Parameters
70B
Added At
12/22/2024
base_model:
- flammenai/Mahou-1.5-llama3.1-70B
- mlabonne/Hermes-3-Llama-3.1-70B-lorablated
- rombodawg/Rombos-LLM-V2.6-Nemotron-70b
- nbeerbower/Llama-3.1-Nemotron-lorablated-70B
- nbeerbower/Llama3.1-Gutenberg-Doppel-70B
- flammenai/Llama3.1-Flammades-70B library_name: transformers tags:
- mergekit
- merge license: llama3.1
Llama-3.1-Saoirse-70B
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Model Stock merge method using mlabonne/Hermes-3-Llama-3.1-70B-lorablated as a base.
Models Merged
The following models were included in the merge:
- flammenai/Mahou-1.5-llama3.1-70B
- rombodawg/Rombos-LLM-V2.6-Nemotron-70b
- nbeerbower/Llama-3.1-Nemotron-lorablated-70B
- nbeerbower/Llama3.1-Gutenberg-Doppel-70B
- flammenai/Llama3.1-Flammades-70B
Configuration
The following YAML configuration was used to produce this model:
models:
- model: nbeerbower/Llama-3.1-Nemotron-lorablated-70B
- model: flammenai/Mahou-1.5-llama3.1-70B
- model: nbeerbower/Llama3.1-Gutenberg-Doppel-70B
- model: flammenai/Llama3.1-Flammades-70B
- model: rombodawg/Rombos-LLM-V2.6-Nemotron-70b
merge_method: model_stock
base_model: mlabonne/Hermes-3-Llama-3.1-70B-lorablated
dtype: bfloat16