Everything on Edge0-145B
1 mention, newest first.
Edge0 open-sources its 145B and 120B models with an inference framework
Edge0 open-sourced Edge0-145B and Edge0-120B with its inference framework and deployment tools. Weights and an Edge0 App are available. Reported laptop results for Edge0-145B: 29GB peak memory, 1,100+ tokens/s prefill, 80+ tokens/s decode, approximately 120W typical power.
Usable today