LLM Inference - Neural Digest

Sign in Subscribe

LLM Inference

A collection of 1 post

Microsoft Open-Sources BitNet: The 1-Bit LLM Inference Framework That Runs 100B Models on Your CPU

Microsoft Open-Sources BitNet: The 1-Bit LLM Inference Framework That Runs 100B Models on Your CPU

Microsoft open-sources BitNet (bitnet.cpp), an inference framework that runs 100B parameter 1-bit LLMs on consumer CPUs with up to 6x speedup and 82% energy reduction.