Discussion
We introduce CodeGeeX4-ALL-9B, the open-source version of the latest CodeGeeX4 model series. It is a multilingual code generation model continually trained on the , significantly enhancing its code generation capabilities. Using a single CodeGeeX4-ALL-9B model, it can support comprehensive functions such as code completion and generation, code interpreter, web search, function call, repository-level code Q&A, covering various scenarios of software development. CodeGeeX4-ALL-9B has achieved highly competitive performance on public benchmarks, such as and . It is currently the most powerful code generation model with less than 10B parameters, even surpassing much larger general-purpose models, achieving the best balance in terms of inference speed and model performance.
We have been developing a Local AI to create a decentralized developer kit (DDK). We've alternated between using CodeQwen for its speed, larger context window, and DeepSeek for its higher accuracy but very limited context window, which is crucial for code generation within the existing code context.
But wow! We just tested CodeGeeX4-ALL-9B (Quantized 3 bit). It accepts more than 1500 tokens as context, evaluates prompts at 172.99 tokens/s, and generates more than 25 tokens/second on my M1 machine while consuming 4.5 GB of RAM.
The future of all personal AI is local, not cloud-based. We all have the machines to make that possible; we just need to look around!
GGUF
If you are facing issues with llama.cpp, just update it to the latest release.
News
