What is ds4?
ds4 is an open-source inference engine for running selected large language models locally on hardware people can own. The repository calls the project DwarfStar and describes a small, native implementation focused on DeepSeek, GLM and Qwen models. ds4 targets local model users and developers who want conversational inference, coding assistance or model serving on supported personal machines and workstations.
ds4 deliberately prioritizes a limited model set rather than acting as a general GGUF runner. Model loading, prompt rendering, tool calls, KV state, the HTTP server and the coding agent are built and tested together. The repository also includes tools and data for GGUF processing, quantization calibration, quality evaluation and speed testing.
What ds4 can do
ds4 combines local inference with interactive, coding and serving workflows. Hardware support includes Metal on Macs, NVIDIA CUDA systems and ROCm on Strix Halo systems such as the Framework Desktop. Some model capabilities depend on the backend, while supported distributed configurations can combine resources across GPUs or machines.
- ds4 runs multi-turn conversations through an interactive CLI.
- ds4 provides a native coding agent that runs inference directly.
- ds4 exposes an HTTP server for API access and multiple sessions.
- ds4 supports SSD streaming when models exceed available memory.
- ds4 supports tensor parallelism and pipeline parallelism in documented configurations.
- ds4 provides vision support for selected models and hardware backends.
How ds4 works
ds4 starts with cloning the GitHub repository, choosing a hardware-specific build and downloading a supported model. The platform guides cover prerequisites, memory sizing and setup for Apple Silicon, DGX Spark, Strix Halo and generic CUDA configurations. Downloads go into the repository's gguf directory, and repeating the download command can resume an interrupted transfer.
ds4 offers separate executables for interactive inference, the native coding agent and the HTTP server. The CLI supports reading files, changing context settings and interrupting generation. The coding agent stores sessions and can reuse compatible local KV snapshots to avoid rebuilding a prompt. External clients including Pi, OpenCode, Codex CLI and Claude Code use the server instead.
Who ds4 is for
ds4 is for developers and local model users working with supported Macs, NVIDIA machines or Strix Halo systems. The documented examples include personal inference, coding assistance and multi-user serving. Users with larger models can consider SSD streaming or supported distributed setups, but must account for model memory, runtime buffers and context requirements.
- ds4 supports local conversations with selected open-weight models.
- ds4 supports programming workflows through its native coding agent.
- ds4 supports multi-user LLM serving on multiple CUDA cards.
- ds4 supports combining machines to run models that need more memory.
ds4 access and licensing
ds4 is available as an open-source GitHub repository under the MIT license, rather than as a described subscription service. The documented access path is to obtain the source, build the appropriate backend and download project-supported model files. Hardware selection and model memory requirements are part of that setup, and the repository provides platform-specific guidance.
ds4 strengths and limitations
ds4 brings the CLI, coding agent and server into a self-contained project tested alongside its inference implementation. Support for SSD streaming and distributed execution provides options for models that do not fit in a single machine's memory. However, the project explicitly describes the software as fast-changing and beta quality, with possible instabilities and regressions despite release testing.
- ds4 requires the GGUF files produced by the project.
- ds4 may remove model support when a better replacement arrives.
- ds4 cannot yet save coding-agent sessions containing images.
- ds4 warns that saved conversations and traces may contain private information.
ds4 FAQ
What is ds4?
ds4 is an open-source local inference engine for selected DeepSeek, GLM and Qwen language models. The repository describes the project as DwarfStar and includes an interactive CLI, a native coding agent and an HTTP server, with hardware support through Metal, CUDA and ROCm.
Is ds4 free and open source?
ds4 is available as an open-source project under the MIT license on GitHub. The documented setup involves cloning the repository, building for supported hardware and downloading project-supported models, rather than signing up for a hosted subscription.
What hardware does ds4 support?
ds4 supports Metal on Macs, NVIDIA CUDA systems including DGX Spark and Ada Lovelace cards, and ROCm on Strix Halo systems such as the Framework Desktop. Model choice affects memory requirements, and smaller machines may need SSD streaming.
Can ds4 run any GGUF model?
ds4 is not a general GGUF runner and requires GGUF files produced by the project. Model support is deliberately narrow and may change when replacements become available, so users need to follow the repository's supported model downloads and hardware guidance.
Does ds4 work with coding clients?
ds4 provides an HTTP server and a client setup guide for Pi, OpenCode, Codex CLI and Claude Code. ds4 also includes a native coding agent that runs inference directly without a separate HTTP server and supports saving and resuming eligible sessions.
What are ds4's main limitations?
ds4 is beta-quality software with possible instabilities and regressions. ds4 requires project-produced GGUF files, supports a changing selection of models and cannot yet save coding-agent sessions containing images. Hardware and memory requirements also vary by model and execution configuration.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




