Oobabooga

What is Oobabooga?

Oobabooga is an open-source desktop application designed for running large language models (LLMs) locally on your own machine. It prioritizes privacy and user control, offering a 100% private experience with no telemetry. The app supports text generation, vision capabilities, tool-calling, and web search, all accessible through a user-friendly interface and an OpenAI/Anthropic-compatible API. This makes it a versatile tool for developers, researchers, and enthusiasts who want to experiment with LLMs without relying on cloud services.

Key features include support for multiple model backends such as GGUF (via llama.cpp), ExLlamaV3, and Transformers, allowing users to choose the best performance for their hardware. The application is available for macOS, Linux, and Windows, with portable builds that include all dependencies, making setup straightforward—simply download, unzip, and double-click. For advanced users, a full installation option is available for additional backends, training, image generation, and extensions like TTS and voice input.

Oobabooga is ideal for use cases such as local AI assistants, content generation, code completion, and research. Its API compatibility allows integration with existing tools and workflows. Technical highlights include support for CUDA, Vulkan, ROCm, and CPU-only execution, ensuring broad hardware compatibility. Memory usage can be estimated with the GGUF Memory Calculator, and recommended quantized models are available via LocalBench. The app also supports multi-file models like 16-bit Transformers and EXL3 models, which require a subfolder structure. Overall, Oobabooga provides a powerful, private, and flexible platform for local LLM deployment.