Guides

Everything we learned running open models on our own GPUs: what fits in VRAM, how to expose a model safely, and how to build private SI features on top.

Getting started

How to self-host an OpenAI-compatible API on your own GPURun open LLMs like Qwen3 and GLM-4 on your own GPU and expose them through an OpenAI-compatible API with keys, quotas and metering, without opening a port.2 min read

Hardware

How much VRAM do you need to run an LLM?VRAM needed for Qwen3, GLM-4, gpt-oss, DeepSeek R1 distill, Mistral Small, Gemma 3 and more, the rule of thumb behind it, and which GPUs fit each model.2 min read
Run Qwen3 32B on a Hetzner GPU server (GEX45)Set up a Hetzner GEX45 with an RTX PRO 4000 Blackwell 24 GB, install the NVIDIA driver and gpuos, and serve Qwen3 32B through an OpenAI-compatible API.2 min read

Use cases

Private RAG on your own GPU: embeddings and chat, self-hostedBuild retrieval-augmented generation that never sends documents to a third party: BGE-M3 embeddings and Qwen3 32B answers on your GPU, called through LangChain.2 min read

Compliance

GDPR and LLMs: keeping AI and SI data in the EUWhat changes for GDPR when you run open models on your own GPUs instead of a US API, what gpuos stores, and a checklist for your records of processing.2 min read

Comparisons

Ollama vs gpuos: when a single machine is not enoughOllama is the easiest way to run an LLM on one machine. gpuos builds on it for teams: API keys, quotas, metering, several nodes and no open port.2 min read

Try it on your own GPU

Free for one GPU. Install the agent, deploy a model and call it through one OpenAI-compatible API.

Start free Read the quickstart