Run Alfred 40B · Q4_0 on your own GPU

Compare GPU memory for Alfred 40B · Q4_0

Catalog estimate at Q4_0, not a measured benchmark. Chat estimates assume a short context; embedding memory depends on input and batch size. The comparison includes a 5% screening margin; it does not establish runtime performance. More in how much VRAM an LLM needs.

Check this estimate against your GPU with the VRAM calculator

• RTX 4060 Ti 8 GBMore headroom needed
• RTX 4070 Ti Super 16 GBMore headroom needed
• RTX 4000 Ada 20 GBMore headroom needed
• RTX 3090 / 4090 24 GBMore headroom needed
• RTX PRO 4000 Blackwell 24 GB (Hetzner GEX45)More headroom needed
• NVIDIA L4 24 GBMore headroom needed
• RTX 5090 32 GBEstimate fits
• A100 / H100 80 GBEstimate fits
• RTX PRO 6000 Blackwell 96 GB (Hetzner GEX131)Estimate fits

Call Alfred 40B · Q4_0 with the OpenAI SDK

Use the supported chat completions endpoint with your gpuos base URL, API key and deployed model id. Validate any model-specific features your app needs. Setup for LangChain, Continue, Open WebUI and more is in integrations.

from openai import OpenAI

client = OpenAI(base_url="https://gpuos.si/v1", api_key="gpuos_key_…")
stream = client.chat.completions.create(
    model="alfred-40b",
    messages=[{"role": "user", "content": "Summarize our refund policy."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

import OpenAI from "openai"

const client = new OpenAI({ baseURL: "https://gpuos.si/v1", apiKey: process.env.GPUOS_API_KEY })
const reply = await client.chat.completions.create({
  model: "alfred-40b",
  messages: [{ role: "user", content: "Summarize our refund policy." }],
})
console.log(reply.choices[0].message.content)

curl https://gpuos.si/v1/chat/completions \
  -H "Authorization: Bearer $GPUOS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "alfred-40b", "messages": [{"role": "user", "content": "Hello"}]}'

Questions

How much VRAM does Alfred 40B · Q4_0 need?

The catalog estimate is 26.5 GB at Q4_0 for a short context (4–8K tokens). The RTX 5090 32 GB clears this estimate with screening headroom in the comparison list. This is a planning estimate, not a tested hardware requirement. Validate context length, batch size and concurrent requests on your node.

Can I use Alfred 40B · Q4_0 commercially?

Alfred 40B · Q4_0 is released under the Upstream terms. Review the publisher's license before deployment. Rights have not been classified automatically.

Is Alfred 40B · Q4_0 compatible with the OpenAI API?

Through gpuos, Alfred 40B · Q4_0 uses the supported /v1/chat/completions endpoint with the model id "alfred-40b". Configure your OpenAI-compatible client with the gpuos base URL and API key. Check model-specific features and any other API operations your app needs before switching.

Which gpuos plan includes Alfred 40B · Q4_0?

Alfred 40B · Q4_0 is part of the full catalog on the Pro plan, $29 per GPU per month.

Related models

• Qwen3 32BChat · ≈ 22 GB · Apache 2.0
• GLM-4 32B 0414Chat · ≈ 20.5 GB · MIT
• Qwen3 30B A3BFast MoE · ≈ 20 GB · Apache 2.0
• DeepSeek R1 Distill Qwen 32BReasoning · ≈ 22 GB · MIT