Run GLM-5.3 754B UD-IQ2_M (community GGUF) on your own GPU

Compare GPU memory for GLM-5.3 754B UD-IQ2_M (community GGUF)

Catalog estimate at IQ2_M, not a measured benchmark. Chat estimates assume a short context; embedding memory depends on input and batch size. The comparison includes a 5% screening margin; it does not establish runtime performance. More in how much VRAM an LLM needs.

Check this estimate against your GPU with the VRAM calculator

• RTX 4060 Ti 8 GBMore headroom needed
• RTX 4070 Ti Super 16 GBMore headroom needed
• RTX 4000 Ada 20 GBMore headroom needed
• RTX 3090 / 4090 24 GBMore headroom needed
• RTX PRO 4000 Blackwell 24 GB (Hetzner GEX45)More headroom needed
• NVIDIA L4 24 GBMore headroom needed
• RTX 5090 32 GBMore headroom needed
• A100 / H100 80 GBMore headroom needed
• RTX PRO 6000 Blackwell 96 GB (Hetzner GEX131)More headroom needed

Call GLM-5.3 754B UD-IQ2_M (community GGUF) with the OpenAI SDK

Use the supported chat completions endpoint with your gpuos base URL, API key and deployed model id. Validate any model-specific features your app needs. Setup for LangChain, Continue, Open WebUI and more is in integrations.

from openai import OpenAI

client = OpenAI(base_url="https://gpuos.si/v1", api_key="gpuos_key_…")
stream = client.chat.completions.create(
    model="glm-5.3-754b-ud-iq2-m",
    messages=[{"role": "user", "content": "Summarize our refund policy."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

import OpenAI from "openai"

const client = new OpenAI({ baseURL: "https://gpuos.si/v1", apiKey: process.env.GPUOS_API_KEY })
const reply = await client.chat.completions.create({
  model: "glm-5.3-754b-ud-iq2-m",
  messages: [{ role: "user", content: "Summarize our refund policy." }],
})
console.log(reply.choices[0].message.content)

curl https://gpuos.si/v1/chat/completions \
  -H "Authorization: Bearer $GPUOS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "glm-5.3-754b-ud-iq2-m", "messages": [{"role": "user", "content": "Hello"}]}'

Questions

How much VRAM does GLM-5.3 754B UD-IQ2_M (community GGUF) need?

The catalog estimate is 246.5 GB at IQ2_M for a short context (4–8K tokens). This is a planning estimate, not a tested hardware requirement. Validate context length, batch size and concurrent requests on your node.

Can I use GLM-5.3 754B UD-IQ2_M (community GGUF) commercially?

GLM-5.3 754B UD-IQ2_M (community GGUF) is released under the GLM-5.3 License. GLM-5.3 has its own license. Model-as-a-Service businesses exceeding $10 billion in aggregate annual revenue require Z.AI's security review before commercial use. Review the publisher's full terms.

Is GLM-5.3 754B UD-IQ2_M (community GGUF) compatible with the OpenAI API?

Through gpuos, GLM-5.3 754B UD-IQ2_M (community GGUF) uses the supported /v1/chat/completions endpoint with the model id "glm-5.3-754b-ud-iq2-m". Configure your OpenAI-compatible client with the gpuos base URL and API key. Check model-specific features and any other API operations your app needs before switching.

Which gpuos plan includes GLM-5.3 754B UD-IQ2_M (community GGUF)?

GLM-5.3 754B UD-IQ2_M (community GGUF) is part of the full catalog on the Pro plan, $29 per GPU per month.

Related models

• Qwen3 32BChat · ≈ 22 GB · Apache 2.0
• GLM-4 32B 0414Chat · ≈ 20.5 GB · MIT
• Qwen3 30B A3BFast MoE · ≈ 20 GB · Apache 2.0
• gpt-oss 20BAgentic coding · ≈ 14 GB · Apache 2.0