Hands-On Tutorial

Open WebUI in Production: Self-Hosted LLM UI on Kubernetes

The interface layer of a sovereign AI stack, deployed properly — Helm, GPU scheduling, OIDC single sign-on, network policy, and the state changes that let it scale past one replica.

UI for every backend — Ollama, vLLM, any OpenAI-compatible API

Prompts leaving your perimeter when self-hosted

SSO

OIDC / LDAP auth with per-user RBAC built in

RAG

Document Q&A without standing up extra services

Why This Stack

The Missing Front Door of Self-Hosted AI

Most teams get a model serving on their own hardware and then discover the adoption problem: engineers will curl an endpoint, but the analysts, lawyers, and operators who'd benefit most won't. Open WebUI closes that gap — a polished, ChatGPT-style interface with document upload, chat history, and a prompt library, running entirely inside your perimeter. It's the piece that turns self-hosted inference from an engineering capability into something the whole organization actually uses.

This tutorial deploys it the way we deploy it for clients: on Kubernetes, behind SSO, with signup disabled, network policy applied, and state externalized so it scales. If you're still deciding whether self-hosted AI is the right call at all, start with the Sovereign AI guide — this page assumes that decision is made and gets you to a hardened deployment.

Architecture

Four Layers, One Interface

Open WebUI is deliberately just the front. Everything below it is swappable — which is exactly what you want from an interface layer.

Open WebUI

Interface

The chat interface your team actually uses: multi-model chat, document upload with built-in RAG, prompt library, user workspaces, and admin controls. Runs as a stateless-enough web app in front of whatever serves the models.

Ollama or vLLM

Serving

Open WebUI speaks to Ollama natively and to anything OpenAI-compatible — vLLM, Triton with the OpenAI frontend, or a hosted endpoint behind your governance boundary. Start with Ollama for simplicity; graduate to vLLM when throughput matters.

GPU-scheduled Kubernetes

Compute

The NVIDIA GPU Operator handles drivers and device plugins; the model server pod requests a GPU like any other resource. The same scheduling, autoscaling, and self-healing your applications already get.

Postgres + persistent volumes

State

Chats, users, uploaded documents, and RAG indexes are state. SQLite on a PVC works for a pilot; production means an external Postgres and object storage so replicas can scale and backups are routine.

The compute and orchestration layers are covered in depth in the AI Scaling guide — this page stays focused on getting the interface layer production-ready.

Deployment

Six Steps to a Hardened Deployment

From empty namespace to a team-ready, SSO-gated interface. Each step is small; the order matters.

Prerequisites

A Kubernetes cluster with at least one GPU node (NVIDIA GPU Operator installed), an ingress controller, cert-manager for TLS, and a default StorageClass. Any conformant distribution works — RKE2, EKS, DOKS, OpenShift.

kubectl get nodes -o wide
kubectl get pods -n gpu-operator   # operator healthy
kubectl get storageclass           # default class present

Deploy the model server

The official Open WebUI Helm chart can bundle Ollama as a dependency, which is the fastest path: one release, wired together out of the box. The Ollama pod requests a GPU and persists pulled models on a volume so node restarts don't re-download 20GB of weights.

helm repo add open-webui https://helm.openwebui.com/
helm repo update

Install Open WebUI

A minimal production values file: bundled Ollama with GPU and model persistence, Open WebUI persistence for uploads and chat history, and signup disabled from the first boot — the interface is inside your perimeter, but it still shouldn't be open enrollment.

# values.yaml
ollama:
  enabled: true
  ollama:
    gpu:
      enabled: true
      number: 1
    models:
      pull:
        - qwen2.5:14b
  persistentVolume:
    enabled: true
    size: 100Gi

persistence:
  enabled: true
  size: 20Gi

extraEnvVars:
  - name: ENABLE_SIGNUP
    value: "false"
  - name: DEFAULT_USER_ROLE
    value: "pending"

# helm install open-webui open-webui/open-webui \
#   --namespace open-webui --create-namespace -f values.yaml

Ingress and TLS

Expose the UI through your ingress controller with a cert-manager-issued certificate. Internal-only is a legitimate choice — many teams publish it solely on the corporate network or behind a VPN and never give it a public DNS record.

ingress:
  enabled: true
  class: nginx
  host: chat.internal.example.com
  tls: true
  annotations:
    cert-manager.io/cluster-issuer: internal-ca

Wire up SSO

Open WebUI supports OIDC, so it plugs into Entra ID, Okta, Keycloak, or any standards-compliant IdP. Combined with DEFAULT_USER_ROLE=pending, new logins land in a queue an admin approves — access is a decision, not a default.

extraEnvVars:
  - name: ENABLE_OAUTH_SIGNUP
    value: "true"
  - name: OAUTH_CLIENT_ID
    valueFrom:
      secretKeyRef:
        name: openwebui-oidc
        key: client-id
  - name: OAUTH_CLIENT_SECRET
    valueFrom:
      secretKeyRef:
        name: openwebui-oidc
        key: client-secret
  - name: OPENID_PROVIDER_URL
    value: "https://idp.example.com/.well-known/openid-configuration"

Scale past one replica

The default SQLite database pins you to a single pod. For a team-wide deployment, move state to Postgres and add Redis for websocket coordination — then replicas scale like any stateless service while Ollama or vLLM scales independently on the GPU side.

extraEnvVars:
  - name: DATABASE_URL
    valueFrom:
      secretKeyRef:
        name: openwebui-db
        key: url          # postgresql://...
  - name: WEBSOCKET_MANAGER
    value: "redis"
  - name: WEBSOCKET_REDIS_URL
    value: "redis://redis.open-webui.svc:6379/0"

Hardening

Production Posture: What "Hardened" Actually Means Here

A self-hosted LLM UI concentrates prompts, documents, and conversations in one place. That's its value — and its blast radius.

Authentication is not optional

Disable open signup before the first user ever logs in, front the UI with OIDC SSO, and leave DEFAULT_USER_ROLE at pending so every account is explicitly approved. An LLM interface accumulates sensitive prompts fast — treat it like an internal system of record, not a demo.

Network policy around the namespace

Open WebUI needs to reach its model backends, its database, and your IdP — nothing else. A default-deny NetworkPolicy with those three egress exceptions turns 'the AI UI got popped' from an incident into a non-event.

Uploaded documents are data at rest

RAG uploads land in Open WebUI's storage — which makes that PVC or bucket a data store subject to the same classification, encryption, and backup rules as any other. If HIPAA or ITAR data can reach the upload button, the volume inherits those obligations.

Pin images, plan upgrades

Open WebUI ships frequently. Pin a tested image tag rather than tracking latest, read release notes for auth-related changes, and rehearse the upgrade on a staging release. The project moves fast; your change management shouldn't be surprised by it.

Interface security is one slice of the boundary question — which data classes may reach which models is a governance decision, covered in the Sovereign AI guide. Quality is the other half: what the models answer is a promotion-gate problem, covered in LLM Evaluation in Production.

Proof

We Run This Stack — For Ourselves and for Clients

This isn't a lab writeup. The Open WebUI + Ollama/vLLM pattern on GPU-scheduled Kubernetes is the interface layer of the stack our US-based platform engineers operate in production, and the one we deployed in the Six-Week AI Rapid Strike engagement — idle GPUs to a production AI service, interface included, in six weeks.

If your program needs the layers below this one — hardware selection, GPU platform buildout, serving architecture — that's our self-hosted LLM practice and AI consulting work. The tutorial above gets a capable team to production; the engagements exist for teams that want it faster, with the sharp edges pre-filed.

FAQ

Open WebUI Questions, Answered

Open WebUI is a self-hosted web interface for large language models — a ChatGPT-style chat experience you run on your own infrastructure. It started as Ollama WebUI and grew into one of the most widely deployed self-hosted AI interfaces, with multi-model chat, built-in RAG document Q&A, a prompt library, user management with RBAC, and OIDC/LDAP single sign-on. It connects to Ollama natively and to any OpenAI-compatible endpoint, including vLLM and Triton.

No. Ollama is the most common backend because the pairing is zero-config, but Open WebUI talks to any OpenAI-compatible API. In production that's frequently vLLM for high-throughput serving, with Ollama kept for fast model experimentation. You can register several backends at once and let users switch models per conversation — one interface over your whole serving layer.

Use the official Helm chart (helm.openwebui.com). The chart can bundle Ollama as a dependency with GPU resources and a persistent volume for model weights, plus persistence for Open WebUI's own state. Add an ingress with TLS via cert-manager, disable open signup, and connect your OIDC provider. A single GPU node runs a capable open-weight model for a team pilot; the serving layer scales out from there.

Four controls cover most of it: ENABLE_SIGNUP=false with DEFAULT_USER_ROLE=pending so every account is admin-approved; OIDC SSO against your identity provider so access follows your joiner/leaver process; a default-deny NetworkPolicy so the UI reaches only its backends, database, and IdP; and TLS at the ingress. Beyond that, treat uploaded documents as data at rest — the RAG store inherits whatever classification the documents carry.

Yes, once you move it off the default SQLite database. Point DATABASE_URL at Postgres and enable the Redis websocket manager, and Open WebUI replicas scale horizontally behind your ingress like any web application. The model serving layer scales separately — replicas and batching on the vLLM/Ollama side — which is the right decoupling: UI capacity and GPU capacity are different problems.

Control over data flow. With Open WebUI in front of self-hosted models, prompts, uploaded documents, and chat history never leave your perimeter — which is the point for teams handling contracts, patient data, source code, or anything under residency obligations. It also decouples the interface from any one provider: the same UI serves open-weight models on your GPUs today and a governed external endpoint tomorrow. Note the license: Open WebUI is source-available under a permissive license with a branding clause for larger deployments — review it (and the enterprise option) before a company-wide rollout.

Sovereign AI Guide

The pillar this tutorial implements — running production AI inside your perimeter, from hardware to governance.

AI Scaling Guide

Why pilots stall between demo and production, and the six-layer stack this deployment slots into.

LLM Evaluation in Production

Once the UI is live, quality needs a gate — evals, judges, and promotion discipline.

Self-Hosted LLM Solutions

The full consulting practice behind this stack — model selection, serving, RAG, and air-gapped deployment.

GPU Kubernetes

GPU Operator, scheduling, time-slicing, and utilization — the compute layer under the tutorial.

Case Study: Six-Week AI Rapid Strike

Idle GPUs to production inference in six weeks — this stack, deployed for real.

The Interface Layer Is Where Adoption Happens

There's a pattern in stalled self-hosted AI programs: the serving layer works, the benchmarks look good, and usage flatlines at a handful of engineers with API keys. The gap is almost always the interface. People adopt tools that feel finished — chat history that persists, documents they can drop into a conversation, a model picker that doesn't require reading a wiki. Open WebUI's rise to one of the most widely deployed self-hosted AI projects comes down to exactly that: it made private inference feel like the consumer products people already trust, without the data leaving the building.

The production concerns are ordinary web-application concerns, which is good news. Authentication federates to the identity provider you already run. State externalizes to Postgres the way any scaling web app's does. Network policy, TLS, image pinning, and backup discipline are the same controls your platform team applies everywhere else. That ordinariness is the argument for putting the interface on Kubernetes next to the serving layer rather than on a VM someone snowflakes: every hardening pattern in this tutorial is one your cluster already enforces for other workloads.

The strategic value shows up later, when the model landscape shifts under you — and it will. Because Open WebUI fronts any OpenAI-compatible backend, swapping Ollama for vLLM, adding a second model, or routing a workload class to a governed external endpoint is configuration, not migration. Your users keep their history, their prompts, and their habits while the serving layer evolves behind them. That decoupling — interface stable, backends fluid — is what separates an AI platform from an AI experiment, and it's the reason the front door deserves the same engineering attention as the GPUs behind it.

Ready to make AI operational?

Whether you're planning GPU infrastructure, stabilizing Kubernetes, or moving AI workloads into production — we'll assess where you are and what it takes to get there.

Book a readiness assessment Call Us Directly

US-based team · All US citizens · Continental United States only