Conclusion & What’s Next

Congratulations! 🥠

You finished the Monday story: an in-cluster coding agent on ARO, no public LLM. In a half day you published Qwen3.8 as a service, shipped Fortune Cookie from a prompt, and looked at the GPU and token bill.

What You Built

As a Platform Engineer

Component Technology

GPU infrastructure

ARO HCP node pool (fixed 1 node), NVIDIA A100 80 GB

LLM as a Service

OpenShift AI 3.5 dashboard + Models-as-a-Service + LLMInferenceService + Red Hat vLLM + Qwen3.8-27B-INT4

AI-native governance

AGENTS.md — machine-readable platform conventions enforced by OpenCode

Developer enablement

Go app template + OpenShift Dev Spaces pre-configured with OpenCode

GitOps delivery backbone

OpenShift GitOps (Argo CD) + OpenShift Pipelines (Tekton)

LLM observability

NVIDIA DCGM dashboard + OpenShift AI model metrics (Prometheus)

As a Developer

Milestone Details

Personal Git repo

Created in seconds on ephemeral Git — no GitHub account, no SSH key setup

Fortune Cookie app

Generated live by OpenCode + Qwen3.8 — main.go, go.mod, Dockerfile, all from a single prompt

Compile gate

CGO_ENABLED=0 go build -buildvcs=false verified code before pushing — zero broken builds

Container image

Built by OpenShift Pipelines (Tekton): git-clone + buildah — no local Docker required

GitOps deployment

Developer-owned Argo CD instance synced deploy/base/ → app live at a public HTTPS route

Compliance audit

OpenCode reviewed and fixed resource limits, probes, security context, TLS — all per AGENTS.md

The Key Insights

1. ARO HCP makes GPU infrastructure a Platform Engineer concern, not a developer concern. make gpu adds one A100 node pool in the subscription. Developers never pick a VM size or a taint.

2. OpenShift AI makes LLM serving a catalog item, not a project. oc apply -k deploying an LLMInferenceService plus MaaS CRs gives every team a governed OpenAI-compatible API — keys, quotas, and the same interface they already know.

3. AGENTS.md shifts governance from documentation to code. Platform conventions encoded in a Markdown file are read and enforced by the LLM at generation time. The developer gets a compliant app without opening a checklist.

4. The same LLM powers the developer inner loop. OpenCode uses Qwen3.8 to write the Fortune Cookie app. Qwen3.8 is a platform capability.

5. Observability closes the loop. GPU utilization, token throughput, and TTFT are visible in the same console as all other cluster metrics. Token quotas on the MaaS gateway make consumption attributable.

What We Did Not Cover (But You Can Explore)

  • LLM scale-out with llm-d: This workshop pins one vLLM replica. OpenShift AI’s path is Distributed Inference with llm-d on LLMInferenceService: vLLM engines, an inference gateway and scheduler, KV-cache-aware routing, and LeaderWorkerSet when the model spans GPUs or nodes. Replica count is still moved by KEDA (it owns the HPA). The scaling signal is the workload variant autoscaler (WVA) — KV cache (vllm:kv_cache_usage_perc) and queue (vllm:num_requests_waiting), not CPU or memory. WVA is Developer Preview in OpenShift AI 3.5.

  • Multi-environment GitOps: Add deploy/overlays/stage/ and deploy/overlays/prod/ with Kustomize overlays; use separate Argo CD Application objects per environment

  • Webhook-triggered pipelines: Add an EventListener + TriggerBinding to Tekton so git push triggers builds automatically

  • Application autoscaling: Add an HPA to the Fortune Cookie Deployment to scale based on HTTP requests

  • Model fine-tuning: OpenShift AI Data Science Projects support fine-tuning workflows on proprietary data using distributed training

  • ARO hosted control planes: This workshop already runs on ARO HCP. Microsoft hosts the control plane. Worker nodes are node pools in your subscription.

  • OpenShift Service Mesh: Do not install OSSM 3 by hand on OpenShift 4.22 — this cluster uses the Gateway API (openshift-default). Istio remains optional for app-to-app mTLS later.

Get Started with ARO

Talk to Us

If you have questions or want to explore ARO for your organization, our team is here to help. Use the blue hat icon in the Red Hat Hybrid Cloud Console to start a conversation.

Thank you for attending the AI-Powered Developer Platform on ARO workshop!

May your deployments be swift, your models be accurate, and your fortunes be wise. 🥠