The Netyeti's Journal

The AI Homelab Story: From Manageable Mess to Agentic Framework

4 min read

The AI Homelab Story: A Cohesive Narrative

The Instructor's Perspective

We didn’t just wake up with a working AI stack. It was a journey through “dependency hell,” broken drivers, and a lot of “stupid questions.” This is the story of how we built a Local-First AI Command Center on Intel hardware, and why every layer matters.

Phase 1: The “Manageable Mess” (The Early Days)

In the beginning, we tried to run LLMs the “old way”—manual builds of llama.cpp, battling with Python virtual environments, and hitting OOM (Out of Memory) kills on a daily basis. We had tools scattered everywhere:

  • Ollama running as a system service.
  • LocalAI in a separate VM.
  • Aider for coding, but struggling to connect to local backends.
  • Mistral/Llama models taking 20 minutes to load.

We were “blind” because standard tools like intel_gpu_top couldn’t see the new Xe Driver on the Meteor Lake i9 Ultra. We knew the hardware was there, but we couldn’t “reach out and touch it.”

Phase 2: The SYCL Breakthrough (The Tactical Win)

The turning point was April 5, 2026: The Containerized SYCL Victory. We stopped trying to “fix” the host OS and moved everything into Docker. By using the intelanalytics (and later the community ava-agentone) images, we packaged the oneAPI and Level Zero drivers into a stable, reproducible environment.

The Lesson Learned: Don’t fight the OS; containerize the complexity. By passing the DRM nodes (/dev/dri/card1 and renderD128) directly into the container, we unlocked 100% GPU offload on the Intel Arc iGPU. Suddenly, 14B models were snappy, and the “Error 500” retries vanished.

Phase 3: The PACE Plan (The Strategic Framework)

With a working engine, we needed a strategy. We adopted the PACE Plan to ensure we always had “Signal Discipline”:

  • Primary (P): Local Ollama. Our “boots on the ground.” It handles 90% of the work—coding, summarization, and private data—without ever leaving the LAN.
  • Alternate (A): Gemini Cloud. The “High Command.” Used for complex synthesis, cross-vault strategy, and when we need the “big brain” for massive context windows.
  • Contingency (C): Fallback nodes like Sterling or manual builds.
  • Emergency (E): Physical books and human expertise.

Phase 4: OpenClaw (The Command & Control Layer)

The final piece of the puzzle was OpenClaw. We moved from “chatting” to “operating.” OpenClaw became our C2 (Command and Control) layer, orchestrating the connection between:

  • The Brains: Switching seamlessly between local Ollama and Cloud Gemini.
  • The Tools: Giving the AI “hands” via Gog (Google Places), Search, and a controlled Shell.
  • The Infrastructure: Accessing the entire stack securely over Tailscale.

Phase 5: The “Force Multiplier” (Where we are now)

Today, the stack is a seamless, layered connection:

  1. Local Priority: Heavy tasks (coding with Aider, local RAG) always hit the Ollama (Primary) layer first. It’s faster, cheaper, and private.
  2. Single Source of Truth: On April 8, 2026, we decommissioned the legacy llama-server (Port 8081). All tools—OpenClaw, Aider, and the shell—now point to Port 11434 (Docker). We’ve replaced the “switching” logic with a multi-model hub.
  3. Agentic Framework: Through OpenClaw, our AI can look up documentation in this Obsidian vault, search the web for the latest driver updates, and even draft the very guide you are reading.
  4. The Pi Connection: Using low-power nodes (like the Raspberry Pi) as lightweight gateways or monitoring stations for the heavier x86 compute nodes.

The Four Pillars of Local-First AI

Why do we go through the trouble of building this local fortress? Because it rests on four non-negotiable pillars:

  1. Resilience (Offline Capable): If the internet drops, your lab shouldn’t turn into a paperweight. Having a Local Primary (P) means you can still code, summarize, and learn even when you’re “off the grid.”
  2. Accessibility (Reduced TCO): High-end cloud AI is expensive. By reducing the Total Cost of Ownership, we “raise the water level” for everyone. If it runs on an iGPU, a student can build it.
  3. Data Sovereignty (Local by Default): Keep your “intel” in your own fortress. By keeping data local, you don’t have to worry about a cloud provider “learning” from your private notes or sensitive code.
  4. Sustainability (Green Computing): Massive data centers have massive carbon footprints. Running a 14B model locally on efficient Intel silicon is a responsible way to use AI as a “force multiplier” without the “anti-green” overhead.

Lessons Learned: The “NetYeti” Golden Rules

  1. Containerize Everything: If it has complex drivers (AI/GPU), put it in Docker.
  2. Align your Bind Mounts: Half of our “broken” services were just Docker trying to mount non-existent folders.
  3. DRM Node Precision: On modern Intel chips, passing the right /dev/dri nodes is the difference between CPU crawling and GPU flying.
  4. Local First, Cloud Second: Keep your data in your “fortress” (The Homelab). Use the cloud as a “Force Multiplier,” not a crutch.

Related: AI Command Center, The Stack, index

Graph View

Backlinks