Running an AI Agent in a Home Lab: Security First

I’ve been running an AI agent in my home lab for a bit, and it’s become one of the most useful tools in my workflow. But before I get into what it does, I want to talk about how I set it up, because I think the security approach matters just as much as the capabilities.

The agent is Hermes Agent, an open-source AI assistant by Nous Research that can use tools, run code, manage files, and interact with external services. It’s the kind of thing that sounds dangerous the moment you describe it — an AI with access to your shell, your GitHub account, your filesystem — and honestly, it is dangerous if you’re not deliberate about it. This post is about being deliberate.

The Problem

When I first started experimenting with Hermes, it ran as a regular process on my Mac Studio, the machine I use for a lot of things in my home lab. It had my user’s permissions, access to my filesystem, my SSH keys, my environment variables, and anything else my user account could touch. That’s a lot of trust to give to anything, let alone an AI agent that reads web pages, executes tool output, and processes untrusted input from external sources.

The concern isn’t theoretical. While I was using local models for the initial experimentation, AI agents still process text from files, web pages, API responses, and chat messages, all of which could potentially contain adversarial instructions (so-called prompt injection). If an agent can be tricked into running a command, and it’s running as your user on your machine, the blast radius is your entire digital life. I didn’t want to find out what that looks like.

I had a specific worry: I’d hate for the agent to be used to perform actions as me on my Mac Studio. The solution was to move it into a container.

Containerization (and Then Some)

I run OrbStack on my Mac Studio for container workloads and local Kubernetes, so the infrastructure was already in place. The plan was straightforward: build a container image with Hermes installed, mount only what it needs, and run it as a non-root user inside the container.

But a bare Docker container with a restart policy only gets you so far. Since I already had a Kubernetes cluster running (my blog is deployed there), I decided to deploy Hermes as a proper K8s Deployment. That gave me liveness probes, resource limits, restart semantics, and declarative configuration, all for free. The manifest lives in a version-controlled, local repo alongside (and it contains the Hermes Dockerfile), which means the entire setup is reproducible.

The key security decisions:

• Non-root user: the container runs as a dedicated user, not root. Even inside the container, privilege escalation requires explicit effort.
• No Docker socket mounted: this is a common shortcut for container management, but it’s effectively root access to the host. I deliberately left it out.
• Bounded workspace: the agent gets a read-write workspace directory, but it’s on a separate volume (/Volumes/HermesData), not my home directory. If it goes rogue and fills a disk, it fills that disk, not my root filesystem. It also means I choose what Hermes gets access to intentionally. It slows down certain things (like when I want to try another plugin or feature), but it also means I get to take time to think through things carefully.
• Secrets in .env with 600 permissions: API keys and database credentials live in the container’s environment, readable only by the agent’s user. Hermes also supports Bitwarden and 1Password integrations for a more robust secrets pipeline, though I haven’t set that up yet.
• Dedicated SSH key: I generated a separate SSH key for the agent’s GitHub operations, added to my GitHub account. No access to my normal personal SSH keys.
• Scoped GitHub token: the gh CLI is authenticated with a fine-grained PAT limited to specific repositories, dedicated to Hermes.
• Resource limits: memory and CPU caps prevent a runaway process from consuming the entire machine.

The result is an agent that can still do meaningful work — clone repos, write code, file issues, open PRs — but whose worst-case scenario is a compromised Linux container with limited filesystem access, no macOS keychain, no host user session, no Docker socket, and no access to my photos or messages. That’s a blast radius I can live with.

Models: Cloud and Local

One of the things I love about this setup is the model flexibility. Hermes supports any OpenAI-compatible provider, and I’ve wired it up to take advantage of both cloud and local models.

The main model is currently glm-5.2, accessed through Ollama‘s cloud routing (I’m on the Pro plan). This handles the primary conversation: reasoning, tool use, decision-making. It’s a frontier-class model that handles complex tasks well.

But what’s really nice is the delegation feature. Hermes can spawn subagents for specific tasks, and I’ve configured it to delegate to local models running on the Mac Studio’s (64GB of unified memory). The primary delegation model is gemma4:31b-mlx, which runs locally via MLX, which is Apple’s machine learning framework.

The practical effect: routine development work, codebase study, test file generation, and mechanical refactoring get delegated to local models that cost nothing per token and keep sensitive code on my machine. The cloud model handles the reasoning-heavy work where its capabilities actually matter. This keeps token usage reasonable while giving me the best of both worlds.

Discord as the Interface

I chat with the agent through Discord, in a private channel on a server I run. Hermes has a gateway system with connectors for a wide variety of platforms — Discord, Telegram, WhatsApp, Slack, and more. Almost none of them require inbound Internet connectivity, which is a nice security property for a home lab setup.

From my perspective, it’s just a conversation. I type a message in Discord, the agent processes it, calls tools as needed, and responds. It can also proactively send messages; for example, when a cron job finishes or when it needs my input on something, I see a new message pop up.

What It Actually Does

So what does a contained AI agent with GitHub access and local models actually do? Here’s a concrete example: I have it run a nightly job that scans my open-source repositories for health and hygiene. It checks for stale issues, outdated dependencies, failing CI, and missing tests. Then it takes action:

• It opens issues for improvements it identifies — for example, authify/authify#119, where it noticed code duplication in the codebase and filed a detailed issue about it.
• It reviews PRs from contributors — for example, gadget-bot/gadget#143, where it reviewed a contributor’s pull request and left constructive feedback.
• It creates its own PRs to fix issues it finds — for example, authify/authify#129, where it implemented a fix and opened a PR for my review.

All of this happens while I’m asleep. The development work is delegated to local models, so I’m not burning cloud tokens on routine code changes. When I wake up, there’s a summary waiting in Discord with links to anything that needs my attention.

The important part: it never pushes to main, and it never merges PRs without my explicit approval. I set that policy early on, and the agent respects it. Everything goes through pull requests and issues, and I review the code before anything gets merged. All of this activity reflects on me personally, so I’m cautious and deliberate about what the agent does on my behalf.

What I’ve Learned

A few months into this setup, a few things stand out:

The container boundary is the foundation. Everything else (the scoped tokens, the dedicated SSH keys, the resource limits) builds on top of it. Without containment, the rest is just a “gentleman’s agreement” with software that reads untrusted input. With it, even a worst-case scenario is bounded.

Local models are practical for real work. I was skeptical at first, but a 31B parameter model running on an M2 Max is genuinely useful for code generation, test writing, and refactoring. The machine wasn’t cheap, but it didn’t break the bank. It’s not going to replace the cloud model for complex reasoning, but it doesn’t need to. The delegation pattern means each task goes to the model that’s best suited for it.

The agent is most valuable when it’s proactive. The nightly OSS maintenance job is the best example. Historically, I’ve struggled to maintain my open-source projects as much as I should. Now, the agent helps: it files issues and opens PRs. I just review. It’s like having a very junior developer who works nights and never gets tired (though, like a junior developer, you do need to review their work).

What’s Next

I’m still hardening the setup. A few things on my radar:

• Network egress restrictions: right now, the container can reach anything on the Internet. I’m researching how to restrict outbound traffic to only the known endpoints it needs (Discord, LLM providers, GitHub). This would prevent a compromised agent from exfiltrating data to an arbitrary server.
• Secrets manager integration: moving from .env to Bitwarden or 1Password for credential management, so secrets don’t sit on disk at all.
• Multi-user isolation: my family might join the Discord server, and right now there’s no per-user memory isolation. The agent’s memory is shared across all users in a channel, which is a gap I’d like to close.

If you’re thinking about running an AI agent with real system access, I’d encourage you to think about containment from the start. It’s not paranoia; it’s good hygiene. The agent is still useful, still capable, and still does meaningful work. It just does it inside a boundary that you control.

If you want to learn more about Hermes Agent, check out the documentation. If you have questions about my setup, feel free to reach out! I’m always happy to talk about home labs, AI, and the intersection of the two.