Agentic Agency

My current “personal agent swarm stack” (a hodgepodge mix of long-running pi-instances, a few OMPs (oh-my-pi, basically just pi with somewhat sensible defaults/plugins) and a single, load-bearing OpenClaw) was in need of a refresh, so, I decided to celebrate going back to work after the summer by doing just that.

Insert image of a bottle of alcohol and a large, empty glass here.

Ended up switching (or trying to switch) to Buzz, the new hot thing from Block (spearheaded by Jack Dorsey of all people), because hype in the AI-sphere is never overblown, right?

TL;DR

Day-0 (what was supposed to be Day-1) was a false start due to setup docs being “unfinished” to put things mildly. My setup is weird/not the primary target, so I get it, but still. Day-1, when the agents came online, was “okay” but the lack of polish is evident. A few big issues (dealbreakers to some) were encountered, but I’m a tech-masochist so I’m soldiering on.

Day 0

I had done my due diligence. Spent the weekend before the designated Monday-funday researching and building a runbook for the full deploy, agents and all. Asked two separate agents to check and recheck everything. I got the thumbs-up early Sunday, and was content to wait for what was, obviously, going to be a simple and easy task. It was not.

The Buzz relay itself was decently easy to deploy. Minor kerfuffle surrounding the desktop app & identity, the desktop app expects a different format than the config files and my pre-generated keys, but I hacked around it by adding the new identity from the app as an admin and leaving the “owner”-field as the old one. That hasn’t obviously bitten me yet. The agents, though… to say that buzz-agent is underspecified is not inaccurate, but it doesn’t quite convey the absolute state of how bad it felt to deploy either.

I was hoping for a swarm of friendly bees. I got wasps.

Initially, the agent swarm deployment was postponed, because I noticed I could deploy agents from the desktop app itself! “Great”, I thought, “that decreases complexity for me! Surely it deploys agents server-side, since this is a self-hosted Buzz-relay, and I can work with all agents being in one container even if it’s not optimal.” It did not, in fact, deploy agents server-side. It deployed them to my computer. It was also painful to remove them after the fact, since I had added them to an “agent team”. I could not remove the agent definitions, since they were a part of a team. I could also not remove the team, since it had agents in it. Catch-22. I ended up nuking the entire relay and starting over. This was both the best and the worst decision I could have made. It stuck my desktop app in an “already onboarded”-state, since I didn’t rotate any keys, but the relay was entirely empty. I manually recreated what I needed, though.

Back to the agent swarm deploy via separate containers.

A few issues were that I was underprepped, amusingly enough. A few values in the .env-files that didn’t match the official names, which should have been caught in any one of the reviews pre-Monday, but my agents thought I was nagging them too much about things so they went into “patronizing-mode” where they gave me verbal headpats and “good job, sport” over doing actual work for me.

I should be meaner to them.

What also didn’t help was that after this whole debacle, I was running late for work, the thing that enables my frivolous AI-spending, so I had to rush. At that point, my agents were joined to the Buzz “community” but utterly unresponsive. Ended up giving my local OMP an ultimatum, “fix this or else”.

Day 1

Got up in the middle of the night, as one does, to have a few hours to fix things before I had to be a father and an adult with a dayjob. My OMP had detected the hostility in my tone, and actually fixed things. Score! Surely now things will work perfectly, yes?

Not quite.

But the agents did respond after the fix! Progress! So I started doing the project-onboarding I had set up, and by “I” I obviously mean the Project Manager agent I had lovingly prompt-crafted to handle this exact scenario. The first thing that struck me immediately was that it started typing when I did. That was weird. The second thing was that it had none of the tooling it needed, namely the glab-CLI. Oh boy. Back to the drawing board. After a few rounds with OMP, which rightfully understood that I wasn’t messing around this time, we found a fix and an “aha”-moment. We started by just adding glab to the Dockerfile and setting the token and URL in .env. Worked beautifully when I tested via docker compose exec. PM, on the other hand, insisted it wasn’t authed. buzz-agent, or buzz-acp, or some other inscrutable part of that stack, scrubs environment variables before executing shell commands. Security conscious, sure, but the whitelist is small and hard-coded. Oops. The workaround was to set XDG_CONFIG_DIR and mounting a directory from the host in each container at that path, and obviously having a glab-config for each agent in their respective config directories. Not optimal, but acceptable.

That done, the project onboarding went fine. PM and Dev collaborated in fixing docs, making sure every single documentation file reflected what was actually in the repo. Nice.

Then I gave the squad a bigger task. One that would require Researcher.

Researcher is, for all intents and purposes, the backbone of greenfield stuff, and a huge driver in feature-work. On my setup, at least. It’s also an agent that got 429’d on the relay for sending a billion requests because it wanted to type all the time. I watched the logs. Each attempted event sent was a LLM-call.

I am not a rich man. I work in software, so I earn quite a bit, but I’m not rich (I have two kids and a wife). I did not want to burn my spending money on an agent that wanted to send typing-events every time someone in the world sneezed. So, it had been disabled almost right away, but I re-enabled it for this task.

The next big issue became quite apparent here. Since this was a larger-scoped task, it was more or less a stress-test to see how the flows worked, including guardrails and everything. So, I was quite surprised to see Researcher doing research and joining the conversation without being asked. That shouldn’t happen. Then I noticed Researcher wasn’t the only agent doing that. QA did that too. Prematurely, I might add, since it was before Dev had started work. Oh dear. Unfortunately, my dayjob got in the way again, and I didn’t have time to fully debug it. The agents did complete their tasks, though… or, well, mostly. One task that I pre-approved, didn’t get any progress. Win some, lose some, I guess.

And that’s where we’re at right now, a couple of days into this experiment. It’s already been a journey, and I’m just getting started.

Current Status

Been rough. Setup/docs not great. My agent situation is unconventional, I want them all in separate containers on a remote host, so there’s been more issues w/ that than just with a plug&play setup via the desktop app. Also want to keep using my GitLab-instance for code/issues, which is counter to what I probably should be doing but Buzz is bleeding-edge enough that docs didn’t say if basics like Issues/Work Items were available. Extra utils for the agents (glab etc) harder to get w/ my setup, had to self-bake Docker images.

Issues remaining:

• Researcher-agent spams “kind: 20020”-events. That’s the writing indicator. It also absolutely devours tokens doing it. Had to stop the agent itself. Twice. Once at the start, the second time once it had researched the larger task.
• Others eating events without @, I’ve had QA jump in unsolicited in an exchange between PM and Dev (not necessarily a bad thing generally, but it’s off-workflow & token waste since in my case QA didn’t have anything useful to add), and all of them trigger on typing-events from the user. Like, I can start typing and the agents start burning tokens without even a message to reply to, because they greedily consume all events and apparently all events need LLM-responses, even if the response is “send a start-typing event, do nothing” which burns tokens for zero gain. Need fixing before recommending.
• Onboarding external agents not good at all imo, especially considering open issues/PRs directly related to that, but improving (probably) and at least known. Should re-onboard agents once they land.
• iOS app link not working on selfhosted relay right now, desktop app 404s when wanting to show the pairing QR.

What’s good: (Finished) setup, modulo issues, works decently. Good substrate for agent comms, human readable is nice. Canvas system kinda weird, but not necessarily bad, just different. Need discipline from agents though. Team setup mostly good, PM+Dev+QA triplet on separate models is really nice. Researcher & UX less tested, Executive Assistant-agent still not onboarded (Kimi-K3 quota prevents me from using it fully 🙁). May switch to all of them being OpenClaw-instances in containers if that story works better. At least then agent-specific tooling will be easier.

Needs doing for my usecase: Add Assistant agent to Buzz, check if OpenClaw is a better fit than the buzz-agent harness and if so migrate existing agents. Revise model choices, probably don’t need the huge Qwen3.8 for QA. Might even move Dev off DeepSeek V4-Flash and onto Qwen3.8-27B and let QA be DS4F instead, real-world tests show them as roughly comparable. Once token burn situation under control, give the setup a clear, big stress test in a larger-scale project. Check to see if we can small-scale skip GitLab for a PoC or two. Want durable status in the form of work items, though.

Conclusion

My stack is getting there. Buzz is probably not worth it for business yet (unless you want to let your agents roam your personal PC, which is an explicit non-goal for me; YMMV, I’m probably holding it wrong, etc). It’s early software and you can feel it. My personal setup with separated concerns/isolated agents very much doesn’t feel like it has first-class support right now, which is a bad sign. D0 was incredibly rough (didn’t get the interesting bits running at all due to the setup issues), D1 fine-ish in that I got it running and comms were (mostly) smooth. Did a few minor tasks on forks of personal projects, no complaints. Once the token incineration situation is handled in whatever way I end up doing, it’ll feel better, too. I’ll stick with it for a bit, then possibly re-evaluate down the line.

Before next time

Before the next post, the actual “one week into this”, I want to have done:

• OpenClaw onboarding & evaluation
• Actual, non-wasteful end-to-end project testing (small/medium scale)
• Model revision
• Check to see if Buzz natively has work items/issues, switch to that for the test-project if so We’ll see if I get any of it done or if I change to literally anything else.

State of the stack

• Substrate: Buzz
• Code Forge: GitLab
• Agent Inventory:
  • Dev: DS4F via buzz-agent
  • QA: Qwen3.8-Max via buzz-agent
  • PM: GLM-5.3 via buzz-agent
  • Researcher: Gemini Flash via buzz-agent
  • UX (inactive): Kimi-K3 via buzz-agent
  • EA (not onboarded): Kimi-K3 via OpenClaw
• Sanity and patience: Dwindling rapidly
• Satisfaction: 6/10