This post is about how I test GateBolt Gateway. It covers the method, which stays the same from run to run. Each run, with its own LLM and set of tests, gets its own post.
What the gateway does
The gateway sits between Claude Code and Anthropic. It gives Claude an extra tool for declaring a plan, and asks Claude to use it first: the files, commands, tools and websites it will use. The gateway has two levels:
- Record: everything is logged against the plan, nothing is blocked.
- Enforce: anything outside the plan is refused.
What I want to find out
- Does it work? Is everything logged, and is anything outside the plan refused?
- What does it cost? Do tasks take longer, use more tokens, or fail more often?
How
I run the same tasks three ways: Direct (no gateway, the baseline), Record and Enforce. Each task runs in Claude Code, in a fresh Docker container, in my lab.
Claude Code runs on my own subscription, signed in with a token from claude setup-token. So any cost I report is an estimate, not a bill.
Keeping it fair
Any difference should come from the gateway, nothing else:
- Same setup. Same LLM, Claude Code version, containers and tasks.
- Nothing custom. Claude Code and Harbor run with their standard settings. If Claude Code behaves differently behind the gateway, that’s part of what the gateway costs, so I measure it rather than hide it.
- Mixed order. The three ways take turns, so a slow network hits them equally.
- Usage limits aren’t failures. The run waits and tries again.