Running a Local Coding Model: opencode + GLM on Apple Silicon

A complete, measured account of putting a 30B coding model on a dedicated Mac and
driving it from opencode over a network. Every number here was read off the
running system, not estimated. Where an earlier estimate was wrong, the correction
is kept and labelled — the wrong figures are as instructive as the right ones.

Posture: one machine, one resident model, loopback-only, verified by a script
rather than by belief.

---

1. Hardware and the real ceiling

Host	dedicated Mac, 48 GB unified memory, Apple Silicon
Usable for inference	~37.4 GiB, not 48 — macOS wires roughly 75% for the GPU
Server	Ollama 0.32.15, loopback only
Model	glm-4.7-flash:latest — 30B-A3B MoE, Q4_K_M
Client	opencode on a laptop, over an SSH tunnel

The 48-vs-37.4 gap is the first thing that catches people. Budget against 37.4 GiB
or you will be debugging an eviction loop.

Measured memory, read from the load log

load_tensors:  MTL0 model buffer size = 17958.41 MiB
llama_kv_cache: size = 5076.00 MiB ( 49152 cells, 47 layers, 2/2 seqs )
sched_reserve:  MTL0 compute buffer size = 213.21 MiB  (+ 56.01 CPU)

• Weights: 17,958 MiB (17.5 GiB)
• KV cache: 51.6 MiB per 1K tokens per slot — 5,076 MiB ÷ 49,152 cells ÷ 2
  sequences. An earlier note in our own docs said "~0.1 GB per 1K"; that was 2×
  pessimistic and led to under-provisioning context.
• Resident total: 23,304 MiB ≈ 22.8 GiB, which matches Ollama's reported
  24.38 GB (decimal).

Registry size ≠ resident size. Resident is weights + KV at the loaded context

• compute graph. Read size from /api/ps; never infer it from the download.

What memory permits, versus what is useful

With ~11.6 GiB free:

slots	max context/slot
1	~329K
2	~164K
4	~82K

Memory is almost never the binding constraint. Latency is — see §7.

---

2. Installation on macOS

The published install.sh is Linux only and will fail. Use the headless
tarball; nothing needs sudo and nothing lands outside $HOME.

# the tarball is FLAT — extract into a dedicated dir or it spills ~40 files
mkdir -p ~/.local/ollama-dist && tar -xzf ollama-darwin.tgz -C ~/.local/ollama-dist
ln -sf ~/.local/ollama-dist/ollama ~/.local/bin/ollama

The binary loads its dylibs as siblings, so the directory cannot be split up.

launchd, not a shell background job

<!-- ~/Library/LaunchAgents/com.llmmac.ollama.plist -->
<key>EnvironmentVariables</key>
<dict>
  <key>OLLAMA_HOST</key>           <string>127.0.0.1:11434</string>
  <key>OLLAMA_CONTEXT_LENGTH</key> <string>49152</string>
  <key>OLLAMA_KEEP_ALIVE</key>     <string>24h</string>
  <key>OLLAMA_NUM_PARALLEL</key>   <string>2</string>
</dict>
<key>KeepAlive</key><true/>
<key>RunAtLoad</key><true/>

Bootstrap into gui/$UID so it survives ssh logout and starts at login:

launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.llmmac.ollama.plist

Why each variable matters

variable	value	reason
OLLAMA_HOST	127.0.0.1:11434	Ollama has no authentication. Never 0.0.0.0.
OLLAMA_CONTEXT_LENGTH	49152	Clients cannot set num_ctx over /v1. This is the only place context can be pinned.
OLLAMA_KEEP_ALIVE	24h	opencode does not propagate keep_alive (opencode#2979), so the model would otherwise unload between turns and pay a 54–84 s cold load.
OLLAMA_NUM_PARALLEL	2	Two slots at 49,152 each. See §7 for why more is not better.

---

3. Network: an SSH tunnel, and two traps

opencode → 127.0.0.1:11435 (laptop) → ssh → 127.0.0.1:11434 (host) → Ollama

The local port is 11435, not 11434, deliberately. A local Ollama would own
11434, and a silently-failed forward would route the agent to the local model
while you believed you were testing the remote one. Different port, loud failure.

Trap 1 — the forward must not multiplex

Put the forward on its own host alias with its own connection. If the keeper
multiplexes onto a shared ControlMaster, its forward inherits ControlPersist
and dies with it — a tunnel that evaporates after ten idle minutes, presenting as
Cannot connect to API with a perfectly healthy server.

Trap 2 — ExitOnForwardFailure kills ordinary ssh

Once the keeper holds 11435, every ssh host that also requests the forward
fails with Address already in use — and with ExitOnForwardFailure yes the whole
session dies, breaking log tails and health scripts. Split it:

Host llmmac-tunnel          # tunnel only; MUST come first (first value wins)
  ControlMaster no
  ControlPath none
  LocalForward 11435 127.0.0.1:11434
  ExitOnForwardFailure yes

Host llmmac llmmac-tunnel   # everything else, shared
  HostName 10.x.x.x
  ControlMaster auto
  ControlPersist 10m

The launchd keeper targets llmmac-tunnel; humans and scripts use llmmac.
Verify with the only honest check:

ssh -G llmmac        | grep -c localforward   # want 0
ssh -G llmmac-tunnel | grep -c localforward   # want 1

Then prove KeepAlive works rather than assuming: kill the keeper's PID, wait
past ThrottleInterval, confirm a new PID owns the port.

---

4. The opencode provider

{
  "provider": {
    "llmmac": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://127.0.0.1:11435/v1" },
      "models": {
        "glm-4.7-flash:latest": {
          "tool_call": true,          // false or unset => tools never fire
          "reasoning": true,
          "limit": { "context": 46000, "output": 8192 }
        }
      }
    }
  },
  "compaction": { "auto": true, "prune": true, "reserved": 8000 }
}

tool_call: true must be declared per model. Without it the model narrates what
it would have done and never calls a tool — a failure that looks like
incompetence rather than configuration.

The sizing inequality

num_ctx on the server covers prompt plus generation, so:

(limit.context − compaction.reserved) + limit.output  ≤  server n_ctx_slot

Worked against a 49,152 slot:

context	reserved	output	trigger	worst case	verdict
48000	8000	8192	40,000	48,192	valid, 960 margin — too thin
46000	8000	8192	38,000	46,192	2,960 margin
48000	8000	4096	40,000	44,096	valid, but see §5

Exceed it and the request is accepted; the server truncates the oldest tokens
and the model behaves as though it simply forgot
(ollama#14259). No error, no
truncation field, HTTP 200.

---

5. The output budget is not a safety valve

Reflexively halving limit.output to bound runaway generations is wrong, and cost
us two failed attempts to generate one file.

GLM writes its reasoning trace into the output budget. A task producing a
~5,000-token file therefore needs room for trace and file:

attempt	limit.output	generated	result
1	4096	4,096 (capped)	nothing written
2	4096	4,096 (capped)	nothing written
3	8192	4,969	file written

Truncation and runaway present identically — the turn hits the cap, burns
~290 s, produces nothing. The distinguishing signal: a runaway hits the cap on
turns with nothing large to produce; truncation hits it on the one turn with a big
legitimate file. Same number, opposite fix.

The undersized cap also silently degraded compaction, whose summary is just
another large output:

output cap	compaction result
4096	44K → 22.6K (weak, summary truncated)
8192	36K → 9.3K (clean)

---

6. Verification: eleven layers, in order

Debugging by guessing which layer broke wastes afternoons. A doctor.sh that
checks in dependency order and names the failure does not:

1. ssh config resolves the host
2. passwordless login works
3. key file ownership
4. connection multiplexing alive, connect time sane
5. terminfo present on the remote
6. port 11435 bound locally
7. Ollama reachable through the tunnel
8. server refuses the LAN address (still loopback-only)
9. expected models installed
10. residency — exactly one heavy model, with its context
11. client limit.context ≤ server context, per model

Step 11 is the one that catches silent truncation before it costs you a session.

---

7. Results: what the thing actually does

Solo, verified uncontended, same prompts at each depth:

prompt depth	prefill	generation
1,360	1,183 tok/s	60.5 tok/s
2,305	1,017 tok/s	55.9 tok/s
2,933	926 tok/s	53.1 tok/s
~9,800	—	34.5 tok/s
~27,200	—	17.9 tok/s
~65,000	—	~8 tok/s

Generation halves for roughly every doubling of depth, because the KV cache is
re-read for every generated token. Context is not a one-time prefill cost; it is a
tax on every token you subsequently generate.

The second slot is not free

OLLAMA_NUM_PARALLEL=2 gives concurrency, not throughput:

configuration	throughput
one worker, ~11K depth	33.0 tok/s
two concurrent, aggregate	18.4 tok/s
six dispatched at once	2.4 tok/s

Aggregate throughput falls. Two decodes complete less total work per second
than running them sequentially — memory bandwidth is the bottleneck and the KV
reads do not share. Parallel agent dispatch on this hardware is net-negative.

The prefix cache buys TTFT, not decode

Four distinct ~16K prompts stayed cached simultaneously (n_slots = 1 limits
concurrency, not retention), each returning in 0.1–0.2 s after the others had
been served. But at identical depth, cache hit rates of 18% and 90% produced
20.7 and 21.2 tok/s — indistinguishable. The cache skips recomputing prefill;
the KV is still re-read per generated token.

---

8. Failure modes that produce no error

The ones worth memorising, because nothing in the stack reports them.

symptom	cause
"The model forgot what I sent."	client limit.context > server OLLAMA_CONTEXT_LENGTH. HTTP 200, normal finish_reason.
"Every message takes a minute."	The resident model is not the one being requested; each turn evicts 22–25 GB and reloads.
"Slow after I ran two things."	Two heavy models competing. Only one fits.
"First message hangs."	Cold load, 54–84 s. Normal. Warm it.
Cannot connect to API, server healthy	The tunnel died — see §3.
Turns cost 620–860 s for 50 tokens	A context shift. Unrecoverable; restart the session.

Measuring correctly

• Divide by the server's eval time, not wall clock. An early attempt divided
  completion tokens by total wall time and produced "2.21 tok/s at 27K" — nonsense;
  prefill dominated the clock. The log prints prompt eval time and eval time
  separately.
• Check for a competing session. With two slots, another client silently halves
  your numbers. Confirm all slots are idle and that only your task ids interleave.
• Distrust impossible orderings. A contaminated benchmark once read slower at
  1.3K than a live session was getting at 29K. Generation cannot improve with
  depth; if it appears to, you measured contention, not throughput.

---

9. What this setup is good for

GLM at 46K context, 2 slots, on one dedicated Mac, reaches roughly 50–60 tok/s
on shallow contexts and 17 tok/s at 29K, with retrieval that stays reliable to
at least 47K. That is a usable coding assistant for bounded, well-specified work,
and it costs nothing per token.

It is not a substitute for a frontier model on ambiguous work, and the discipline
it demands — shallow contexts, one model resident, serial dispatch, verified
configuration — is the price of running it at all.

The companion article, Tuning a Local Model, covers what to change when it is
not behaving.