Doc SHARD-01 Rev 0.10.0 Subject Distributed inference

++ ++

Field unit // Llama 3.2 3B // 28 layers

One model.
Many machines.
No server.

A language model split across the machines in one room. No server to shut down, no address to block. Once every machine has it, the model keeps answering with the internet cut.

Deploy it Pitch deck Source

Model: Llama 3.2 3B
Layers: 28 · split by range
Discovery: UDP · LAN sweep
Compute: llama.cpp RPC
Delivery: Pear · peer-to-peer

§ 01 The problem

Using a hosted model is a permission. Permissions get revoked — and usually not by the company. By a government.

countries where hosted assistants like ChatGPT are unavailable

ChatGPT availability — World Population Review, August 2026

313

internet shutdowns across 52 countries in 2025 — a record

Access Now, #KeepItOn

Both numbers describe the same lever. A model reachable at an address can be blocked at that address, and a model that needs the internet dies when the internet is turned off.

§ 02 The answer

Take the address away. Shard splits the model's 28 layers across the machines you can see, and they find each other on the local network. No coordinator decides the split: every node runs the same arithmetic over the same peer list and lands on the same answer.

Nothing leaves the room. Once the app and the weights are on each machine, an internet shutdown does not reach it.

Fig. 01One machine presentRefused

MODEL INCOMPLETE — LAYERS 15-27 MISSING 15 of 28 layers. The model cannot think until another machine joins.

laptop1 · 0-14

missing · 15-27

Fig. 02Second machine joinedOperational

MODEL COMPLETE 28 of 28 layers served by 2 machine(s)

laptop1 · 0-14

laptop2 · 15-27

No operator assigned those ranges. Both machines worked them out alone.

§ 03 How it works

01 · Distribution

Pear + Bare

One standalone binary, fetched by key and updated over the air. Only changed blocks travel: 80MB on disk, about 6MB on the wire.

02 · Discovery

UDP on the LAN

Each node sweeps its own subnet and announces what it can hold. No tracker, no bootstrap server.

03 · Planning

No coordinator

Peers sorted by a stable id, layers handed out in order. Nothing votes and nothing is elected.

04 · Inference

llama.cpp RPC

Every machine keeps its layers in GPU memory. The asking node drives them all through one --rpc list.

Field notes

Hyperswarm never closed the hole punch between our two laptops.

Same WiFi, different WiFi, hotspot on CGNAT — timeout every time. On the LAN we sweep UDP instead.

A layer server cannot tell you when it is ready.

ggml-rpc-server block-buffers stdout on a pipe, so we knock on the port instead of reading its output.

llama-cli writes its answer to the terminal, not to your pipe.

It passed our tests, which had no TTY, and failed on a real machine. --simple-io is the fix.

§ 04 Deployment

On every machine that will serve layers. Steps 01 to 03 are one-time; after that it updates itself.

Step 01

Install the Pear runtime

Node 22 or newer. Running pear once finishes the setup.

npm i -g pear
pear

Step 02

Build llama.cpp with RPC

The RPC backend is off by default. On Apple Silicon add Metal.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_RPC=ON -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j8

Step 03

Get the weights

1.9 GB. Shard looks in the working directory, then $HOME.

mkdir -p ~/models
curl -L -o ~/models/Llama-3.2-3B-Instruct-Q4_K_M.gguf \
  https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/resolve/main/Llama-3.2-3B-Instruct-Q4_K_M.gguf

Step 04

Install Shard

No registry, no release page. The target directory has to exist first.

mkdir -p ~/shard
pear install --to ~/shard pear://k6c99su98pmobmw1c4xqtoacsage1is5ayhq9mqdsc8gobdzg8bo

Step 05

Run it

--offer is how many GB this machine lends — about 15 layers per GB. Then open localhost:7777.

~/shard/hello-pear-bare --label laptop1 --port 50053 --offer 1
~/shard/hello-pear-bare --label laptop2 --port 50052 --offer 1

§ 05 Known limitations

Read before deploying on a network you do not control

LAN only. Machines on different networks will not find each other.
No relay for symmetric NAT. The Pear CLI does not expose one.
The RPC link is neither encrypted nor authenticated. Anything on your LAN that reaches the port can use your GPU.
No proof a peer ran your layers. You trust the machines you invited.
macOS on Apple Silicon only. The only target we built and staged. Nothing else has been compiled or tested.
No redundancy. One owner per layer; lose it and the model stops.
The weights are not distributed. Each machine fetches the GGUF itself.

§ 06 What is next

Beyond the LAN: Reach peers across networks, with a relay for the symmetric-NAT case.
Redundancy: More than one machine per range, so a peer leaving degrades instead of stopping.
Paying the peers: Designed, not built: micropayments in USDT via WDK.

Your machine holds a shard.
Together, they think.

mkdir -p ~/shard
pear install --to ~/shard pear://k6c99su98pmobmw1c4xqtoacsage1is5ayhq9mqdsc8gobdzg8bo