Run Splash on M1 & M2 Macs

Splash is making waves in LLM inference on Apple silicon:

"It delivers 2× the decode speed of the next-fastest engine we measured on Qwen3.8-27B and stays ahead at every context length we tested, to 32K tokens. At four parallel subagents, the speedup increases to almost 4×."

The official release requires M3 or newer, but Erik Paperniuk has backported Splash for M1 and M2 Macs:

You can now run Qwen3.8-27B on a 2021 M1 Max at 39 tok/s: I ported Splash (M3+ only) and wrote custom Metal kernels
Splash on M1, part 2: 35B-A3B at 144 tok/s on a 2021 M1 Max, plus a head-to-head with oMLX and MTPLX (speed, temperature, power, memory)

Steps

Install: curl -fsSL https://github.com/paperniuk/splash/releases/download/1.0.2-m1/install-m1.sh | sh
Serve: splash-m1 serve --model incoai/Qwen3.8-27B-Splash (First run downloads the 17.4GB model package and prepares it for the GPU; later starts are much faster.)
Chat at http://127.0.0.1:8000. To disable the chat page, add --no-webui to the command in step 2. More settings.
Set http://127.0.0.1:8000 as the OpenAI-compatible API base URL in apps like Goose. No key is needed unless you set one. Several coding agents are supported too, e.g., splash-m1 opencode.

Update

Shortly after posting, stumbled onto this tweet from @ashxhart earlier today:

M1-4 silicone, I got you🫡

TensorFold v0.3.4 is up! Enabling lanes on M1–M4 more than doubled Qwen3.8-27B code decode on M3 Ultra to 141–158 tok/s.

Lane batching is coming next, with concurrency already showing early gains.

Follow the progress at tensorfold.dev.

Installed TensorFold 0.3.4 on a 2021 MacBook Pro (M1 Pro, 32 GB) via uv tool install git+https://github.com/ashhart/TensorFold.git and ran it with TF_ROW_MATMUL=row_qmv tensorfold serve Vontra/Qwen3.8-27B-MLX-4bit. The TF_ROW_MATMUL=row_qmv setting switches to an older fallback path, because the new v0.3.4 kernel for M1-M4 Macs crashes at load on M1 and M2 models.

Benchmarked both projects with a script crafted by Opus 5.5 in Claude Code:

#!/usr/bin/env python3
"""Chat decode speed of any OpenAI-compatible server. No dependencies.

    python3 llm_bench.py http://127.0.0.1:8000

Sends one chat prompt with thinking off, 5 times sampled and 5 times greedy,
and prints the median tokens/second (first token to last, like most benchmarks).
"""
import json, statistics, sys, time, urllib.request

URL = (sys.argv[1] if len(sys.argv) > 1 else "http://127.0.0.1:8000").rstrip("/").removesuffix("/v1")
PROMPT = "Explain how matrix multiplication uses a GPU in plain English, then give a small numerical example."
TOKENS, RUNS = 64, 5


def post(body):
    req = urllib.request.Request(URL + "/v1/chat/completions", json.dumps(body).encode(),
                                 {"Content-Type": "application/json"})
    return urllib.request.urlopen(req, timeout=600)


def run(model, temperature, seed):
    body = {"model": model, "messages": [{"role": "user", "content": PROMPT}], "max_tokens": TOKENS,
            "temperature": temperature, "top_p": 0.95, "top_k": 20, "seed": seed, "stream": True,
            "stream_options": {"include_usage": True}, "ignore_eos": True,
            "chat_template_kwargs": {"enable_thinking": False}, "reasoning_effort": "none"}  # thinking off, two ways
    first = last = None
    chunks, tokens = 0, None
    for line in post(body):
        line = line.decode().strip()
        if not line.startswith("data:") or line == "data: [DONE]":
            continue
        data = json.loads(line[5:])
        tokens = (data.get("usage") or {}).get("completion_tokens", tokens)
        for choice in data.get("choices", []):
            delta = choice.get("delta") or {}
            if delta.get("content") or delta.get("reasoning_content") or delta.get("reasoning"):
                last = time.perf_counter()
                first = first or last
                chunks += 1
    return ((tokens or chunks) - 1) / (last - first)


model = json.load(urllib.request.urlopen(URL + "/v1/models"))["data"][0]["id"]
print(f"{model} at {URL}: chat, thinking off, {TOKENS} tokens, median of {RUNS}")
run(model, 0, 0)  # warm-up
for name, temperature in [("sampled", 1.0), ("greedy", 0.0)]:
    speeds = [run(model, temperature, 1234 + i) for i in range(RUNS)]
    print(f"  {name:8}{statistics.median(speeds):6.1f} tok/s   (runs: {', '.join(f'{s:.1f}' for s in speeds)})")

Results

Splash-M1 ran Qwen3.8-27B chat about 1.3-1.5x faster than TensorFold, though that may change once the M1/M2 bug is fixed:

Splash-M1

incoai/Qwen3.8-27B-Splash at http://127.0.0.1:8000: chat, thinking off, 64 tokens, median of 5
  sampled   13.0 tok/s   (runs: 10.1, 13.0, 14.4, 11.2, 16.4)
  greedy    14.5 tok/s   (runs: 11.6, 14.4, 14.5, 14.5, 14.5)

TensorFold

Qwen3.8-27B-MLX-4bit at http://127.0.0.1:8080: chat, thinking off, 64 tokens, median of 5
  sampled    9.7 tok/s   (runs: 9.8, 9.5, 9.7, 9.4, 9.7)
  greedy     9.7 tok/s   (runs: 9.6, 9.7, 9.7, 9.7, 9.7)

❧ 2026-09-26