Lily

Perplexity's on-device inference engine for Mac, now open source.

Lily is the local half of hybrid compute in Perplexity Computer, which splits a task between cloud models and a model running on your Mac
Written in Rust with custom Metal kernels, with neither PyTorch nor MLX in its execution path
Specialized for one model, Qwen3.6-35B-A3B, on Apple silicon
Handles prompt prefill and token-by-token decode as separate workloads rather than one general path
Perplexity measured 1.23x faster prefill and 1.35x faster decode than MLX-LM on an M5 Max MacBook Pro, with output quality effectively unchanged