Image Optimizer
An imgix/Cloudinary-style image service in a single Go binary — on-the-fly resize/convert/compress, HMAC-signed URLs, a three-tier cache, and objective quality metrics.
The problem
Developers and teams want a Cloudinary-style workflow — modern formats, responsive picture sets, colour-accurate output — without a SaaS bill or vendor lock-in. Image Optimizer runs as one Go binary plus a one-command Docker stack, distinguished by real colour management and objective, measured quality on every result.
Challenges
On-the-fly transforms without re-processing
Rendering a variant per request would re-run libvips constantly. Each unique variant is built exactly once and reused across a three-tier cache (CDN → in-memory LRU → persistent tier-2); concurrent requests for the same variant are coalesced with singleflight, and arbitrary widths snap to a fixed ladder to keep the cache-key space small.
Tamper-proof signed URLs
An unsigned on-the-fly endpoint can be abused to mint unlimited unique variants and force an expensive encode each time. URLs are signed with HMAC-SHA256 over the path plus a canonical, sorted query (so parameter order never matters), with an optional expiry folded into the signed material and constant-time verification that fails closed.
Bounding libvips CPU and memory
govips is cgo over native libvips, so unbounded concurrency risks resource exhaustion. libvips starts once with a small operation cache and per-op concurrency of 1, a buffered semaphore caps simultaneous encodes, and uploads are capped to bound per-request memory.
Colour fidelity over naive pipelines
Most pipelines assume sRGB and shift wide-gamut colours. The pipeline applies EXIF auto-orientation, then converts any embedded ICC profile (Adobe RGB, P3, CMYK) to sRGB before resize/encode; an inspect endpoint quantifies the difference by reporting mean ΔE between managed and profile-ignored renders.
Implementation
Shared vips transform pipeline
The core package runs decode → auto-rotate → ICC→sRGB → resize/fit → encode, mapping fit modes to libvips crop strategies (centre, content-aware attention, or no-upscale contain) and encoding AVIF/WebP/progressive JPEG/PNG with metadata stripped. It is free of HTTP/storage concerns, so both the server and the batch worker reuse it.
Visually-lossless AutoQuality
AutoQuality binary-searches encoder quality for the smallest file whose SSIM versus the reference stays at or above a target (default 0.99). SSIM/PSNR/RMSE/ΔE are implemented in pure Go, so every result carries objective quality metrics.
Tiered cache and storage abstraction
A byte-bounded, goroutine-safe LRU fronts a tier-2 Store interface with disk (sharded gob files, atomic writes) and S3 implementations, while masters sit behind a separate interface. loadOrBuild walks memory → tier-2 → build and reports the serving tier via an X-Cache header.
Signed delivery and responsive generation
The delivery endpoint negotiates format from Accept, sets immutable cache headers with ETag/304 support, and never serves a re-encode larger than the source. Companion endpoints publish a signed (optionally time-boxed) URL and emit signed srcset URLs across the width ladder for ready-to-paste picture markup.
Developer API and Docker/CDN stack
Routing uses Go 1.22+ ServeMux method+wildcard patterns; a file-backed, hashed API-key store and token-bucket rate limiting (per key or IP) protect the API. Docker Compose wires nginx (CDN cache) → app → MinIO, auto-creating the bucket as storage boots.
Why this stack
- Go
- A single static binary serves UI, API, and delivery, with concurrency primitives (singleflight, semaphores, rate limiting) the caching design leans on directly.
- libvips (govips)
- A streaming, low-memory image engine with a fast thumbnail op, attention-based smart crop, and proper ICC handling — the features the product is built around.
- S3 SDK v2
- One S3-compatible client covers AWS, MinIO, and GCS for both masters and the shared tier-2 variant cache.
- SvelteKit
- Compiles to a static SPA the Go binary serves directly, giving a rich playground/dashboard with no separate Node runtime in production.
- Docker Compose
- Delivers the whole CDN → app → storage topology as one command, so the caching architecture is reproducible and production-shaped.
What it does
- AVIF/WebP/JPEG/PNG with Accept-based content negotiation
- Content-aware smart crop and EXIF auto-orientation
- ICC→sRGB colour management with a ΔE inspector
- Visually-lossless auto-quality with SSIM/PSNR/RMSE/ΔE metrics
- HMAC-signed, optionally time-boxed URLs with immutable caching and 304s
- Responsive picture/srcset generation, LQIP placeholders, batch processing, and a keyed developer API
Get in touch
Building something interesting? Let's talk.
Whether it's a hard distributed-systems problem, a platform that needs to scale, or just a second opinion — I'm generally up for it.