Mcaster1YPMan
High-performance C++17 daemon synchronising the entire Xiph.org radio directory into a local MariaDB. Crawls, verifies, sanitises, and serves 14,700+ internet radio stations via REST API and YP-XML export. Atomic table-swap pipeline. Zero-downtime sync. The brains behind yp.casterclub.com and audiorealm.net.
By The Numbers
What a YPMan-powered directory looks like in production.
The Sync Pipeline
A 10-stage pipeline runs every hour. Concurrent thread pools per stage. Atomic table swap at the end.
Key Features
Engineered to never serve stale or partial data, even under upstream failure.
Xiph.org YP Mirror
Pulls dir.xiph.org/yp.xml (5+ MB, ~14,700 entries) every hour. Conditional GET with ETag & Last-Modified. No upstream hammering.
Concurrent Crawler & Verifier
Phases 1, 2, and 3 run concurrently using a hardware-aware thread pool. Auto-sizes to CPU cores and available RAM. Full 14,700-station sync completes in minutes.
Metadata Enrichment
Genre normalisation (3,639 distinct buckets), bitrate detection from ICY headers, country/language inference, station-type detection (Icecast vs SHOUTcast vs Steamcast).
Deep Liveness Check
Verifier opens the stream URL like a real media player, reads the first audio frames, and confirms playable content — not just a 200 OK. Dead streams flagged, retried, eventually pruned.
Atomic Table Swap
All writes go to temp_stations. After sanity checks (count ≥ 100, integrity), a single RENAME TABLE temp → live swap. Zero downtime, zero partial reads.
Swap Guard Safety
If a sync cycle ends with fewer than 100 stations in temp_stations, the swap is aborted and live data preserved. Upstream outages don’t corrupt the directory.
REST API
Embedded HTTPS server exposes the directory data: station search, genre browse, station detail, listener counts, station-rank, mount lists. JSON output, Bearer auth, rate limiting.
Public YP-XML Export
Generates a clean /yp.xml for white-label consumers (audiorealm.net) and federation peers. Drop-in replacement for the upstream Xiph YP feed.
30-Day Archive
Every fetched yp.xml is datestamped and retained for 30 days under archive/. Historical replay, diffing, and forensic analysis available out of the box.
Technical Specifications
Engineering details for evaluators.
| Language | C++17 (daemon), PHP 8.4 (admin web UI) |
|---|---|
| Build | Autotools, make -j4 |
| Architecture | Celenite Stack — C++ core + PHP-FPM admin via FastCGI |
| Dependencies | libcurl, libxml2 (SAX), OpenSSL, yaml-cpp, libmariadb |
| Threading | Hardware-aware thread pool, auto-sized to CPU/RAM |
| Database | MariaDB / MySQL, prepared statements only, no FK constraints |
| Sync Interval | Configurable, default 3,600s (hourly) |
| Atomic Swap | MySQL RENAME TABLE with zero-downtime guarantees |
| Public Ports | 9689 (web), 9688 (REST API) |
| Output | JSON REST API + public /yp.xml feed |
| OS | Linux (Debian Trixie tested, any modern glibc) |
| License | Proprietary — MCaster1 LLC |
| Production Deployments | yp.casterclub.com, audiorealm.net |
| Repository | https://github.com/davestj/Mcaster1YPMan |
Who It's For
Three audiences this was built for.
Radio Directory Operators
Build your own TuneIn / Streema / iHeart-style discovery experience. You own the data, the user, the revenue. No upstream licensing fee. No SaaS dependency.
Internet Radio App Developers
Build a station browser app without scraping Xiph yourself or paying a directory aggregator. JSON REST API, real-time station status, genre taxonomy already done.
White-Label Radio Networks
Run your own private directory of curated stations with the same battle-tested pipeline that powers the public deployments. Sanitiser and ranker are pluggable.
Part of the Mcaster1 Ecosystem
YPMan feeds the directory data consumed by audiorealm.net and yp.casterclub.com. Listener traffic to those stations is routed through Mcaster1StreamProxy for HTTPS wrapping, and protected by Mcaster1BackDraft at the WAF layer.