Project Entangle, Koherence’s OTA platform, hit a milestone over this weekend: it’s archive passed 100,000 hours of (unique) recorded OTA shows!
How was the archive created?
Here at Koherence we believe in dogfooding, so we use Entangle DVRs to record and watch OTA TV. The TiVo DVRs, which we are fond of – and helped craft many of! – are serving a new life as doorstops. We also have a couple “mega” boxes set up with a large number of tuners, multiple hard drives for storage, and are set up to record a very large number of shows. They’re one way that Entangle is tested to ensure good performance for average use. One of the platforms has 16 tuners, is connected to two antennas, and has two hard drives for storage. (Strictly speaking this configuration should use three hard drives to guarantee that hard drives shouldn’t be a bottleneck. One hard drive failed about a year ago and we’ve been looking at the performance with two drives, which has been surprisingly good.) Although the DVR has 16 tuners, it can record more than 16 show simultaneously. At the moment the 16-tuner DVR is recording 30 shows, for example. Entangle optimizes tuner usage – if two virtual channels are broadcast on one frequency, it’s recording scheduler knows to allocate only a single tuner for both shows. Similarly if recordings overlap (e.g. you set an extra hour of padding for the football season pass) then only one tuner is used for the overlapped period.
Recordings from the mega-boxes are periodically moved to the archive, where they are stored for statistical analysis and testing against upgrades to the playback side of Entangle. Every so often we see oddities or errors in the broadcast and flag those as test material. These include cases where timestamps wrap in odd ways, where actual audio sample rates don’t quite match the advertised sample rate, and streams that are corrupted in “interesting” ways. Like a recent case of an EAS spliced into the broadcast with audio and video properties (and timestamps) rather different than those of the broadcast. We also look at bitrate, subchannel multiplex, and other trends over time and can, for example, compare the picture quality of a 15 Mbit/s CBR fixed-GOP broadcast circa 2012 (several stations were using fixed-GOP CBR back then) to the currently more popular 8-9 Mbit/s VBR variable GOPs. GOP structure combined with general encoder improvements has yielded significant improvements in picture quality over the years, and in part what has allowed the rise of diginets.
How much storage is that?
The archive currently holds 281.43 tebibytes of recordings spread across 144 hard drives. The storage system consists of twelve raid-5 5+1 arrays, each of which are mirrored.
What does 100,000 of shows look like?
-
- 69655 HD recordings (246.94 TiB / 61543 hours)
- 45497 SD recordings (34.49 TiB / 38667 hours)