Tuesday, July 19, 2011

Getting a replay on my ReplayTV

In 2003 I purchased a ReplayTV DVR unit after watching an interesting show on the now defunct TechTV network about DVRs and hacks available for the ReplayTV units.

The ReplayTV has a lot of faults, but the technology was quite new for the time, and its execution was better (in some respects) than the generation one Tivo DVR units. Generic ReplayTV systems made their way into numerous cable and satellite system customer premise equipment, becoming the basis for early DVR receiver/decoder offerings for these systems.

I purchased the nearly $400 unlimited service activation (required to get access to electronic programming quide data, without which a DVR becomes little more useful than a traditional VCR), figuring I would own the unit more than two years, the time necessary to earn back the money spent on the month-by-month service option.

Overall, it's been a really good experience. The unit fulfilled its promise to change the way we watch TV. It did that instantly. At my house, we were chained to the TV schedule. A VCR alleviated some of this, but programming it successfully required error-prone transcription of a paper or web schedule. Even then, you only had 2-7 hours of recording time to work with, assuming you start a new tape.

VCR technology was game-changing for its time, and DVRs like the ReplayTV re-revolutionized the TV viewing experience, allowing one to watch-pause-instant replay live TV. Traditional VCR-like functions of recording individual shows were made an order of magnitude simpler by integrating a guide grid into the unit's user interface. No more scanning a paper or web table and manually entering times. Scroll to your show with the on-screen guide and press record. Done.

The DVR further revolutionized TV by harnessing the power of database-driven scheduling of recordings. The ReplayTV and Tivo units diverge slightly in this area, and some regard the Tivo as being stronger in this regard. But, the ReplayTV is pretty decent at this too, allowing for variable-priority theme channels, which attempt to record shows on supplied criteria, giving way to other such themes you set at higher priority (record Scrubs, unless Mythbusters is on, then record that instead), and giving way once more to specific individual or repeated-timeslot recordings you specify.

I was able to pore over a generic TV lineup at the beginning of a season, and setup this database to essentially automate our TV watching. After a few days, the ReplayTV started to fill up with a library of constantly updating shows of my interest. I stopped watching TV and began watching the ReplayTV. I was unchained from broadcasters' schedules. If there was first-run programming coming on which I was desperate to see that night, I made sure to wait at least ten to fifteen minutes into the show before starting to watch the recording as it was being made. This way, I could watch the show from start to finish without waiting during commercial breaks (advertisers note: I often found myself entranced by some commercials, and never skipped them, even though the skip button was always under my thumb).

While groundbreaking tech for 2003 on, here in 2011, much has advanced. Were it not for some specific hackability of my model unit, I would have lost the use of my ReplayTV within the first two years. Its original hard drive failed rather alarmingly quickly, and a replacement drive I installed myself failed after several years in service. In both cases, thanks to backups I made of the unit's software, I was able to return the unit to service after replacing the drive and its factory software.

My ReplayTV unit was designed before HDTV standards were solidified, and almost no providers were offering HD programming. Today, over-the-air TV has transitioned to digital broadcast, and most broadcasters have adopted an all HD format. I am a cable customer, and my provider, in addition to having a standard-def. lineup, over the last few years has made continual improvements such that essentially the entire lineup is available also on HD channels further up the "dial." There are also plenty of the earlier special-purpose HD channels which originated from day-one as an HD only offering (when HD was still new and quasi-experimental), and have no SD equivalent.

Anyone who has bought a TV set in the past few years has purchased an HD model, with QAM digital tuners able to get the OTA broadcast and in increasingly-many cases even new digital cable programming without the aid of a special receiver.

I am still using a quite serviceable SD tube-type analog TV. And ReplayTV, designed for the analog SD TV world, is becoming increasingly irrelevant in this universe of digitally delivered HDTV. However, if you don't mind the SD, this legacy equipment still works great when paired with a receiver from my cable provider.

I was saddened then to hear the news that after many years of near-trouble-free operation of my ReplayTV, the corporate entities behind it were going to finally switch off the internet servers delivering the electronic program guide data that make the DVRs so useful. Forever.

After July 31, all ReplayTV units still functioning out there will lose guide data, and will (gracefully, we all hope) fall back to functionality equivalent to a VCR, just with random-access digital storage.

The die-hard user community has been working tirelessly toward an alternative solution to deliver guide data after this date, and with ReplayTVs corporate stewards offering some modicum of help (or at least not standing in the way), appear to have forged a likely workable path forward.

Your home PC, with special software, will stand-in for the outgoing ReplayTV guide data servers. It will get its guide data for a minor fee paid to a non-profit company with a mission to promote open-source DVR technology.

Last night I endeavored to set this software up on my home PC and give it a test. After a lengthy series of hiccups and other not-insurmountable minor gotchas and snags, I was ultimately able to start my own server, and get my ReplayTV DVR to accept it. The server ingests schedule data from a third-party (right now I am testing it with software known as an XMLTV scraper, which reformats publicly available TV listings into a format which this server software can ingest), and offers it to my ReplayTV upon request.

The result? So far, it's working beautifully! I have cut-ties 100% from the corporate-sponsored ReplayTV guide data servers, and have every expectation that after July 31, I ought to continue to be able to run this way.

The server need not run continuously, but often enough to keep the ReplayTV DVR's schedule database populated as far into the future as data availability and convenience dictate. On the server-side, the experimental XMLTV scraper doesn't download much data, but requires a good 15-20 minutes to assemble and reformat it, making it the slowest and weakest link in this chain.

During this testing-phase, I imagine initiating a manual update about once-or-twice a week, given this. After July 31st, assuming everything continues to work, I will likely buy a cheap subscription to the third-party data service, replacing a 20min free scrape with a cheap download lasting only seconds.

One problem likely solved, but still uncertain, is the ability of this home surrogate-server software to set the time-of-day clock in the ReplayTV units. I believe this has been solved, based on an experiment I did.

As part of its nightly network connection, the ReplayTV units use the NTP protocol to reset their clocks by polling an NTP server also maintained by ReplayTV's corporate stewards. You wouldn't want this clock to become off, or the DVRs idea of the current time would yield partly or completely missed recordings.

The community forums speculated that the NTP time updates were cryptographically signed with a key maintained in the corporate servers, and the DVRs would not accept unsigned NTP time replies. I don't know if this is the case or not, but I did notice that the home surrogate-server software came preconfigured with the address of the ReplayTV corporate NTP server, but that this was changeable.

I updated the configuration to point my server at my favorite NTP source: north-america.pool.ntp.org, a domain-name that reverses (in the load-balancing mechanism of DNS) to any of a number of publicly available NTP servers aimed at covering the continent.

The server software accepted this address without complaint, appeared to use it in the logs, and offered the time derived from it to my ReplayTV DVR, which likewise "appeared" to accept it (there's no easy way to tell, except to enter a hidden code to get the unit to display its clock onscreen, which had previously been 5 seconds off, but is now less than one second offset from my radio-controlled wall clock).

So I have every reason to be optimistic that after July 31st, I will be able to continue enjoying my ReplayTV experience in all its 2003 analog standard-definition shininess!

In the future, I expect that once my existing SD TV goes, I will modernize my whole viewing experience. I expect to become a Roku box customer, perhaps even playing with the idea of ditching my cable-TV subscription in favor of a combination of OTA digital signals and a more expensive, but higher-bandwidth cable-internet offering with a Netflix account riding on top. To be the best, the Roku or similar solution would need to be able to ingest OTA HDTV signals in addition to having connectivity with my home network to play media stored their or stream content from the internet. That way I get full DVR-style flexibility with local broadcast content too!

Until that heady future, I'll enjoy a replay of the future circa 2003, and continue enjoying my ReplayTV, faithfully (more or less) recording content for presentation on my analog TV!

(My analog TV is a 32" model from Sharp. I bought it in 2000, and it's still bright and saturated, undistorted and color accurate. I've been astonished how many newer-model tube-type analog TVs have degraded to unsuitable picture quality after far fewer years in service! It's only problem is a stuck relay, which prevents it from being turned off occasionally. I have to get up and pull its plug, but other than that, it's still going strong. In fact, it's outlasted a couple of Sony Trinitron models my folks have owned!)