Contributors

Monday, November 21, 2011

Film is Dead, or Why the Cloud is Evil

Movies are doomed. At least, movies on 35mm film are doomed. According to a report from IHS Screen Digest Cinema Intelligence Service (via Scientific American):

A report from the IHS Screen Digest Cinema Intelligence Service said that 35 mm film, which has been the dominant projection format in movie theaters for more than 120 years, is nearing the end of its life, as the majority of cinema screens in the U.S. are expected to go digital in 2012. 
In fact, IHS expects 35 mm will be replaced by digital technology globally by 2015, the report said. By the end of 2012, 35 mm film in movie theaters is expected to decline to 37 percent on a global scale, which is a dramatic decline from 68 percent of global cinema screens in 2010.
A lot of people don't think this is a big deal. But I'm not so sanguine about losing these analog formats. Old technology, such as books, newspapers, sheet music, piano rolls, photographs, slides, 35mm movies, phonograph records, and even AM radio broadcasts are extremely simple to read and interpret. With a little bit of observation and ingenuity, anyone can simply look at them, run some experiments and figure out how to view or listen to them. But our new technology is not quite so simple.

Back in the 1950s 8mm cameras became cheap and popular, and anyone could afford to make home movies. This tiny, soundless movies were far cry from the glory of 35mm films with stereo sound, but they were the first step toward ubiquitous audiovisual entertainment industry. In the 60s Super 8 was developed, adding a magnetic strip to 8mm film. In the 1970s two competing home video recorder standards were introduced, Betamax and VHS. Ultimately VHS won out. LaserDiscs were introduced in the late 70s, and film aficionados loved them. But when DVDs were introduced they died off. As high definition TV was being developed DVDs just didn't have enough capacity, so two competing formats were (again) developed, and eventually Blu-Ray won out.

A similar thing occurred with phonograph records. Originally wax cylinders were used, but 10" 78 rpm records soon became the standard. They were two-sided and could hold several minutes of music. As technology improved, the size and rotation rate were reduced and 45 rpm records were introduced, and then 12" 33 rpm albums came out that could hold a dozen songs. The introduction of stereo culminated in what some people still think the ultimate in audio perfection: the vinyl stereo record. As LPs reached their pinnacle audio cassettes were introduced (and later, 8-track tapes which were used almost exclusively in cars), but tapes were never able to reach the same high level of quality that records achieved. After briefly flirting with quad, vinyl LPs gave way to CDs in the early 80s, though in some quarters LPs are still preferred. Now MP3 audio files are replacing CDs.

Proponents view this constant progression of ever-obsolescent formats as the natural advance of technology. Skeptics consider it a marketing ploy to get you to buy the same thing over and over. You only ever need to buy one copy of Philip K. Dick's book, Do Androids Dream of Electric Sheep?. But you could buy Bladerunner, the movie based on the book, on VHS. Then you could buy it on LaserDisc. Then you could buy the Criterion collection edition on LaserDisc. Then you could buy the director's cut on LaserDisc. Then you could buy the theatrical release on DVD. Then the director's cut again on DVD, with an alternate ending. Then you could buy the whole sequence again on Blu-Ray.

The same thing happened with records. You could buy an album on vinyl, then on cassette, then on 8 track, then on CD, then as an MP3 download, and then again as part of a subscription to an online music service.

And I'm certain to have missed dozens of formats that shone briefly, only to be found decades later in people's basements, causing them to wonder what possessed them to waste so much money on something that so obviously had no future.

In parallel with audio and video technology, computers were becoming more sophisticated, and now they've been recruited to replace all other forms of media: records, film, video, books, and newspapers.

When I was a kid in junior high people used teletypes and yellow paper tape punched with holes to store data. When I was in college we used punched cards to write programs in FORTRAN for big mainframes, or used CRT terminals to dial in to those mainframes. Personal computers started appearing in the late 70s. With computers like the TRS-80 programs were stored on cassette tape. Then 8" floppies came into vogue, to be quickly supplanted by 5.25" floppies with the Apple ][ and the IBM PC, then the 3.5" floppy with the Macintosh.

In the early 80s personal computers started getting hard disks, first 5MB, then 10MB, then 20MB. At the same time the CD started to supplant vinyl records, and later CDs were used to store computer data. A profusion of different types of double-sided, high-density floppies, Bernoulli boxes, and myriad digital tape formats came and went.

To save disk space, many operating systems began to compress data in files. Compressed data is essentially encrypted, and the more sophisticated the compression algorithm, the more difficult it is to read.

Like most people, I wound up getting a new computer every few years. With each upgrade I would copy important data from the floppies and hard disks on to the new computer. I would keep the old disks around for a decade, but then, when I no longer had a computer that could read the format, I'd ultimately toss them all out. The 8" floppies went in the 90s in the first big media purge. The 5.25" floppies went in the early 2000s. The 3.5" floppies got tossed a couple of years ago.

In the 2000s computers started having DVDs, and fortunately the drives that read them can still read CD and CDROM formats from 20 and 30 years ago. Now Blu-Ray discs are becoming common.

Digital cameras and cellphone cameras started supplanting film cameras ten years ago, and now most people store pictures on their hard disks in JPEG format, which is a compressed "lossy" format. Because the pictures are so big and people take so many, most people never back them up or print them out. Most families' picture albums will therefore be lost the next time their hard disk crashes.

The once ubiquitous book is disappearing. Millions of people now read books solely on e-readers, tablets and cellphones.

And now even the upstart CD format is becoming obsolete. People aren't buying CDs anymore, they get MP3 downloads and put them on MP3 players or phones, or they're subscribing to music services and don't actually own their own music.

The latest thing that Apple and Amazon and other companies are pushing is "The Cloud." The idea is that you don't even have your own data: they keep it all on their servers. This supposedly solves the disk crash problem because your family picture album and home videos will be in the cloud. But what are the odds Apple and Amazon will still be in business in 40 years? Forty years ago the big names in computers were IBM, Univac, Control Data, Burroughs and DEC. IBM is no longer in the computer business, having sold its line of PCs to the Chinese company Lenovo several years ago, and all those other companies are long gone. So what are the odds your grandkids will be able to find pictures of their grandparents, the way my mom could find sepia-toned images of her nineteenth-century ancestors?

And instead of having billions of books and photos and videos and records in every house in every country, we'll have only a few hundreds of single points of failure, all run by companies that will probably not last for more than a few decades. If even they don't go under, how long are people going to pay subscription fees to house ancient data that they never look at on server farms in the cloud? Despite what YouTube and PhotoBucket might have you think, they can't store that stuff for free forever.

Now consider how much power it will take to keep those hard disks spinning at these mammoth server farms day and night for decades, on the off chance that you'll want to look at the video of the kids horsing around in the pool. Already it's estimated that the Internet consumes 1% of all electricity generated in the world. Archiving infrequently used data on spinning hard disks is a ridiculously outrageous waste of energy. At some point it will simply become unsustainable.

The point of this litany is: formats are fleeting, but information should be forever. That was true with paper books, newspapers, vinyl records and 35mm film, as long as you took care of them. Those media can literally last for centuries, perhaps even thousands of years if properly preserved.

Floppies and computer tape are notoriously fragile. Tape starts to stick to itself after decades of sitting on the shelf. Magnetic domains on tape and floppies start to wander. Mylar becomes brittle with age and trying to read these media can destroy them. That's if you can find a compatible tape drive or floppy disk drive, which would be doubtful in five years and impossible in 50 or 500. Someone finding an old floppy could easily erase it by exposing it to electric and magnetic fields during their analysis.

The e-books on the Kindle and Nook are stored in non-volatile ("flash") memory, which stores data as puddles of electrons in floating gate transistors. These electrons will simply leak out after a few years. By comparison, there are still copies of the Gutenberg Bible that are more than 500 years old, and hand-written scrolls that have survived for millennia.

Newspapers have documented world history in tremendous detail for the last three centuries. But papers are being slowly strangled by the Internet. The problem is, the web is a notoriously poor historical record: websites come and go by the minute, and data in computer files is completely mutable. Finding an article in a century-year-old newspaper at the library might be time consuming, but finding a day-old web page can be impossible: it may simply no longer exist anywhere.

CDROMs, DVDs and Blu-Ray discs may last longer than magnetic media, perhaps several decades, but they are compressed or  intentionally encrypted to prevent theft. Since they're optical, future generations will find them easier to read than magnetic media just by examining them with powerful microscopes, as long as they don't degrade too much.

But future archaeologists may still be hard pressed to read what few surviving discs and tapes they might find in few hundred years. If they can cobble together the hardware to read this stuff, they'll still have to figure out the compression and the encryption.

But the problem isn't just future historians not being able to decipher our long-lost culture. If we store our data in the cloud, it doesn't belong to us anymore. It belongs to some big company. If they go out of business, or they get shut down by their networking provider in a lawsuit over networking tariffs, or their data center gets hit by a hurricane, or hackers infect it with viruses, we stand to lose everything. (And no, I don't think 40-year-old backups will be readable.)

Because our systems are so complex and interconnected they will not fail gracefully. The United States would be very vulnerable to persistent power failures. These could be brought on by underinvestment in power grid infrastructure, or failure to develop alternative sources of energy before fossil fuels run out, or outages caused by increasingly severe weather accompanying climate change, or intentional sabotage by foreign powers, or an EMP pulse detonated in the atmosphere above the central US. American businesses that depend on the cloud for day-to-day operations would all fail, and they would fail hard.

Worse, nearly all the technology we depend on for accessing this data is built in Asia, much of it in China. If for some reason the supply is disrupted it would have dire consequences. Already we're getting a taste of this: all disk drive manufacturers locate their factories in Thailand, which is now being inundated by a huge flood. The supply of disk drives will be severely affected, which affects American computer manufacturers and everyone who buys computers. A particularly ominous fact: the American military now depends on computers and components produced by companies owned by the Chinese Red Army.

I'm no Luddite. I like high-def TV, listen to MP3s on my phone, watch DVDs, and read e-books and stream movies on my tablet. But these technologies are extremely fragile. This is the information age, but all this information is so fleeting. A tremendous amount of technological infrastructure is needed to retrieve the data. If any part of the very long supply chain is broken, our capability to access our information is destroyed.

If we persist in storing our information in ever tinier and more fragile containers, and we persist in destroying the easily readable hard copies such as books, photos and newspapers, then even the slightest disruption in our highly technological society could bring everything crashing down. And since we would have no way of getting at any of the information without that technology, getting back up again may well be impossible.

It is thought that the greatest collection of knowledge in the ancient world was lost when Christians burned the Royal Library at Alexandria and presaged the dark ages. If we're not careful, we may suffer the same fate by placing all our knowledge in objects that we will one day be incapable of getting it out of.

6 comments:

Anonymous said...

I blame (R)epublicans.

GuardDuck said...

There's a saying, once it's on the internet it's always on the internet....


About grandma pulling out an old sepia toned photo of great grandpa: Notice that was phrased in the singular? Taking photos on celluloid (or daguerreotype or collodion etc) and then printing them out is expensive relatively speaking. That limits the number of pictures taken of great grandpa.

Then the in hand paper pictures have to be preserved and stored for forty to fifty years risking water damage, mold, paper eating bugs, heat cold, handling damage loss. If you still have that one image after multiple moves then that is why grandma pulled out the one surviving and bad quality image of great grandpa.

Now mom can take hundreds and thousands of pictures of the future great grandpa and they can be stored in multiple locations and in multiple formats. All increasing the chances that great grandson can view many high quality images of great grandpa fifty years from now.


And you say this is a bad thing?

Juris Imprudent said...

Hey, note the date - I fundamentally agree with N, and that isn't likely to happen again for a while.

A. Noni Mouse said...

It is thought that the greatest collection of knowledge in the ancient world was lost when Christians burned the Royal Library at Alexandria and presaged the dark ages.

I looked up the history on that when I saw the move Agora. Apparently there had not been any books in the library for at least a generation when it was burned. Christians did not destroy the books. That movie is far more fiction than fact.

Larry said...

There was some destruction by the Christians, but as you say, Noni, neglect had done far more damage in previous generations. There was also (accidental) destruction by fire of part of the Library during the fighting when Julius Caesar took sides in the Ptolomeian dynastic war. And the final destruction of what remained took place in the Arab/Muslim conquest. Damned impermanent papyrus/paper/parchment -- they should've been inscribing on stone!

Actually, I worry about electronic media, particularly in event of EMP attack or massive solar flare. However, anybody who doesn't make backups of their data doesn't really care about their data.

A lot of people don't take any precautions with their non-electronic media, though. How many people keep their negatives in a fireproof safe? We do, but we don't want to lose them. Lets face it, most people are only one house fire away from losing all that.

6Kings said...

Ironic that this post complains about digital media being too easy to lose data and laments the loss of hard copy information yet the huge loss of data from the Alexandria Library was to ... hard copy data.

No data is safe whether hard copy (due to fire, water, people, air, time, etc.) or electronic copy (errors, formats, fire, water, people, time, etc.).

So, just take care of your data (hard copy or soft) and make sure you backup in more than one place if it is that important.