Archiving Digital: An Update

Yes, two posts in the same day. Not common here at the old ranch, but the number of emails I received, and IM’s and the like were enough to prompt me to add some updates.

1. I overestimated my data archive. I thought it was 50 TB, but it’s actually a little more than 40TB. But here is the thing. I’m not a heavy shooter, so I have a sinking suspicion that many others out there have FAR larger reserves than I. My archive contains the RAW files, subset folders of JPGs in varying size. In some cases TIFF or DNG, but not always. 14 sets of 2TB, Firmtek, two internal drives when combined add up to 30 TB. Six more of portables and this does not include my entire supply of 4x DVD’s. Or my film scans, not that those are essential to have in digital form, only convenient. I’m guessing the grand take is little over 40TB.

2. These were JOBS. These images are not things I feel good about throwing away. Now, anyone who, like me, came up in the film era, shudders when we hear “Throw away,” but let me tell you it is commonplace in the digital age. I know because people have been emailing me and telling me. When I first did my research back in 2002 timeframe I spoke to many different photographers who had already come to grips with losing their archive. Many seemed totally okay with it because they knew they didn’t have it in them to migrate with time and technology, something I’m REALLY feeling today. Now, the worst offenders were wire service shooters who were shooting jobs, erasing cards in the field and starting anew, ONLY saving the one or two images they transmitted. I get it. It sucks but it’s true. Each genre offers different challenges.

3. There ARE people with a plan and with savvy. I don’t know this person personally, but they were kind enough to message me with all kinds of pertinent information, and the’ve been on plan for many years. You are a RARITY my online friend at least in my experience. RW Boyer Design/Photo.

4. I don’t have it in me to migrate with time and technology, which means I’m coming to grips with losing everything that remains on portable media. Not the drive based work, but the rest of it will probably not survive. I can’t even fathom sitting down, or paying someone else, to sit down and load up my DVD’s, one at a time, and transfer that data. Talk about time and money. Only to have to migrate another 4 years later. There is certainly nothing wrong with doing this, heck I encourage you to, but knowing what I have gone on, and the limited time I have to get to it…never gonna happen.

5. This is my problem whether or not I ever shoot another frame of digital ever again. This 40TB ain’t going away. I’m sure my mega-drive is around the corner, and maybe I want till it’s here before I make a move. All 40TB on three redundant drives, then wait four years and do it again. THAT I can probably handle. Probably.

6. Unfortunately I do know people in motion, and in Hollywood, and I have heard THEIR horror stories too. Just so you know, I recently attended an awards show for motion and stills. One of the winning filmmakers got up in front of the crowd and said “Thank you for the award, but just do you know, my first and second films are gone.” “Entirely gone.” Her latest film, the award winner, had been archived by one of the major archiving houses here in the US. Also spoke to one of the archive and color houses in Los Angeles. Their guy said “There is no budget for archiving so we are throwing films away.” Not all obviously but more than you would imagine.

I know that regardless of what I say here little will change. As long as you can go to a box warehouse and buy a cheap, portable drive people will just say “What are you gonna do?” I get it. I for one am not happy with that. I love film for a bevy of reasons, but I know it’s not for most. Also, the comments section of my original post has some good information as well. Thanks to those of you who took time to write in. Much appreciated.

12 Responses to “Archiving Digital: An Update”

  1. Robert Catto says:

    It’s like the silent era of film again, practically speaking. They thought so little of their old films, they actually used the flammable nitrate stock to burn things in the new talkies they were doing…sigh. So much rich & fascinating history, lost then and losing anew now. Well, I’m doing my best to rage against the dying of that particular light, but even that may not be enough, ultimately. Time will tell!

  2. Ciaran Lee says:

    I always enjoy your posts Daniel. Regardless of whether we shoot film or digital, this is a problem all photographers face. For a while I rationalised film usage in part as money I would not have to spend on storage. Then I switched my workflow from strictly darkroom to contact printing digital negatives, so acquired a drum scanner. The files this thing produces are massive, far eclipsing raw camera digital files. So as practical a medium as film might be for storage, it is hardly the final word in protecting (and crucially accessing) an archive. Maintaining this scanner is a major pain. The software runs only on an old version of windows. The connection is SCSI. I do not know of anyone local who can service these heavy machines, or find a reliable source for replacement parts. It’s also a time drain to use. In short, I feel about as secure with my digital storage solutions, as I do with my film archive. It’s not just about longevity, but the likelihood I am going to be able and feel inclined to access and work with that archive in the future.

    Thankfully storage prices are always coming down. I hope they continue to decrease faster than the file sizes from modern cameras goes up!

  3. David Sax says:

    Hi Daniel
    Sounds like a situation that’s challenging for you, full of angst actually. Images on DVD’s, drives etc. A lot of data. Overwhelming. And it seems like the idea of throwing away old stuff in general or going through and storing only the “selects”, culling the rest and the like isn’t something that resonates with you either. It would be cool if there was this magic, perfect, self-expanding drive that had back-up ESP. But I guess this is like the concept of the perfect camera – one that does it all.

    So regardless of what you “should” do or “could” do or “need” to do, Truth is, these are personal decisions. You have to do what feels right for you.

    You already use a NAS device so you get it in terms of the basic technology that’s available and it sounds like others who know a lot more than me have been guiding you. I noticed someone also mentioned Amazon Glacier as a less expensive cloud option for data that isn’t needed routinely, like older projects. Maybe your solution rests in a combination of strategies that also takes into account what you want to spend – what it’s worth to you in terms of time and money. Maybe sit and meditate on it. You’ll find Your answer. Probably already do ;)

    Cheers.

  4. Mike says:

    Playing devil’s advocate here, how do people archive their film? Film is tangible and exists without a machine to read it, but it only exists once i.e. if it is lost in fire, flood etc. it is lost forever. With digital the photographer has the means to copy files to multiple drives in multiple locations – but then has to deal with hardware obsolescence. Shoot film and copy the keepers to digital media?

    Mike.

  5. Digital had made things more ephemeral than ever. Even if you could find a solution for storing the 1s and 0s forever in an easily accessible fashion’now you have an indexing and search problem. Possibly solvable?

    Facebook, instagram, flickr. All so-called permanent, but pile a few thousand pictures into the timeline and the early ones might as well be gone. Different problem of course, but same cause. Digital allows us to make so much stuff.

  6. Mike says:

    Andrew, good point: so much ‘stuff’.

    Mike.

  7. lionelB says:

    Sony have a 185Tb cassette tape. Putting a duplicate in an airtight container along with plenty of silica gel and ideally flushing it with Nitrogen (Argon might be easier to source) may mitigate the physical deterioration. Earth at every stage to prevent static damage. Dust free environment, obviously.

    As for transfer rates, for everything you possess now to pass just one way, once, via a Thunderbolt port will take around 60 hours. Much slower if you select or intervene in any way. 100 hours if you rely on USB 3.0.

    Storing the data is only half the problem. The format (and the version of the format) will have to be understood by the software which is available when the time comes to extract the files.

    For ultimate safety, you need two entirely different strategies running in parallel. So e.g. not two sets of cloud storage.

    To sleep at night, laser etch onto granite.

  8. Joseph says:

    Daniel,
    A solution that might not help your historical archive but would help future digital photos would be to treat your CF/SD cards as one time use only. They are cheap, tough, tiny, and are pretty reliable if not constantly being rewritten and edited. I had one go through the washer and dryer and still work.

Leave a Reply