Backup Options for Filmmakers
As filmmakers, we put a tremendous amount of trust in our equipment. With the rise in popularity of solid-state media, many of us are no longer shooting on tape or film. This offers many advantages but also several disadvantages. Unlike a film workflow in which many prints are made, or a tape workflow where the tapes are digitized to a hard drive and then stored safely away, filmmakers are often erasing their solid-state media and relying on a single hard drive copy to last them throughout the post production process and onto distribution/release.
Three things are guaranteed in life - death, taxes and hard drive failure. It is a fact that every hard drive will eventually fail. No-one can predict when it will happen and you may not have an opportunity to salvage the data before it does.
So it is wise to assume it will happen and have a good backup policy in case it does. Here are some of the options available for data backup.
Data Tape

While there used to be many consumer options available such as the Iomega Ditto, tape backup is now almost exclusively geared towards enterprise users. Higher-priced options have high capacities and autoload capabilities - i.e. where you insert multiple tapes at once and the system switches them automatically so you don't have to. Lower-priced options will generally require you to change the tape far more often - if you're fine with that, you can save a lot of money.
Most backup drive manufacturers are moving towards the LTO format. This is an open standard so you know that the data you backup now will work on a competitor's drive for the next 1-2 LTO generations.

One way to save money is to use an older, less advanced format. A lot of manufacturers still offer older proprietary systems for sale. These are not as advanced as the LTO options (nor as compatible with competing products) but they are much cheaper. As stated earlier, cheaper drives have lower storage capacities and are less likely to have autoloading functionality. They are also likely to have much lower data transfer rates, unlike LTO -3 and LTO-4 which can transfer data at higher speeds than a conventional hard drive.
When purchasing a drive, consider also the connector and the supplied software. Some drives use SCSI and will need a SCSI card installed inside your machine, but setup will be much easier if you opt for a Firewire version instead (not to mention allowing greater drive portability). Check if the supplied software is compatible with your operating system and if not, check if there is a compatibility update available.
Because these tapes are designed for enterprise use, they are very sturdy and have extremely low failure rates. They have normally been stress-tested by the manufacturer, and most manufacturers will offer you failure statistics on their site. You are, of course, paying for this privilege though.
Format types: LTO, VXA, SLR, DLT, DDS, AIT/SAIT, Travan, T10000
Drive manufacturers: Tandberg (formerly Exabyte),
HP,
Quantum,
IBM,
Sun StorageTekSoftware: Retrospect,
NetVault,
CommVault,
UltraBac,
PresSTORE,
ATempoPros:* Low failure rate
* Tried and tested
* LTO-4 tapes can store terrabytes of data and access it faster than a regular hard disk
Cons:* Expensive
* Can be complex to set up
* If you opt for a cheaper, small capacity, non-autoloading version, you will spend a lot of time changing tapes
* Overkill if you don't have much data to backup
Verdict:Great if you're backing up a large amount of data and only plan to keep one copy.
Hard disk

The main advantage of hard disks is market penetration. You can go into almost any store and buy a disk at short notice. They are priced very low per GB, there are lots to choose from and (unless you have an old machine) you shouldn't need any new hardware or software in order to use it.
As this is primarily intended as a backup, go for an external drive. Speed is not an issue here so mounting the drive internally will offer no worthwhile performance benefit. Additionally, an external disk safely stored away will not be damaged if something happens to your machine.

It's worth mentioning that some manufacturers offer separate Mac and PC editions of their external disk drives. This is not a marketing gimmick - there is a difference! Some of the I/O controllers in certain drives (often cheaper ones) are not Mac-compatible and you will experience issues if you use one of these drives. Choose a drive enclosure with an Oxford controller where possible. Check online reviews if in doubt.
Finally, one aspect people often don't realize is that hard disks store data magnetically and must be "refreshed" every six months or so to prevent data loss, as Larry Jordan explains in
this article. This must not be overlooked when considering hard disks as a backup medium.
Manufacturers: Lacie,
G-Tech,
Samsung,
Western Digital,
Seagate,
Maxtor,
Hitachi,
ToshibaPros:* Cheap
* Ubiquitous
* High capacity
* Fast transfer rate
Cons:* Relatively high and unpredictable failure rate
* Physical bumps and bashes increase the chance of failure
Verdict:Good for cost-effective backups but multiple backups are advised.
Solid-state flash drives

The main advantages of these drives are their small size, high potential read speed and high damage tolerance. It is likely that these will eventually replace conventional hard disks in the future and this will be a good thing, however current models have certain caveats.
The biggest of these is price. The cost per GB is considerably higher than any of the other formats on this page, and available capacities are much lower than those offered by conventional hard disks. In addition, although read speeds are fast, write speeds are considerably slower than those of conventional hard drives.

One of the most interesting aspects of flash SSDs is the way that they fail. Unlike rotating disk hard drives, SSDs do not suffer from mechanical failure but gradually wear out as you write to them. Although they have quite a high failure rate (each memory cell is limited to 100,000 writes), failure doesn't actually result in data loss. When a memory cell fails, you will be unable to write data to that cell - but you will have no problems at all reading data from it. This would be awful for a scratch disk that is written to many times but perfect for a backup in which reading is more important than writing.
However, controller chips inside the drives have been known to fail, meaning you will need to call a data recovery expert to recover the data from the disk. And there lies another problem - a lot of manufacturers uses proprietary chips that are constantly changing, making it difficult for a data recovery expert to keep up with developments. And in many drives data is difficult to recover by design, as these systems were originally developed for the military to carry sensitive information.
Efforts are being made to resolve some of these issues - such as putting two drives in a RAID 0 to improve write speeds, and balancing write operations across the entire drive to reduce the load on a single cell. Prices are going down and capacities are going up, but they will only become viable when the cost hits $1.50 per GB or less.
Manufacturers: Samsung,
IBM,
Intel,
Corsair,
SanDisk,
ToshibaPros:* Very fast read speeds
* Tolerant of physical abuse / damage
* More predictable failure rate than regular HDDs
* Data can still be read when drive fails
Cons:* Very expensive per GB
* Slow write speeds
* Current drives have relatively low capacities
* Limited number of times the drive can be written to
Verdict:One to look out for the future but limited to small backups right now.
Online Backup / Storage

There are several online backup services available. Some will give you a certain amount of space for free, requiring you to upgrade if you need more, while others offer a trial service. The advantage of one of these services is that the data is stored elsewhere and so will not be affected by theft, fire, water damage etc affecting your facility. Many of them offer software that runs on your machine and backs up your specified backup folders automatically, ensuring an up-to-date backup even if you forget.
When considering this option, you should consider security as a priority. You are handing over your files to someone else so you need an assurance that they will not end up in the wrong hands. I use Mozy to backup my laptop and all files are encrypted by default. The simplest option is to let Mozy create an encryption key for you but this is not as secure as specifying one yourself. If you do specify a custom one, beware that your data will be permanently inaccessible if you forget it.

The two biggest problems are storage space and transfer speeds. It is impractical to upload files greater than a few hundred megabytes, especially as the service takes quite a while to encrypt them before uploading. So backing up terrabytes of footage is not possible, but these services are very useful for backing up important project files. Many of them will store multiple versions of a file so you can restore to a version several days or weeks in the past. I wouldn't recommend this as your sole backup, but it would be useful as an extra cushion, especially as some companies give a small amount of space for free.
Services: Mozy,
Carbonite,
Dr. Backup,
DropBox,
iDrivePros:* Simple and automatic
* Great for backing up small files
Cons:* Not suitable for large files
* Subscription fees
Verdict:Great as an additional backup but don't rely on this as your only option. Only suitable for small files.
Videotape

Videotape backups offer some distinct advantages over data tape backups. Firstly, unlike the LTO specification that has various revisions, video formats adhere to strict standards that rarely change. These standards tend to stick around for a long time. A brand-new DVCAM deck will play a DVCAM tape made a decade ago with no problems. LTO drives are only backwards-compatible with the last 1-2 generations which could cause problems with long-term backup.
Secondly, there is greater predictability. Tapes are rated at the hour or half-hour, making it much easier to calculate how many would be needed and how long it would take to record/play the footage, which is invaluable if you are planning to rent a deck. The downside of this, of course, is that transfer speeds are much lower than modern LTO drives.

However, when creating a videotape backup of your footage, it is important to choose the format carefully to avoid quality loss. If you shot on the Panasonic HVX-200, which shoots DVCPRO HD, it is recommended to output to DVCPRO HD tapes. If the codec you are using does not have a tape equivalent, output to a tape format that closely matches the frame size, frame rate, color sampling and approximate data rate of the original footage. Some formats like Redcode RAW 4K do not have tape equivalents and so a different backup method must be used unless you are willing to lose information.
Also make sure your tape timecode matches the timecode of the original footage, otherwise your NLE will not be able to accurately reconnect the footage to the clips on the timeline. You will spend a lot of time manually rearranging and synchronizing footage.
Formats include: MiniDV, HDV (varies per manufacturer), DVCAM, DVCPRO, DVCPRO HD, HDCAM, HDCAM SR, D1, D2, D3, D5, BetaCam, DigiBeta.
Manufacturers: Panasonic,
Sony,
JVCPros:* Durable
* Established standards
Cons:* Limited to real-time capture and playback
* No tape equivalent of certain formats
* Information must be captured rather than simply copied to a hard disk
Verdict:Great for backing up established formats. Excellent for long-term backup / archival.
Blu-ray

Blu-ray discs can store around 50 GB and have a relatively low price per GB. They are compact and, if stored in a solid case, are relatively durable compared to regular hard disks. They are not as durable as tape-based alternatives, however.
Blu-ray hasn't caught on as well as everyone had hoped after the format war ended, and Apple has not yet implemented hardware or OS support for it yet. This has severely limited the availability of Blu-ray burners and software for the Mac, with the only option for data discs being Roxio Toast 10 plus the Blu-ray plugin. PC users have a lot more choice but even despite this, it has still not fully caught on in the PC market either.
Transfer rates are quite low and the format doesn't offer as much disc space as other formats. However, BD-R does have an advantage as a backup medium because it can only be written to once. Every other format can be written to or erased (some more easily than others) after a backup has occurred.
Drive Manufacturers: Sony,
LaCie,
LG,
PioneerSoftware Manufacturers: Roxio,
Sonic Solutions,
Nero,
Adobe EncorePros:* Discs relatively cheap per GB
* Data cannot be overwritten
Cons:* Not much support on the Mac
* Slow read/write speeds
* Low disc space compared to other offerings
Verdict:Don't rely on it as a sole backup. Not suitable for large amounts of data.
A Game of Chance
None of the options listed above are infallible, however the point is to lower the
chance of losing your data. If you have one backup, that lowers the chance of critical data loss to 0.5. Make another one and it goes down to 0.25. Backups are especially crucial if you're running a RAID because the chance of data loss increases with every drive you add (unless it is a RAID 1 of course).
So it doesn't really matter which option you choose, as any one of them will reduce that chance - some more than others of course. A mix of multiple types is the safest way to go. And when thousands or millions of dollars, plus the culmination of months or perhaps years of hard work are at stake, it helps to have a pro-active backup policy planned from the start. The fate of your movie might well depend on it.
The links to companies and products in this article are intended for guidance and not as an endorsement.Posted by Jon Chappell on Feb 12 2009 to
Hardware,
Video Editing,
CamerasPermalinkSpeculation on Mac Pro refresh
It's been more than a year since the current Mac Pros came out. Although the software to fully take advantage of them is still not here, a year is a long time and technology has moved on. As a 2009 refresh is very likely, here's some speculation on what the new machines are likely to be like.
First of all, a release to coincide with Snow Leopard is very likely, as the Mac Pros are the best-equipped machines in Apple's lineup to show off the new OS. Snow Leopard will be released (assuming no iPhone-related delays this time) in either Q1 or Q2 2009, discounting any hardware that is due to be released after this date.
CPU

I'm almost certain this will be a Xeon 5500 series "Gainestown". That will be Intel's most powerful CPU line until Beckton towards the end of the year.
Although the clock speeds are identical to the Harpertown range inside current machines, they offer significant performance boosts elsewhere. The biggest of these is an on-die memory controller (i.e. built into the CPU instead of on the motherboard). This reduces bottlenecking significantly and is something AMD's chips have had for several years. It will provide a major performance boost.
Another new technology is QuickPath Interconnect, which replaces the legacy Front-Side Bus (FSB). The FSB connects all system devices and memory to the CPU through a single interface. This is now split up, with the new memory controller handling memory and QuickPath Interconnect dealing with the other system components. You may have noticed that the caches on these CPUs are smaller than their predecessors - this is because the greater efficiency in the new architecture means data spends less time in cache memory.
There have been several optimizations to multithreading, including hyperthreading which allows two threads per core (great for Snow Leopard). Although Gainestown is limited to four cores per CPU (8 in total in a Mac Pro), Apple may market it as a 16-core machine because of the 8 extra logical cores. True 16-core machines will not be available until the end of the year when the Beckton series is released.
Memory

You're probably familiar with dual channel memory which doubles memory bandwidth. That's why you have to install Mac Pro memory modules in pairs.
The new architecture has triple channel memory! Mac Pro motherboards are expected to have three rows of four memory sockets, totaling a maximum of 96 GB (12 x 8 GB). You would be required to install chips in groups of three.
Hard Disk Options

2 TB internal hard disks are now available so it seems likely that Apple will offer these in their machines. This will allow up to 8 TB in total.
What about solid-state disks? I doubt they will be offered for the Mac Pro just yet. They're too small for media storage (unless you can afford to RAID them) and although they are a decent size for boot disks I don't think they are suitable for that purpose just yet.
I always put Western Digital Raptors (10,000 RPM) as boot disks inside my machines because they reduce boot times, load applications faster, and everything is generally more responsive as memory can be paged to and from disk a lot faster. So the fast read speeds of SSDs are welcome from me, but there are two problems. Writing is slow. The OS writes a lot of data to disk and you will notice a slowdown when multitasking or using a lot of memory at once.
The second problem is that each memory cell inside the drive has a lifetime of around 100,000 writes. You can easily exceed that in the lifetime of your computer, particularly if it is switched on constantly. Modern chips try to get around this by writing evenly to all areas of the disk in order to balance it out, but the usefulness of this will depend on the total size of the disk.
SSD technology looks very promising and these are problems that will be overcome as time goes on, but they are not ready for this kind of usage just yet in my opinion.
Graphics cards

I don't know what the stock graphics card will be but the following cards may be offered as options:
Radeon 4870 - This is a high-end card capable of accelerating H.264 and MPEG-2 encoding and decoding. ATI's drivers are traditionally much better optimized for Core Image than NVIDIA's so Final Cut Studio performance is likely to be good. Crossfire technology is available on the PC version of the card - it remains to be seen if this is something Apple will embrace. Another version, the 4870 X2 with dual GPUs, will not be available for the Mac.
It comes with DisplayPort and HDMI connectors. Apple may request a Mini DisplayPort connector instead (or in addition).
GeForce GTX 285 - This is a single-GPU card that is SLI-capable (NVIDIA's equivalent to Crossfire), should Apple choose to utilize that technology. This card does not offer DisplayPort by default so this is something likely to be requested by Apple for the Mac version.
Let's hope NVIDIA improve their Mac drivers to offer better Core Image performance.
Quadro FX 5800 - This card supports DisplayPort technologies as well as the brand-new OpenGL 3.0 specification (which the other cards do not). It's very fast but also very expensive. It is not necessarily guaranteed to make a significant difference to Pro App usage because, while it has a lot of raw power, it does not have some of the optimizations that the other cards have (e.g. H.264 hardware acceleration), and I don't know how well-optimized for Core Image they are. In addition, raw power is worthless if it is bottlenecked by other components in your system.
With the launch of Snow Leopard, Apple will be trying to sell us multiple graphics cards in our machines. I would personally wait until I have read reviews before deciding on purchasing multiple cards for Pro App usage. Also, I would have to wait until Final Cut Studio 3 came out anyway because multiple cards with FCS 2 cause problems.
Things you shouldn't hold your breath for
* eSATA - Apple has completely ignored this particular technology in the past - this time around will most likely be no different.
* FireWire S3200 / USB 3.0 - It's early days and I'm not aware of any devices currently supporting the new standards.
Conclusion
It's pure speculation but if my predictions come true, there will be a hell of a lot of new tech in the Mac Pros. Which is great because they'll be leaving a lot of legacy technologies behind - but on the other hand, new things carry a certain degree of risk.
The first-gen Mac Pros needed a couple of firmware updates to fix minor issues, and I have my own rule that says I refuse to use an operating system for professional work until at least 10.x.4. This rule will be particularly crucial for Snow Leopard which now has a 64-bit kernel, meaning every system component will need new drivers. This is fine for the hardware that ships with your machine but what about third-party capture cards, graphics tablets etc? I don't think it's going to be a huge problem but it's worth thinking about in advance.
Either way, 2009 looks pretty good for video professionals.
Posted by Jon Chappell on Feb 4 2009 to
Apple,
Hardware,
AnalysisPermalink