Search the Community

I wanted to share a problem I recently encountered on a feature I shot in May/June of 2017. This website is a great resource to search for solutions to film specific problems and I feel my experience with G Drives, partition maps, formatting, and Red Undead can help others who may end up in the same situation. I will go in depth on what I learned and end with a simple tutorial on how to fix the problem. Detailed Story The production I was shooting decided to have post-production be a Windows OS based workflow. Per usual, I checked in with the post-team to figure out their OS and together advised the production on which types of hard drives to get. Remember, a FAT32 file system is largely unsuitable for video production because of the file size limitations. The production was opening formatting all of the hard drives FAT32 so they can work on either a Windows or Mac OS, but I advised against it for the above reason. I haven't had much experience with ExFat, which holds larger file sizes and is cross platform. We decided a conservative approach and felt formatting the hard drives for Windows was our best solution. The film was low budget and slightly inexperienced, so we received our hard drives on the first day of production. We used G-Drive branded hard drives with 3TB each. Our camera system was the RED Epic with normal SSD magazines. The first night after shooting we started to format hard drives and discovered that G-Drive is specifically designed for Mac. It can work on Windows, but G-Drive's workflow for formatting hard drives on and for a Windows environment is convoluted. Instead of having the hard drive in a FAT32 format that both systems can recognize for initial formatting, the drives came pre-configured for Mac. To format them, we had to use disk utility and command prompt (see instructions here: https://support.g-technology.com/KnowledgeBase/answer.aspx?ID=14080) A key thing to note about this process: command prompt uses the clean command that erases the information that tells the computer the layout of the file structure of the hard drive. In a nutshell, every hard drive has a file that is a roadmap of all the data it stores; where each physical gate is, what row, how it's fragmented, etc. Windows and Mac have very specific structures, hence why Mac hard drives are largely unreadable by Windows and vice versa. [TANGET] Software exists that allows the OS's to read hard drives for opposing OS's [TANGENT] The clean command erases that file, but nothing else. When you're in a Windows environment, a Mac formatted hard drive will not even be recognized by the OS. The only way a user can see it is through the Disk Utility program; at that point a user can only see that a hard drive is connected, but nothing else (because Windows sees nothing else either!). When you use clean in command prompt on the Mac hard drive, it creates the drive as an "unallocated" or unused hard drive. You can then create a new volume out of this unallocated hard drive and format it to a Windows structure. From there you have a hard drive that Windows can read/write to. What if you accidentally use a clean command? What if that command was used on a Red SSD that has undownloaded footage? That happened to my production. There is no way to undo a clean command; the partition file that describes to the OS what the structure is becomes completely unique once content is written to the hard drive because the partition file is updated to reflect where that new information is written. It's like a dynamic rosetta stone; without it, there's no way to decipher the data written. It is better described as a partition map. With the partition map erased, the SSD had undownloaded footage with no way to find it. This became extremely stressful for me. However, I did find a solution. Red Undead is a Mac based program developed by Red. It's designed for recovering data from accidental formatting of SSD's. Essentially, when an SSD is formatted in camera, the partition is re-written to display the structure and all of the data is made available for re-writing. But, none of that data is changed; physically the bits are still in the position they were last in (either a 1 or 0). Red Undead takes advantage of these "old states" and reads the data. From there, based on how Red's own R3D files are constructed, Red Undead can re-create the lost footage. To recap, we had accidentally erased the partition map of a Red SSD. There's undownloaded footage sitting on the magazine, but there's no map available to tell the computer a) that a hard drive exsists and B) what and where each file is. So, we decided to try Red Undead. We formatted the Red SSD in camera (to write the Red partition file) and then brought it into Red Undead. The program then spent an hour or so reading every gate, eventually spitting out nearly all of the lost footage. We only lost a portion of the first clip, which is a lot better than losing the entire card. NOTE: do not perform a secure format of the SSD in camera. A secure format rewrites the partition file and writes 0's across the entire SSD. A secure format truly erases footage. There are a few things to learn from my experience: Hard Drives must be delivered during prep. Be adamant about it as a DP and DIT. Erasing a partition map is part of the formatting process; the damage is done. An unsecured or simple format doesn't erase the already written data, but makes it available to be written over A secured format erases the partition map and writes 0's across the entire card/hard-drive, fully erasing footage. Red Undead reads old written data that hasn't been written over yet Red Undead is only available for the Mac OS. (we should keep pestering them for a Windows port) The obvious one: DO NOT FORMAT HARD DRIVES WITH CAMERA MEDIA CONNECTED AT THE SAME TIME Of course, this technology may change over the years, but the concepts should hold. Tutorial: How to fix a Red SSD Mag that has its partition map erased If the partition map has been erased, do nothing else to the SSD mag if it's still connected to a computer. Insert the SSD mag and do an unsecure format in camera. Reconnect the SSD mag to a Mac computer. Run Red Undead in the computer. Verify which clips were saved and which were not. Ensure recovered clips are backed up before formatting the card again. --- I hope my experience can help others! I'm in no way an engineer (software or hardware), so please correct me if my terminology or concepts are wrong. Has anyone else had experiences similar to this? Does ARRI, Canon, or Panasonic offer data recovery software like Red Undead?