Jump to content

Digital "Loading"


Chris Keth
 Share

Recommended Posts

How about if you use redcine-x to go through all the r3d files and see if they play, and maybe scroll through the clip quickly? Would that be a failsafe way to check your files?

 

 

I do my first checksumed backup, then run them through red rushes at 1/8 to make sure they transcode fine. If you get an error or drop frames, you'll know you've got some hairy footage. The checksum only really helps validate that the data was copied correctly. So you could've copied corrupted data correctly. I do my other 2 backups while redrushes is running.

 

 

Unless you need to use a Red drive to keep up the data-rate, shoot on CF cards. They are faster to format, faster to backup and more reliable. I really, really, really do not like to use the red-drives. There is nothing comforting about a spinning-drive that has most likely been reformatted over a thousand times, used in heat, cold, dust and shaken. :wacko:

Link to comment
Share on other sites


I've been using Disk Utility to create DMG's ever since a P2 data wrangler gig I had about a year ago, where the post prod supervisor explained to me why he preferred this method. I have to say it made complete sense, in comparison with trusting finder to copy over each and every file identically.

 

I've never heard a peep from any shoots since then about my methods, so I assume it's actually preferred by most people who work in post.

 

So you would use disk utility to make a disk image rather than just copy the contents of the card/drive to a folder? Why?
Link to comment
Share on other sites

I've been using Disk Utility to create DMG's ever since a P2 data wrangler gig I had about a year ago, where the post prod supervisor explained to me why he preferred this method. I have to say it made complete sense, in comparison with trusting finder to copy over each and every file identically.

 

I've never heard a peep from any shoots since then about my methods, so I assume it's actually preferred by most people who work in post.

 

 

This thread creates a lot of question marks:

 

1) Would you only use the dmg-method because of the fact that disk utility does everything one at a time, instead of the finders multitasking capability? Or are there other reasons?

 

2) Isn't it just safer to watch all the files you copy? Going through the r3d file with redcine-x or some other software? Why would you need checksums if you can just check the raw files and see if they play correctly?

 

3) What kind of reports do you make? If there's a continuity report then the editor can easily match roll numbers with scene, slate and take numbers, so to me it seem like a waste of precious time to make those. What information would be useful?

Link to comment
Share on other sites

  • Premium Member
This thread creates a lot of question marks:

 

1) Would you only use the dmg-method because of the fact that disk utility does everything one at a time, instead of the finders multitasking capability? Or are there other reasons?

 

2) Isn't it just safer to watch all the files you copy? Going through the r3d file with redcine-x or some other software? Why would you need checksums if you can just check the raw files and see if they play correctly?

 

3) What kind of reports do you make? If there's a continuity report then the editor can easily match roll numbers with scene, slate and take numbers, so to me it seem like a waste of precious time to make those. What information would be useful?

 

Technically I would assume if any of the Proxies played correctly even in Quicktime then the clip had been processed properly. I would guess also for post a DMG as a back-up might be fine, but they're going to (again, more than likely) want an easily editable format for their Main hard drive.

 

Sometimes there aren't continuity reports, or camera reports, depending on the type of job. I find that it's best to just do the do every time, be able to drop a PDF on the hard drives, and make it easiest for editorial to match clips, etc than having to wade through multiple reports, especially if there aren't any.

Link to comment
Share on other sites

  • Premium Member
2) Isn't it just safer to watch all the files you copy? Going through the r3d file with redcine-x or some other software? Why would you need checksums if you can just check the raw files and see if they play correctly?

 

At some point fairly early on, yes, it's best to watch the *whole* file to be sure it plays correctly. But that's a lot of time and effort. Spot checking won't catch every problem, just massive ones.

 

Checksums are a very quick and easy way to be sure that copies of files are identical to the original -- identically good, or identically corrupted. Once you've watched to be sure the file's OK, the checksum is a vastly more efficient way to be sure it stays that way.

 

 

 

 

-- J.S.

Link to comment
Share on other sites

At some point fairly early on, yes, it's best to watch the *whole* file to be sure it plays correctly. But that's a lot of time and effort. Spot checking won't catch every problem, just massive ones.

 

Checksums are a very quick and easy way to be sure that copies of files are identical to the original -- identically good, or identically corrupted. Once you've watched to be sure the file's OK, the checksum is a vastly more efficient way to be sure it stays that way.

 

 

 

 

-- J.S.

 

Thanks John and Gus. Now to get it all straight, would you hire me if I did this?

 

1) load the media

2) check the clips with redcine-x to see if there are any problems

3) use r3d data manager to create checksums

4) copy the files to a number of locations with data manager

5) verify the files using the checksums

6) create reports for post

7) render small quicktime rushes

7) render a number of tiffs for the d.o.p.

Link to comment
Share on other sites

  • Premium Member
Thanks John and Gus. Now to get it all straight, would you hire me if I did this?

 

1) load the media

2) check the clips with redcine-x to see if there are any problems

3) use r3d data manager to create checksums

4) copy the files to a number of locations with data manager

5) verify the files using the checksums

6) create reports for post

7) render small quicktime rushes

7) render a number of tiffs for the d.o.p.

 

Unless you had a Tower with a Red Rocket I wouldn't expect you to render Rushes. I work with a Macbook Pro, which has a lot of punch, but rendering a ton of files is just too much, especially when you're using the same software to do a number of different operations. I think spot checking with REDCine-X isn't necessary - I'm still a proponent of the fact that if a Proxy runs all the way through then the camera did its job.

 

Otherwise, I pretty much do all of that, plus assist my fellow Camera Dept members in some of their typical operations. So, sure.

Link to comment
Share on other sites

Unless you had a Tower with a Red Rocket I wouldn't expect you to render Rushes. I work with a Macbook Pro, which has a lot of punch, but rendering a ton of files is just too much, especially when you're using the same software to do a number of different operations. I think spot checking with REDCine-X isn't necessary - I'm still a proponent of the fact that if a Proxy runs all the way through then the camera did its job.

 

Otherwise, I pretty much do all of that, plus assist my fellow Camera Dept members in some of their typical operations. So, sure.

 

Yeah I like Redcine-x because I can just drag the file in the player instead of going through the folders, but other than that it's not different from watching the proxies in the finder. The project I'm working on now is just small enough so I can render some 1/8 resolution h.264 quicktimes for the director to watch in the evening. Anything bigger would be too much on the processor, I agree.

And I hear you on the assisting of the camera dept. I wasn't born to sit behind a laptop all day!

Link to comment
Share on other sites

  • Premium Member

I've probably done more of this as it pertains to the Red than anyone else and for longer than anyone else (since Oct 07).

 

While the reality is you'd love to run everything through R3D data manager with full bit by bit comparison it typically isn't viable. I had one job where we were shooting with two cameras (3D) and Offhollywood refused to provide drives just provided 8x 16GB Cards. I had brought 6 cards of my own. If I hadn't we never would have made the day with just doing checksums, much less a bit by bit compare.

 

Forget the Proxies. Depending on the Red firmware build you can have corrupt proxies and all the footage will be fine. Also RedCine-X isn't the fastest program in the world. If you want a visual verification just open it in RedAlert, do a quick drag through the shot (don't try to play it). You'll notice any issues much faster than with RCX. Also depending on the camera firmware some files will only play in certain versions of RC/RCX whereas RedAlert has been able to handle just about all of them.

 

Process

 

Do a Visual verification that the files on the media (drive or cards) matches whats on the camera report from the 2nd AC. If there is a number discrepancy you should note it as a potential problem.

 

Copy the files to one external drive

 

Do a checksum

 

Copy the fils to a different drive

 

Do a checksum

 

Do random spot checks of files as well as a visual directory verification.

 

Print out the R3D data manager report as it pertains to the copy!!!! (even if you just print -> make pdf)

 

If you feel there is a chance for a bad file or a bad drive etc do a more complete verification of the contents.

 

Personally if I can I try to copy everything to a third drive that belongs to me. The reason being I've had too many times where someone else in the chain of custody did something stupid and it saved me to say "yes I have a copy".

 

Make sure you have enough drives, and they are a sufficient type for this work. Don't let production give you and excuse and by the cheapest drives. By RAID drives from an established brand. I had one job where we literally ran out of space in the middle of a shooting day. The only thing that saved me was for the four days prior I kept telling production that due to a change they made in the chain of custody I needed more drives. It also meant that making proper backups wasn't happening and all sorts of other issues occurred. Because I voiced my needs consistently and loudly when the figure pointing started I was covered. You need to be clear as to the requirements you give them.

 

As for LTOs. If your on a larger job (especially earlier on) the bonding company was requiring LTO tapes. The problem with LTOs is that they are expensive and time consuming to make. There are different formats that are not compatible with other drives.

 

My personal preference is to have a RAID 5 or RAID 0+1 system as your primary backup with a 2 drive RAID as the "off site" backup. At the end of each day (or sometimes more) the off site drive is taken to another location (sometimes to the post house) where it is either copied again onto a larger system or stored for archival purposes. This is just in case something happens on set to the other drive. The other advantage of a RAID 0+1 or 5 is that technically there are two copies of everything included in just that drive enclosure. So if a drive fails (and I've had it happen) you can rebuild the array and not lose any data.

 

A few more concerns. On some MBPs when using eSATA cards, they can crash your computer. Most drivers for the eSATA cards are poorly written. Test your card out throughly before using it.

 

~Marque

Link to comment
Share on other sites

  • Premium Member

Check out:

 

http://www.md5summer.org/

 

for checksums.

 

LTO has two generations of backward compatibility. So, LTO-2 and LTO-3 tapes can be read in an LTO-4 drive. The new LTO-5's, just about ready, will read the 3's an 4's.

 

LTO is too slow to be practical on the set. In town, the practice is to go to LTO at the post facility. On distant location, it might make sense to put together a little hotel room rig and make LTO's there.

 

 

 

 

 

-- J.S.

Link to comment
Share on other sites

  • Premium Member

I have a custom program I use for performing the checksums.

 

Check out:

 

http://www.md5summer.org/

 

for checksums.

 

LTO has two generations of backward compatibility. So, LTO-2 and LTO-3 tapes can be read in an LTO-4 drive. The new LTO-5's, just about ready, will read the 3's an 4's.

 

LTO is too slow to be practical on the set. In town, the practice is to go to LTO at the post facility. On distant location, it might make sense to put together a little hotel room rig and make LTO's there.

 

-- J.S.

 

Theoretically this is totally correct. The problem is that I've had too many instances where this simply wasn't the case and the problems go beyond generation compatibility. Some brands of drives don't like certain tapes, even when they should be compatible.

 

~Marque

Link to comment
Share on other sites

  • Premium Member
I have a custom program I use for performing the checksums.

 

Will those checksums be checkable in the future? Our theory is to generate a checksum as early in the process as possible, and retain it for the life of the file. That means years from now, if/when it gets pulled from the archive, there needs to be a way to verify the data by using the original checksum.

 

As for what plays on what, yeah, interchange is a bitch in many technologies, LTO included. Problem is, when they put a gun to your head and say, "Pick a digital archive format", there's pretty much nothing else to say but LTO -- especially now that holographic is out.

 

 

 

 

-- J.S.

Link to comment
Share on other sites

  • Premium Member
Will those checksums be checkable in the future? Our theory is to generate a checksum as early in the process as possible, and retain it for the life of the file. That means years from now, if/when it gets pulled from the archive, there needs to be a way to verify the data by using the original checksum.

 

As for what plays on what, yeah, interchange is a bitch in many technologies, LTO included. Problem is, when they put a gun to your head and say, "Pick a digital archive format", there's pretty much nothing else to say but LTO -- especially now that holographic is out.

 

-- J.S.

 

I print out (usually print to PDF) the log from all the checksums performed after each one and place the PDF file on the drive. This way there is a record of checksum and the results. Sometimes I'll print out a hard copy (paper) if production insists on it. As for LTOs I'm not saying don't use them, but production needs to understand the resources required to make LTO tapes, especially if your doing it on set.

 

Also consider that there maybe times when not only are you doing transfer but there maybe and assist editor on set doing conversions/dailies, etc. And thats without going into the DIT. Please remember the DIT does a lot more than just color correct on set. By having the DIT on set it should actually allow you to avoid doing as much color correction as the DIT should be changing the camera settings as needed.

 

~Marque

Link to comment
Share on other sites

  • Premium Member

Theoretically, yes. But I haven't seen it save much time on single camera shows. You're still going to look at every shot in final timing and spin the track balls -- just not as far. Sitcoms evolved from the live tradition, they get the cameras to match for the line cut, so in that context it works fine.

 

The hurdles to on-set color correction in single camera are that you need a good monitor properly set up, you need a dark environment with D65 illumination and a neutral surround, and the DP has to take the time to get dark adapted instead of just dashing in and out of the tent. I've seen that not work right a whole lot of times.

 

DP's, your job is a lot safer if you have a dailies colorist at the post house who's got your back. That colorist works for you, just like your gaffer and operators do. Shoot tests, time them with the dailies colorist, and communicate the look you want. They'll save some reference frames and give you a consistent look every day.

 

 

 

 

-- J.S.

Link to comment
Share on other sites

Apple are offering a new Mac Book pro with i7 processor and Mac OS X Snow Leopard. Might there be any compatibility issues with the new operating system and red cine, red alert re rocket, r3d data manager or any other red related software/hardware needed for digital downloading?

 

Frankie

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...