Jump to content

Recommended Posts

1,5TB isn't much when you're dealing with a whole film project in DPX, and the drives will slow as the fill, as mentioned.

 

Perfect example, on a feature I shot with a black magic-- primarily a 1 camera show with a B camera on occasion we were pulling between 500GB and 1TB a day in dng raw, 2.5K

Link to post
Share on other sites
  • Replies 54
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Sorry... another post... lastly note on the 4x3 frame...   Yes, if you request "pillar boxing" they should deliver a 1920x1080 frame with pillars on the sides.   If you request a 4x3 2K frame, you

It's true that 2k doesn't make a huge difference for 16x9. But, in 4x3 it's significant, especially if you plan to reframe/crop/ or zoom to 16x9. You get 2048x1556 with 4x3 2k. That's a big difference

The 2K scan scans each film frame as an individual DPX file which forms an image sequence. Each frame (for 2K) is around 12 MB, so it adds up... The amount of information captured and stored in a 2K s

Hi Jeremy,

 

RAIDs work by splitting the data up over many drives and writing to them in parallel. The way this works is different depending on the RAID configuration you have - RAID 0 is generally the fastest, but is also the most volatile. If one of the drives in the RAID fails then all of the data on the RAID is lost. You're better off going with a RAID level like 5 or 6, which allow for any one or two drives to fail, while not losing any data. This buys you time to replace the bad drive and rebuild the array. In the mean time, performance will be degraded, but the data is safe. This is what we have in the PC that controls our scanner.

 

When we first got the scanner, the enclosure for the PC didn't have enough space for 8 drives, so we had to run a 6-drive RAID 0. We could do 30fps 2k DPX (10 bit) scanning on this for a few seconds at a time, and performance degraded significantly as the drive filled up. When the enclosure was upgraded and more spaces were available for additional drives, it was bumped to 8, and a RAID 5 for redundancy. Now we can easily scan to 2k DPX files at 30fps all day long.

 

For what it's worth, we're using cheap WD Green 3TB drives in this RAID. You could get a little more performance out of enterprise drives, but you'll pay a fortune for them. If you're doing an 8-disk RAID, you're not going to see a substantial speed improvement with them (there will be some, but it's not worth the extra cost, in my opinion).

 

With RAID, more drives is always better. Also - the controller matters a lot. We're using an inexpensive HighPoint RocketRaid PCIe card, and it's fine for this purpose. You don't want to rely on the system to do the RAID in software (Mac and Windows can both do this in their disk formatting tools), because that puts load on the CPU and performance is much lower.

 

All that said - if your scanner isn't going to need to work at real time or faster, you can probably get away with a smaller RAID. Just don't expect to be able to play those files back at real time.

 

-perry

Link to post
Share on other sites

We have two grading systems at Cinelab, one has an 8-Drive 16Tb disk array RAID-5 and the other has a 12-Drive 24Tb disk array with 2Tb enterprise drives, both systems can work with 3K DPX and even 4K DPX Disk I/O is around 1000mb/s for the 8-Disk and over 1200mb/s for the 12-Disk and that is over the whole array even when more than 3/4 full. We are using both RocketRaid and ATTo cards, I recommend the ATTOs they are faster.

 

-Rob-

Link to post
Share on other sites

We are using both RocketRaid and ATTo cards, I recommend the ATTOs they are faster.

 

 

ATTO has always made nice hardware.

 

We get about 1100 MB/s on the 8-drive RAID with the Highpoint RocketRaid card in our scanner host PC and generic WD Green drives. Plenty for 2k and probably higher. In our SAN and in one of our Blu-ray encoding machines we've got Areca and 3Com cards respectively, which are awesome, but much more expensive. The SAN has 16 drives in it with the Areca card, and in testing we got a steady 1.5GB/second throughput, in a RAID 6 configuration. The 3Com is similar, but with an 8-drive setup.

 

Both of those machines have desktop drives in them - Seagates or WD - but nothing fancy.

 

-perry

Link to post
Share on other sites

 

ATTO has always made nice hardware.

 

We get about 1100 MB/s on the 8-drive RAID with the Highpoint RocketRaid card in our scanner host PC and generic WD Green drives. Plenty for 2k and probably higher. In our SAN and in one of our Blu-ray encoding machines we've got Areca and 3Com cards respectively, which are awesome, but much more expensive. The SAN has 16 drives in it with the Areca card, and in testing we got a steady 1.5GB/second throughput, in a RAID 6 configuration. The 3Com is similar, but with an 8-drive setup.

 

Both of those machines have desktop drives in them - Seagates or WD - but nothing fancy.

 

-perry

 

It seems like most of the attention here is on drive speed/connection. How much of the ability to handle these files is determined by drive speed, and how much by other factors of the computer like CPU, RAM and GPU? That is, can a more modest machine handle this footage assuming a fast RAID and connection is used? Or, at what point do the other hardware become limiting factors?

Link to post
Share on other sites

 

It seems like most of the attention here is on drive speed/connection. How much of the ability to handle these files is determined by drive speed, and how much by other factors of the computer like CPU, RAM and GPU? That is, can a more modest machine handle this footage assuming a fast RAID and connection is used? Or, at what point do the other hardware become limiting factors?

 

Depends on what you mean by "handling" I think. If you just want to play it back, you can do it even on older machines - Our old MTI Correct system was built in 2004. It was a Windows XP (32bit) PC with a couple of Xeon processors in it. It could handle 2k DPX playback at realtime speeds. But that's just playback.

 

When Iridas had SpeedGrade (Pre-Adobe), they had a little application that could play DPX sequences in real time. This was heavily GPU dependent, but it still required high speed disks. It didn't require a hugely fast computer, but it was just for playback, which is relatively simple to do.

 

If you're talking about multitrack editing, realtime effects, cropping, scaling, text overlays, etc - the requirements would be different than for simple playback. You'd still need the same disk array underneath (possibly with even more speed if you're doing multi-track timeline editing), CPU speed probably comes more into play at this point, and certainly GPU, since most modern software is GPU optimized.

 

It's hard to say though, without knowing the applications in question and the things you want to do. The requirements will vary.

 

-perry

Link to post
Share on other sites

. I've been working with smaller scale image sequence files, uncompressed jpegs which have been very user friendly.

 

 

Jpegs are by definition compressed. Lossy compression too.

 

Freya

Link to post
Share on other sites

The nice thing about a ATTO or RocketRaid is that those cards do all of the processing for the raid and the CPU and GPU are left for image manipulation. DPX is uncompressed so it is more on the drive array than CPU. 2K DPX in Resolve (for example) or Adobe CS will run pretty well on an older Mac-Pro or i7 Win-7 machine. Having at least one fast nVidia CUDA GPU like a GTX480, GTX580 etc will help allot with Resolve.

 

-Rob-

Link to post
Share on other sites

 

Jpegs are by definition compressed. Lossy compression too.

 

Freya

I went ahead and loaded an uncompressed TIFF sequence into my NLE from a 720P scan of S8, just to see how it would play in my timeline. It plays continuous but slow at 1.6MB per frame without any grading or editing. It's a good place to start if you're doing some or all of your own grading, where real time playback isn't necessary. Then render your video file for real time editing.

Link to post
Share on other sites
  • 2 weeks later...

2k scan all the way! If you are concerned with grain I would recommend sending to Process Blue in Connecticut for scan and noise reduction. I sent some push processed Fuji 500D there for grain reduction and was very pleased with the results. They can give you ProRes files so that's handy and I happy with the amount of info in the scan. I chose RAW color vs. best light and was able to do so much in Resolve!

 

I saw some other folks recommended Film and Video Solutions in MD. Is that the same place that used to go by Movielab? If so, they did a great job with processing and I'd be interested in having film scanned there as well.

  • Upvote 1
Link to post
Share on other sites

I saw some other folks recommended Film and Video Solutions in MD. Is that the same place that used to go by Movielab? If so, they did a great job with processing and I'd be interested in having film scanned there as well.

Yes, Movielab is now Video & Film Solutions. They do great scans at a reasonable rate.

Link to post
Share on other sites
  • 3 weeks later...

Hi guys!

 

Sorry if this might have been brought up or mentioned, but I was wondering, has anyone done any Super 8 scanning at 4K? OR knows where it could be done by any chance???

 

I know that is definitely overkill...but I'm after this because I am doing an investigation for my Final Major Project at university.

 

I hope this can be possible...I am having an incredibly hard time doing this! :(

Link to post
Share on other sites

I am not aware of any 4k super 8 services. However, some testing has been done with frame-by-frame "scanning" with 18-20 megapixel DSLR gear. The results are about "6k" raw down sampled to 4k. Search for Justin Cary on Vimeo. He's got a good 4k setup and may be able to help you.

Link to post
Share on other sites

Oh. And about all you get is finer resolved grain. There isn't really any more real resolution to be gained. However, by sampling at 18 or even 20 megapixel and then down sampling to HD you do get about the sharpest and most detailed super 8 possible. It's law of diminishing returns though. I've found that 2k is about all that's useful in super 8.

Link to post
Share on other sites

That is exactly what I am intending to do but from 4K, and the film stocks I will be shooting at will be the Vision3 50D and 500T.

 

I have also seen the DSLR set-ups, especially footage done by Justin Cary, all of which look very good, however they are not considered 'professional' and I would much prefer using industry standard scanners (since they scan in RGB) since 35mm is treated this way :D

 

My closest is Kinetta: www.kinetta.com

 

I have been in contact with them and I think they can do it, have you ever heard of them? or anyone here? Is their scanner as good as they claim?

 

A quick question, does anyone know a telecine house that might have a Spirit Datacine with a Super8 gate perhaps??? (I'm aware that I-dallies in London has one, wanting to know for anywhere else).

Link to post
Share on other sites

If your goal is to scan Super-8mm to 4K on a "real" scanner you are out of luck, a machine like a Spirit can do 2K or 1080 from Super8mm but no chance of 4K. Likewise an Arriscan or Scanity Northlight, etc. have no possibility to scan Super8mm at all because the whole film transport was not designed for the smaller gauge. The Kinetta scanner can do a 3K scan and has been used on "big" projects like "Our Nixon" and 3K is about as much resolution as can be lensed onto a sensor with the optics practically available.

 

Remember that Super-8mm is basically a 1.33:1 format and a 3K scan will fit into a 4K TV or Theatrical image with black left and right.

 

There is no practical point to building a machine which scans Super8 to 4k.

Link to post
Share on other sites

Hi guys!

 

Sorry if this might have been brought up or mentioned, but I was wondering, has anyone done any Super 8 scanning at 4K? OR knows where it could be done by any chance???

 

I know that is definitely overkill...but I'm after this because I am doing an investigation for my Final Major Project at university.

 

I hope this can be possible...I am having an incredibly hard time doing this! :(

I believe the super 8 footage in the recent "Nixon" documentary was scanned at 4K with the Kinetta

Link to post
Share on other sites

Wouldn't a 4k scan of Super 8 be the equivalent of 8k on 16mm or 16k on 35mm? Unless the scanner has optics that blow up the image to fill the sensor...but even then it's kind of like doing an optical blow up to 35mm then scanning that? (albeit without a film generation loss)

 

In other words, to get an 8mm strip of film scanned to 4k you'd need a much higher pixel density than scanning a 35mm image right?

 

I'm sure I'm missing something basic...

Link to post
Share on other sites

When scanning a 4x3 Super 8 frame at 2K you are getting a frame heigh of 1440 pixels. The frame size is 4.01mm. Let's round down to 4mm. That means you are scanning that frame at 360 dpmm (dots per mm) or 9144 DPI. If you scan it at 4K, at 4x3 a 4K standard heigh would be 2160... so not double 2K. But still, your dpmm here is 540 dpmm or 13716 DPI.

 

Assuming a 4x3 Super35 4-perf frame of 18.66mm... a 2K scan gives you only 77 dpmm or 1960 DPI. In a 2160 4K scan that would jump to 115 dpmm or 2940 DPI.

 

Crazy numbers when you think about it.

Link to post
Share on other sites

However... my last note on this is that good machines (such as Millenium at Pro8mm, Spirit at Spectra and ScanStation at Gamma Ray Digital) do use high MTF optics to zoom in on the frame to make full use of the sensor image area.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.




×
×
  • Create New...