Scanning Question

[AJ Young]

https://support.apple.com/downloads/Apple_ProRes_QuickTime_Decoder_1_0_for_Windows

 

I've tried that one; it unfortunately only allows one to view a ProRes file in Quicktime, but not use the codec in DaVinci or any editing program on Windows.

[Tyler Purcell]

Brad Hunt of Kodak did tests in 1990 to determine the optimal resolution for their Cineon digital intermediate system and looking at MTF charts, they determined that there was very little improvement in resolution once the scan hit 4K, in fact, they only saw a 3% improvement between a 3K scan and a 4K scan, hence why they picked 4K for their system.

I'd love to do a 35mm resolution test with modern stocks. Do you think anyone has done it?

[Tyler Purcell]

I've tried that one; it unfortunately only allows one to view a ProRes file in Quicktime, but not use the codec in DaVinci or any editing program on Windows.

That's odd because DaVinci uses the quicktime formats to work with and my mac has nothing special to make them work. Hit up blackmagic's site, download the manual and do a search for "pro res", I'm certain you'll find the answer.

[Tyler Purcell]

So... looks like the 2k Spirit scan does not accommodate Ultra 16mm. Is that correct?

Yea, this is the problem with Ultra 16mm, it's kinda of a pain to find someone who can do the scanning.

 

Give the guys at Pro 8 a call and see what they say.

 

I'm sure Perry knows as well. ;)

[AJ Young]

That's odd because DaVinci uses the quicktime formats to work with and my mac has nothing special to make them work. Hit up blackmagic's site, download the manual and do a search for "pro res", I'm certain you'll find the answer.

 

I did look there; from what I could find it's not a DaVinci issue, but Windows. ProRes will work like a normal codec on a Mac OS, but Apple doesn't supply a version for Windows that works the same way. (It'll work inside Quicktime, but not for other programs). I could only find independent software that supplies ProRes codec that behaves like other codecs for Windows, but they're all at a price.

 

That's a shame, though! I was excited when you mentioned a ProRes codec for free on Windows. At least DNxHD works for now! :)

[Tyler Purcell]

I'm kina shocked DNX works because that's absolutely not a freebee. It must be included in DaVinci.

[AJ Young]

I'm kina shocked DNX works because that's absolutely not a freebee. It must be included in DaVinci.

 

I believe it comes natively with DaVinci on Windows as an export option. Avid does provide a C++ library for free; if one is savvy enough with compiling code, they could easily add the codec to their system. Additionally, there are various version of the codec floating around the web for free; most are up to date.

[Perry Paolantonio]

Hi Perry,

 

Does your scanner and Metropost's Director accommodate Ultra 16mm?

 

Yes. Both will do it, because both scan the film from edge to edge. The gate is larger than the physical width of the film so there's no issue with formats like this.

 

-perry

[Perry Paolantonio]

If you use a SMPTE resolution chart on a film camera, you will be able to understand film's actual available resolution. Sure, the chart is more line based then pixel count, but we can extrapolate pixel count based on the lines we see in the scanned image.

 

This is telling you how much detail from the real world the film is able to resolve. Grain size, optics, camera transport all come into play here, but basically, it's saying that X film stock can resolve Y lines of detail. And that's important information.

...but: The film scanner is not making the initial image, the film camera is. The scanner's job is to make as faithful a digital copy of the FILM as possible (not so much the picture on the film, but the physical film itself). The more faithful a reproduction of the FILM the scanner is able to make, the better the end result. If the scanner is only able to make an image of a given grain of film using say, 1-2 pixels, that grain of film isn't going to have the same detail in the scan as as scan made using a sensor that's using 5-6 pixels to render that same grain.

So you're talking about the film's resolution here, and that's different than how much detail the scanner is able to capture of what's on the film.

 

 

 

First and this is a huge misconception, most television is 1080i/720p and we are damn lucky to have those resolutions. Within the current standards, it will be impossible to update to 2k, let alone 4k. Plus, there isn't a single true 4k consumer-grande television on the market.

 

Second, I recently did research for another thread and found out, only 40% of all movie theaters in the world have 4k projection. That means, most theaters are 1920x1080 or 2k and the vast majority of big hollywood theatrical films are finished AND distributed in 2k.

 

Third, streaming services which claim to be 4k like Netflix, Youtube and Vimeo, are a complete failure because to get that resolution through the already extremely limited bandwidth available to consumers, they have to compress the ever-living snot out of the images, making them very soft, negating any of that added resolution.

 

You're talking about now. I'm not. 10 years ago I worked for a company that authored DVDs. The owner of that company insisted that watching video over the internet could never happen because (at the time) the files were too big. He couldn't imagine how you'd move an HD file over the internet. That's because he ignored the facts that:

 

A) Bandwidth constantly gets wider (I'm talking about internet connection speeds here)

 

B) Compression algorithms continually get better (H265 and others, for example), allowing more data in the same size file, or smaller files, depending on your needs

 

C) Processing speeds continually increase (related to B, allowing better decompression at cheaper costs. Moore's Law).

 

D) Storage costs continually drop and storage systems continually get faster.

 

2 years later, the company went out of business. And here we are 10 years after he said that. DVD and Blu-ray, long my company's bread and butter, are nearly dead at this point. The majority of the work we're doing in that area now is VOD file prep for all the streaming services. We've had the same case of blank DVDs sitting on the shelf since 2013 (we used to go through a case every 2 months on average).

 

 

However, most people shooting S16 aren't doing big theatrical runs and the cost difference between 2k and 4k workflow is astronomical, for resolution they will never see outside of the lab where it was scanned.

 

 

 

A scanner that could do 4k 10 years ago approached a million dollars. It was a massive expense, it was hugely expensive to maintain and operate, in some cases it required special electrical lines and cooling systems and rooms to house it. 4k was expensive because the hard costs of getting into the business were so high. That hasn't been the case for several years.

 

Our 4k rate is about 30% higher than our 2k rate. We can do this because our cost of entry into the business was lower and our overhead is lower. We use a Lasergraphics ScanStation for small gauge and a Northlight 6k scanner for 2/3/4 perf 35mm. We're not alone in offering lower cost scans, there are plenty of places out there that are doing this. So again, the idea that it's astronomical to scan 4k is no longer accurate if you look around. I can't even being to tell you how much 4k we scan for 8mm home movies for people who want to make digital archives of their shrinking and aging films.

 

P.S. I do plan on finishing my upcoming S16 film in 4k using the Blackmagic real-time scanner. This is simply because we will be doing theatrical and most of the film will be low-grain 50 asa stock, so you will see more detail in the image. So yea, you can call me a hypocrite! LOL :)

 

 

 

I don't think you're a hypocrite. I think you're making my point.

 

Though, who knows if that scanner will ever actually be released (it's coming up on 18 months since it was first announced with no sign of it). Also, when I saw it at NAB 2014 it couldn't do 4k 16mm, only 35mm. 16mm is an add-on (again, who knows if that'll ship with the scanner) and at least at the time, it used a window at the center of the sensor, rather than physically moving the sensor back to capture the whole frame using all the available pixels. In fact, it couldn't do true 4k at all, because it's a UHD scanner, limited to a 16:9 at 3.8k.

 

-perry

Edited August 26, 2015 by Perry Paolantonio

[Perry Paolantonio]

That's odd because DaVinci uses the quicktime formats to work with and my mac has nothing special to make them work. Hit up blackmagic's site, download the manual and do a search for "pro res", I'm certain you'll find the answer.

Resolve on Windows doesn't support ProRes. It's a licensing issue with Apple. You can read and play the files in Resolve, but you can't render to it.

 

Our workflow has been to render out to an uncompressed format (DPX, 10bit Quicktime, etc), then use ffmpeg and AVISynth scripts to make ProRes files. We've had 100% compatibility with the files using this method and the quality is great. But it's a bear to pull together a workflow and a lot depends on your setup.

 

There are a couple applications out there that can render ProRes files on Windows, though, ranging from very inexpensive to several hundred dollars.

 

-perry

[Tyler Purcell]

So you're talking about the film's resolution here, and that's different than how much detail the scanner is able to capture of what's on the film.

That's correct and in my eyes, there is no reason to scan at a rate higher then the physical resolution. To me its no different then capturing a Hi-8 tape a 1920x1080. The format is only capable of delivering 420 lines of resolution.

 

A) Bandwidth constantly gets wider (I'm talking about internet connection speeds here)

Bandwidth costs money and the technology existed 10 years ago to make the same network speed we have today, only not enough people were willing to pay for that faster service. As people started paying, the price for the service could be reduced and now we have faster speeds because the cost has been amortized across many people. Mind you, the average bandwidth is still 20mbps, which is slow. As a reference, I've had 100mbps to my house for 13 years. I don't see any substantial increase in average bandwidth with current technology. Verizon is the only company using fiber to the house and it has massive network latency issues. So sure it's fast, but accessing data is slow, because they're overwhelmed with customers.

 

B) Compression algorithms continually get better (H265 and others, for example), allowing more data in the same size file, or smaller files, depending on your needs

.h265 is an interesting standard because it's the first consumer-level streaming platform which can provide 4:4:4 color space at supposedly 8k max resolution. However, this is all just engineering speak at the moment. When products start hitting the market, we should reanalyze the final format and I have a feeling it's far less spectacular then the white papers.

 

Processing speeds continually increase (related to B, allowing better decompression at cheaper costs. Moore's Law).

It's true, if anything, we're getting some super fast/small/cheap processors hitting the market. Just look at the iPhone and how much it's capable of doing in such a small package.

 

D) Storage costs continually drop and storage systems continually get faster.

Yes and there is new storage technology on the horizon which will double current drive sizes and keep the price similar. However, it's not a HUGE savings and it will be a while before these new drives are standard on any modern electronics. So yes, this is a future solution, it's going to take a while to trickle down.

 

2 years later, the company went out of business. And here we are 10 years after he said that. DVD and Blu-ray, long my company's bread and butter, are nearly dead at this point. The majority of the work we're doing in that area now is VOD file prep for all the streaming services. We've had the same case of blank DVDs sitting on the shelf since 2013 (we used to go through a case every 2 months on average).

Yep, the duplication house I used in Chicago went under as well. It's a real shame because as a filmmaker, you have a lot more control over a disk asset then VOD. This is part of the reason why I think 4k BluRay is already dead in the water and why internet speeds are the leading factor for future high resolution streaming services.

 

A scanner that could do 4k 10 years ago approached a million dollars...Our 4k rate is about 30% higher than our 2k rate. We can do this because our cost of entry into the business was lower and our overhead is lower.

Yep and scanner technology will continue going down in price, that's a given.

 

Though, who knows if that scanner will ever actually be released (it's coming up on 18 months since it was first announced with no sign of it). Also, when I saw it at NAB 2014 it couldn't do 4k 16mm, only 35mm. 16mm is an add-on (again, who knows if that'll ship with the scanner) and at least at the time, it used a window at the center of the sensor, rather than physically moving the sensor back to capture the whole frame using all the available pixels. In fact, it couldn't do true 4k at all, because it's a UHD scanner, limited to a 16:9 at 3.8k.

Yep, I'm talking with the Blackmagic guys about it today at an event, I'll report back what I hear. :)

[Carl Looper]

That's correct and in my eyes, there is no reason to scan at a rate higher then the physical resolution. To me its no different then capturing a Hi-8 tape a 1920x1080. The format is only capable of delivering 420 lines of resolution.

 

 

 

Yes, that's right, but with Hi-8 tape the resolution of the system is already quantised by the design of the Hi-8 system. We already know in advance it's limits because they are built into the system

 

In film we don't know what the resolution limit of the film actually is because it's not, in advance, quantising the light field otherwise falling on the film. We have to use other methods to characterise it's resolution. We already know there's a limit in terms of the physics of light, and this will act as a bound, but if we used that bound we'd end up assigning a huge resolution capability to film.

 

MTF charts are the trick we use.

 

But have a look at an MTF graph. Any one will do. Along the x-axis is the cycles/mm of the test chart and along the y-axis is the response of the film. As the cycles/mm increases, the response of the film decreases. But where is this mythical line defining the boundary of the film's resolution?

 

 

All we can see is that, as the cycles/mm of the test chart increases towards infinity, the response of the film descreases towards zero - towards mid-grey (zero contrast).

 

But in the graph we see it just arbitrarily ends at 30% response, or 10% response, or any other response level that the graph author cares to nominate. Is there some sort of abrupt catastrophe that occurs below 30%, or below 10% that the film manufacturer is hiding?

 

No, it's just that 30% or 10% is chosen as an arbitrary cutoff for the graphing of the curve. This cutoff has nothing to do with the film stock, or it's resolution.

 

All we know is that is that as the sharpness (cycles/mm) of the test chart is increased, the response of the film to that sharpness decreases.

 

Should we use 0% as the boundary?

 

No, because in theory the zero response only occurs at infinitely high cycles/mm. And secondly (because of this) we can't show what happens at zero - this is why the response axis (y-axis) stops at 1. If we continued further down all we'd get is just smaller and smaller non zero fractions of a response. The scale used is logarithmic: we can't ever reach zero - it is infinitely south of 1.

 

So given this state of affairs, how do we nominate the maximum number of pixels required to scan the film?

 

We can't.

 

C

[David Cunningham]

My point has been made her myself and repeated, but I'm repeating it again. This is all about perceived sharpness for me. That requires proper resolving of the grain.

 

Perry put it perfectly. We're not trying to fully resolve the image on the film but the film itself... The grains of the emulsion. This is very important the more grainy the stock. Even if you plan to remove the "noise" later, the better resolved that "noise" is the better the removal software does at removing it. But again, I have no desire to remove it. It's part of why I shoot 16 and 8 mm.

[Tyler Purcell]

and when you do experimental non-commercial products you watch at home on the original files, then it doesn't matter.

 

However, most people try to make commercial product and in that case, film grain is a nuisance for both theatrical and home video distribution due to compression and how it deals with noise/grain.

 

I personally can't stand grain, as I've been stuck with it for decades of shooting on film. I'd rather over-light something and reduce that grain to an almost infinitesimal level, then have a film fill of noise. I don't give any credit for people with noisy films. It takes a real talent to produce a grain-free image in-camera.

[Carl Looper]

Increasing the resolution of scans, decreases the grain in the scan, in a way that is not the same as grain cleaning by software means.

 

This is because the grain in a scan is not the native grain of the film. It is an "alias" of the native grain - or "pseudo-grain" if you like. It's otherwise called "grain aliasing". The better you expose the film (the more you reduce the grain in the film) the less grain aliasing in the scan there will be. But to eliminate grain aliasing completely you would have to eliminate grain in film completely.

 

So either by increasing the resolution of the scan, and/or by grain cleaning one will get closer to the much finer grain of film projected in a film projector.

 

It's not about resolving the native grain of the exposed/developed neg so much as counter-acting the grain aliasing (an increase in grain) that scanning otherwise produces. The more pixels you have in the scan the more the grain aliasing is denied a channel through which to find expression. The more you approach what a film looks like when projected.

 

C

Edited August 27, 2015 by Carl Looper

[Bill DiPietra]

I don't give any credit for people with noisy films.

 

That's a pretty narrow-minded view since most people who shoot those kinds of films do so to achieve a certain look for the given project.

[David Mullen ASC]

I like interesting images that create a mood while enhancing the story -- sometimes those images can be smooth as silk, sometimes they can look rough and grainy. They sometimes are saturated, sometimes desaturated, sometimes b&w. They are sometimes as wide as 2.7 : 1 or as narrow as 1.33 : 1. And they are sometimes lit naturally with motivated sources and other times lit theatrically with stylized dramatic lighting that isn't motivated by a source.

 

Of course, what I like as a cinephile and what I tend towards as a cinematographer have some variance, plus I have to adapt to the aesthetics of the director who hires me.

 

Where I might agree with Tyler is that I enjoy some sort of definite look that avoids the run-of-the-mill, so I like super fine-grained film photography, such as in "JFK" with a lot of 50D and 100T stock used for the normal scenes, or the 50D photography of "Pulp Fiction" or the 200T photography of "Shawshank Redemption".

 

But I also like the textured look of "Eyes Wide Shut" and some of Chris Doyle's grittier work for Wong Kar Wei like in "Fallen Angels" or "Happy Together". But as a DP, I also understand the practical reasons for shooting on 500T and just trying to get a decent quality-image out of it and getting the same grain level as most movies get.

[Tyler Purcell]

That's a pretty narrow-minded view since most people who shoot those kinds of films do so to achieve a certain look for the given project.

I believe there is a huge difference between purposely styling the format and being stuck with a character trait of that format.

 

I don't like grain because it's completely unnecessary to creating a good image and telling a story.

 

Crisp, super-fine grain film is just amazing and it's rarely seen in today's films. Everyone lights and shoots as if they're shooting digital, but for a few more minutes of setup, they can light for 200T and have way less grain. Yet, they just don't take that next step because they simply don't care enough. That's why I call them lazy filmmakers, because that decision is wrong, especially today when most films are blown up to IMAX size.

 

This is why on small-format, it's probably smart to shoot as fine of a grain stock as you can. This way, it will lead to a better transfer and stand out amongst the crowd.

[James Gordon]

Glad i could be a springboard for this saga. lol. Good lord i've never been less sure of a decision. Going with 2k however. This is a killer thread, if anyone wants to write me a check for being a catylyst for great knowledge, pm me.

Edited August 28, 2015 by James Gordon