Some DI issues

[Josh Silfen]

I just went through the color correction for my first DI, and there are still some issues with the whole workflow that I don't quite understand. Here's the workflow that we used for this film: shot on Super16; got best light video dailies on DVcam for the offline edit; used EDL to re-transfer selects on a Spirit 2K to HDcamSR 4:4:4 using a flat transfer to preserve all detail from original negative; conformed SR footage in Fire suite; color corrected conformed cut on da Vinci color corrector. That's where we are right now. The SR master will be used to make an HDcam copy for festival projection, and someday, if needed, will be used for a film out as well.

 

My first question is about the dailies stage. In hindsight, it would have been easier and cheaper in the long run to get the HDcamSR flat transfer of all the footage from the beginning. I was reluctant to do it that way for two reasons. One, I was afraid that the flat transfer dailies would look really bad and worry people, in addition to not having a good-looking offline copy which in this case was used for submission to festivals, acquiring additional funds, etc. It turns out that I was pretty unhappy with the way the dailies were timed anyway, so it probably wouldn't have been too big a deal to go that way. The second reason is that I was led to believe that in the process of re-transferring the footage from the negative, there would be more information to play with in the color correction stage than there would from any other single "one light" transfer, even the flat transfer that is designed to use as much info from the original negative as possible. But then the post house we finally went with (we didn't have one from the beginning - part of the problem) wanted to use the flat transfer for the whole thing anyway, so that wouldn't have made any difference either. Does this make sense? Is this really the best way to do it?

 

In the color correction process, some of the digital timing I wanted done involved changing a specific color or luminance range, but according to the colorist, this was made more difficult by the flat transfer. For instance, due to the lack of contrast it was hard or impossible to select bright red without selecting much of the skin tones, or to select just the brightest highlights without selecting a fair amount in the midrange as well. Contrast was added for viewing purposes, as well as for the HDcam projection copy through the use of a LUT supplied by Kodak. The da Vinci, however "looked at" the footage before it passed through the LUT, so it saw much less contrasty footage to work with. I asked why we couldn't use all the contrast we could get out of the negative, rather than using the LUT, as they do for commercials and projects that finish on video, but the colorist told me that then the final image would not be in film gamma and that the quality of the film out would suffer severely. I do not quite understand why this is, but is that true? I had always been led to believe that you could do anything in a DI that you could in a normal telecine session. Is that not the case? Does it possibly have to do with the fact that we were working from HDcamSR and not 2K data files? The colorist also said that the da Vinci is not really made for timing flat transfers and that the Lustre could be a better color correction system for DI work. Does anyone know if this is true?

 

Did we go the wrong route from the beginning, make a wrong turn somewhere along the way, or do everything right from the beginning? Any opinions or answers to any of these questions would be greatly appreciated.

 

-Josh Silfen

[David Cox]

hello,

 

I few comments in the order of your questions...

 

Dailies: When labs transfer one lights, they usually do so without any noise / grain reduction and to be honest, often on older telecines. Thats (sort of) fine for your cutting tapes, but you would want a better quality of transfer for the version you will be using to base your grade on. Also, a HDCAM SR recorder is very significantly more expensive than a DV recorder, so often price is a big concern for your dailies. In general, this part of your route sounds pretty standard and should have been the most efficient way.

 

Colour Picking: Its true that a file transfer (such as DPX or Cineon) are capable of containing more shades of colour than the tape formats. Having said that, for the purpose of colour picking, it should be a marginal difference. It is possible that the flat transfer you had was under-saturated. For a flat transfer to be correct for later grading, the aim is not to loose any data by clipping. In other words, nothing is darker than black, whiter than white or more colourful in any of the RGB channels than 100%. I have seen transfers that are very under saturated though because the original transfer house has been far to cautious and this would cause problems in the colour isolation stage. There is a tendancy to be over conservative with these "technical transfers". It is very common to see many digital bits of information simply wasted because they are being used to describe 100 shades of "black" (shades that will be made black in the final grade). There is an argument for setting a guide contrast range in this first transfer in order to make the best use of the digital bit depths, as well as transferring as colourful as possible before clipping.

 

There are some limitations of Da Vinci for digital work, because it is just a colour grader. We use a new system called Mistika which combines very powerful colour grading with full compositing tools. This way, when picking areas or colour ranges doesn't do it, we can get in there with a matte, tracking tools etc to get the desired result. These are not facilities available on traditional telecine grading suites. Also, unfortunately the Autodesk route is to sell different products for these stages (Lustre for colour correction, Smoke for conforming, Inferno for compositing). I would suggest though that Da Vinci should have been able to isolate distinct colour ranges given a decent transfer to begin with.

 

Are you able to post a frame of your original HD CAM SR transfer? Possibly one of your graded version too? I suspect the root of your problems is a overly flat, flat transfer?

 

David Cox

Baraka Post Production

www.baraka.co.uk

[Dan Goulder]

Josh, I'm curious how the "flat" HDCAM SR transfer looked to your eye. Did it confirm your belief that it wouldn't look very pleasing, or good enough for submission purposes? Also, I was surprised to read that your colorist felt the flat transfer actually limited what he was able to do. Did he recommend doing the transfers in a specific, non-flat way that would optimize his latitude in post? Please let us know. Thanks.

 

I wouldn't say you've gone gone about anything the wrong way. The only thing I might recommend to do differently in the future is something you've already considered, which is to transfer all footage to HDCAM SR, and then strike proxy files for construction of an EDL. It may all boil down to doing that transfer in the most optimal way. I too would be interested in seeing both an uncorrected and corrected screen grab. Also, if you're not uncomfortable in divulging names, I'd be curious to know which post house you used.

[Josh Silfen]

Josh, I'm curious how the "flat" HDCAM SR transfer looked to your eye. Did it confirm your belief that it wouldn't look very pleasing, or good enough for submission purposes? Also, I was surprised to read that your colorist felt the flat transfer actually limited what he was able to do. Did he recommend doing the transfers in a specific, non-flat way that would optimize his latitude in post? Please let us know. Thanks.

 

I wouldn't say you've gone gone about anything the wrong way. The only thing I might recommend to do differently in the future is something you've already considered, which is to transfer all footage to HDCAM SR, and then strike proxy files for construction of an EDL. It may all boil down to doing that transfer in the most optimal way. I too would be interested in seeing both an uncorrected and corrected screen grab. Also, if you're not uncomfortable in divulging names, I'd be curious to know which post house you used.

 

The flat transfer actually did not look as bad as I was expecting. It was certainly not what I'd want people judging when they watched dailies, but you do get a good feel for what's on the negative and for submission purposes I'm guessing it could be made to look pretty decent with some basic color correction in Final Cut Pro or other NLE. I don't think there was anything wrong with the transfer, especially because with the Kodak-calibrated LUT applied, the contrast looked just as I'd expect it to look. In fact, I thought if I went through this process again, a good workflow might be to get all the footage transferred flat to HDcamSR at the beginning, and at the same time to DVcam with simultaneous time code and the LUT applied for offline purposes. Then you'd have good looking dailies, know what's on the negative, and have the flat transfer SR tapes ready for online purposes later.

 

I don't mean to sound like I'm complaining about the post house (Shooters Post and Transfer in Philadelphia) or the colorist. I think they have done a tremendous job. We had four days of color correction, and we accomplished some pretty amazing things. They were really enthusiastic about the project from the beginning and were willing to work with us in terms of budget and schedule to get the whole thing done in a few weeks so it will be ready for Tribeca this month. I have nothing but good things to say about them.

 

I'm just trying to get a better handle on the technical aspects of digital post production that I don't really understand and see what kinds of workflows others have used. The colorist recommended the flat transfer because of the possibility of a film out later. If we knew we'd never be going back to film, we would not have done it that way, and probably could have gone a little further in the color correction. Basically, the way I understand it, the flat transfer approximates film gamma, so when it is recorded back to film it maintains all the detail from the original negative and looks appropriately contrasty. If you were to do a regular telecine-type timed transfer instead, the SR footage would have too much contrast and you would lose much of the shadow and highlight detail when recording back to film. Therefore, a DI would be inherently more limited than a normal telecine for video finish. This is what I don't really understand; the technical aspects of converting from film to digital contrast and color space and back again. Also, when film is scanned at 2K or 4K, isn't it still some kind of flat transfer to data files? There still isn't any color correction applied during the scanning stage, right? If that's the case, then even though this was done on SR tape, the color correction process should be pretty comparable to a 2K DI.

 

Unfortunately I don't have any HD stills at the moment. Maybe I'll be able to get some later. One other question: I've always overexposed by about a third of a stop to get a denser negative. Does the flat transfer eliminate the benefits of doing this?

 

-Josh Silfen

Edited April 4, 2006 by Josh Silfen

[David Cox]

The colorist recommended the flat transfer because of the possibility of a film out later. -Josh Silfen

 

I think there might be a misunderstanding at this point. You would maintain a flat grade from scanning to film out IF you had no intention of changing the colour during the digital processes. For example, if you were only adding digital special effects and not doing a digital colour grade.

 

Because your intention is to change the colour, you should grade the images to look as you would like them within the environment you are working in. For example, if you are working in a HD 4:4:4 environment such as with HD CAM SR, then you can make your images look right within that colour space. The look up tables then get applied "in reverse" at the point you print out to film within the film recorder, to convert from HD 4:4:4 colour space back to the chosen film stock.

 

Working this way, your digital master has the correct colour settings for the format it currently resides on, and therefore is a universal master for your later deliverables. Given that it seems your primary deliverable may not be film, this would be an effective and efficient process.

 

The purist approach differs slightly in that digital colour grading happens with monitors that have look up tables applied to show what would happen if that data was applied "as is" to a particular film stock. You then don't use any reverse look up tables within the film recorder, because your data has been graded to suit a given film stock. The benefit of this route is a better use of colour space for film and so a marginal decrease in potential colour banding. The down side is that you create a data master that strictly speaking is only suitable for one type of film stock. To use any other, or to have a digital master suitable for transfer to DVD etc, you would then at least need to apply look up tables to get back to the correct colour space, or more likely start grading again.

 

David Cox

Baraka Post Production

www.baraka.co.uk

[Josh Silfen]

I think there might be a misunderstanding at this point. You would maintain a flat grade from scanning to film out IF you had no intention of changing the colour during the digital processes. For example, if you were only adding digital special effects and not doing a digital colour grade.

 

Because your intention is to change the colour, you should grade the images to look as you would like them within the environment you are working in. For example, if you are working in a HD 4:4:4 environment such as with HD CAM SR, then you can make your images look right within that colour space. The look up tables then get applied "in reverse" at the point you print out to film within the film recorder, to convert from HD 4:4:4 colour space back to the chosen film stock.

 

Working this way, your digital master has the correct colour settings for the format it currently resides on, and therefore is a universal master for your later deliverables. Given that it seems your primary deliverable may not be film, this would be an effective and efficient process.

 

The purist approach differs slightly in that digital colour grading happens with monitors that have look up tables applied to show what would happen if that data was applied "as is" to a particular film stock. You then don't use any reverse look up tables within the film recorder, because your data has been graded to suit a given film stock. The benefit of this route is a better use of colour space for film and so a marginal decrease in potential colour banding. The down side is that you create a data master that strictly speaking is only suitable for one type of film stock. To use any other, or to have a digital master suitable for transfer to DVD etc, you would then at least need to apply look up tables to get back to the correct colour space, or more likely start grading again.

 

David Cox

Baraka Post Production

www.baraka.co.uk

 

I guess that answers my question. Either the post house got the impression that the eventual film-out is more likely than it actually is, or they just fall into the latter category of "purists" who think the flat transfer route is the best option if there is ANY possibility of a film-out. The plan for the HDcam projection copy and any DVD master is to apply the LUT to the graded flat film-out master. What is color banding, and are there any other potential problems or artifacts from the "reverse LUT" route?

 

-Josh Silfen

[David Cox]

Applying the look up tables to a master graded in a HD colour space to convert it to a given film stock during film recording generates very few problems. There is a mathematical possibility that if the grading choices in the digital master were miles away from the response curve of the film, that the result could be colour banding where there are not enough digital shades to cover a particularly responsive range. This is a mathematical issue though - I've not seen an example of it in 6 years of doing it!

 

Colour banding is where there aren't enough digital bits left to describe all the shades needed to make an image. As an extreme example, imagine a very blurred circle - it should be a single graduation of shades from centre to edge. If there aren't enough digital bits to describe the image, you get definite shades making up the image that can be seen as discrete bands of shades - hence the term "banding". In the above example, you would see obvious concentric bands, stepping down through the shades to make the blur.

 

This happens if there aren't enough bits to describe an image, or if the bits are used inappropriately. Theoretically this can be the case if you colour correct in one colour space, then apply look up tables to force that colour range into another colour space. For example, grading on a normally calibrated TV monitor and then pushing that onto an unusual film stock.

 

But as I said, I've never seen an example of banding caused for that reason. What is a more likely scenario is to have a film transferred "flat" to HD CAM SR (10 bits), that is simply not realistically contrasty. In other words, the blacks are very light and the whites needlessly dark. What happens now is that all the useful information is crammed into a few bits in the middle of the digital range. These are then expanded out during grading to restore contrast, meaning that the image is only made out of the few bits in the middle of the range. This is why I mentioned in a previous post that it is good practise to establish a contrast range in the original digital transfer that will be used as the basis of further digital grading.

 

Its more an issue if an HD tape format is used as the source for digital intermediate grading as it has only 10 bits of range. File formats have unlimited bit depths although common file formats use 16 bits and therefore have more "disposable" range. 10 bits are perfectly fine though for cinema images AFTER grading - i.e. when you have set your contrast range to make the most of every bit.

 

Certainly in your case I would have suggested the route you took as being the most efficient and technically perfectly sound. The only modification would be that the original transfer from film to HD CAM SR should have been set just shy of the maximum contrast you would have liked to see in your final master. At all points through post production, you should be able to see the image as you want it to be seen, no matter what the display device is.

[Michael Most]

Either the post house got the impression that the eventual film-out is more likely than it actually is, or they just fall into the latter category of "purists" who think the flat transfer route is the best option if there is ANY possibility of a film-out.

 

If they were "purists," they wouldn't have been working in linear, video gamma color space in the first place. They would be working in log space, and applying a properly derived, calibrated 3D LUT during color correction to both simulate a film print and provide for a pleasing, usable video version.

 

All of the talk about "flat grade" being "closer to film color space" is nonsense. Film color space is log, most commonly 10 bit log Cineon format, which was specifically designed to represent film print density. Anything done in video color space with video gamma is, by definition, not film color space and not even close to it. While you might be maintaining useful and valuable detail in a "flat" transfer, one should not make the mistake of thinking that this is going to yield the same result. What the "flat" image looks like is completely irrelevant. What the values in that image are is relevant, as it is intended to be manipulated by a color corrector. Given that your primary product here was video, what was done was a very acceptable post route. If your primary product was film, I would re-think that.

 

As for file formats having 16 bits, this is true if you're using linear file formats. However, in almost all cases, film scans are maintained in 10 bit log encoded format, usually either Cineon or DPX (a superset of Cineon). Encoding to 10 bits allows effective maintaining of an equivalent 14-16 bits of linear information, primarily because in a log format, more available levels are assigned to the area of the curve that usually has the most detail, namely the lower-mid range. Fewer levels are assigned to the brightest areas, where less detail is needed. The log file is fed directly to a film recorder, as it represents the proper values for creating a negative that will print correctly. It also allows for more efficient use of storage, faster rendering, and more efficient processing, allowing for more effective real time playback. Most DI color correction systems - with the notable exception of DaVinci's 2K Plus, which was designed as a video device and has been (mis)appropriated for DI use - can and do color correct in either log space or "linear," video gamma space, and can be set to either mode.

 

 

In hindsight, it would have been easier and cheaper in the long run to get the HDcamSR flat transfer of all the footage from the beginning.

 

Easier, maybe. Cheaper, most definitely not. You would be using an HD telecine, an HDCam SR recorder, and HDCam stock instead of a standard def telecine, a DVCam recorder, and DVCam stock. Given the same number of hours of transfer time, the latter is going to be an awful lot less expensive. And even given the necessity of doing a re-transfer for the final conform using the HD telecine and SR deck, the end result is still going to be far more economical using the DVCam offline route. If you want, I could give you some sample figures on this, but not on an open forum like this one. Let me know if you would want that.

[David Cox]

Its important to understand the reasons for all the different files formats and bit depths, otherwise producers run the risk of being lead down the wrong path leading to lower quality or unnecessary expense.

 

To get all the ?information? from a film, 16 bits of data are optimal allowing for 65,536 shades. However, when film first started to be digitised, 16 bit files created lots of data in days when disks when small and computers slow. So it was worked out that one could get close to 16 bit visual accuracy if only 10 bits were used, but the 1024 shades those bits related to were adjusted to be most concentrated in more responsive areas rather then evenly spread throughout the range. In other words, the loss of the 6 bits was ?hidden? in areas where the eye / film are not so responsive to subtle shade differences. This is the Log file format such as Cineon.

 

Where you are trying to make the digital process ?invisible?, for example when you are creating digital special effects but not colour grading, then this is when it is important to ideally keep 16 bits linear start to finish, or 10 bits log if you want to speed the process of file handling up. However, if you are intentionally changing the colour grade then the necessity of 16 bits or a log 10 bit file are reduced. The reason for this is that when you try to capture all the data from a film and keep it, you actually end up with a vast amount of data that describes film grain in the very low light areas. If during grading this is set to be black (as it most likely will be), you are actually ?saving? a number of bits worth of information. This is why a 10 bit linear file AFTER grading is almost indistinguishable from 10 bit log file when put back to film. What happens with the 10 bit log file is that the shades the bits represent are concentrated in certain areas in order to ?extend? the range of shades that are described. Once the black and white points are set, these extended areas are not really needed so the benefits of the log coding are somewhat reduced.

 

So the ideal route is still 16 bits or 10 bit log files throughout, but ?ideal? and ?budget? rarely make happy bedfellows! Files (as opposed to HD tape) increase post production costs because of the time it takes to handle them ? loading, saving, converting etc. They cause cost implications throughout the post because every time someone needs a tape or DVD copy (sound guys, distributors etc) conversions have to be made, LUTs applied, rather than just make a tape to tape copy. So when budget is a very big issue, the step down to using HD CAM SR as the source for digital post work can offer savings, whilst not visually impairing quality in most applications IF it is prepared properly. This preparation involves (as mentioned in earlier replies) making sure that the contrast range is set in the original telecine transfer to HD tape, so that all of the 10 bits carry visually useful information. If a low contrast ?flat? transfer is made, only the middle range of these bits are really carrying useful information and subtle shades often drawn upon in grading are lost.

 

In the context of this thread, the HD CAM SR route was a good choice because (I?m guessing) the budget was tight since there isn?t yet a distribution deal.

 

David Cox

Baraka Post Production

www.baraka.co.uk

[Stephen Williams]

My thanks to Mike Most and David Cox for the above information, its so clear and easy to understand.

 

Stephen

[Phil Rhodes]

Hi,

 

The other thing about being able to select given colour and luminance ranges is that something like a da Vinci isn't capable of modifying the ranges of which it works very much. On Lustre or Baselight (or After Effects, sorry Mike) you can tell it exactly what luminance range you want it to work in.

 

Then there's the fact that sometimes things that look very different are being picked out of your brain by some contextual clue that isn't obvious to the machine. Skin has a lot of red in it. Sometimes things that look like they should key cleanly just don't, and it's not always obvious why unless you consider how vector pickers work.

 

Phil

[Josh Silfen]

Thanks a lot for your informative answers. So, I guess in the future for a low-budget project, my preference will be to get more contrast and "useful" information out of the HD transfer, and then apply a LUT in reverse if/when a film out is ever needed.

-Josh Silfen