Larry Baum

Anyway I need to wait to get another set of files to really tell what is going on. It's hard with potentially bogus metadata and then not being 100% sure exactly what was done and set for each scan.

Some things are still not adding up. And the 5K early samples still ended up with relatively similar color balance in the main image area so you wouldn't think anything should have been so radically different in saturation, etc. (sprocket holes have extremely different color temp, optical track area has only moderately different color temp, might just be sprocket clipping after adjustments applied for the way this system can expose each channel on its own, so may not mean so much and would make you think that the 5K set should not act so differently than the other sets)

Man, I'm starting to think that the initial little batch of a few quick 5K demos I got, that something was set all strangely with them which added a lot to the general confusion along with the seemingly generally bogus metadata in the files. I found another 5K little set that seems to act more like my 6.5K final scans.

Who knows at this point after all the confusion, but I suspect it may all come down in the end to:

5K and 6.5K and so on probably will all act the same (which would make more sense)

Scanstation probably does scan out data in some moderately wider than REC709 gamut format, perhaps something a trace smaller than P3 (still not 100% sure though and you'd think the true RAW data might be even wider than that, not that wider than that would be needed for film)

metadata put out by the scanstation is almost all completely bogus and best ignored, but this also means you need to do some complex type corrections, adjusting primary locations and such (still not 100% sure, but seems likely)

I'm starting to think those initial few quick 5K samples I got maybe had the LED balance on the scanner set in some odd way. Either that or on the new 6.5K files.

The thing that makes it tricky is the darn metadata claims same As Shot WB and scene and LED temp for both! But looking into things like colors in sprocket holes that does not actually seem to be the case.

So it seems the metadata is likely not reliable at all (which isn't a great thing). And I'm not sure that things actually were scanned using same settings as I thought, so at this point it's hard to know exactly what is going on beyond that the metadata appears likely a false mess for the outputs from Scanstation and that things might actually have been scanned in different ways each time contrary to what I thought, the quick demo frames and bits and the real scans at full res. Anyway, I need to get more info at this point to figure out what the real deal regarding anything going on is. All I can say is the early 5k samples I got behave totally differently (actually closer to how the metadata says they should, but that would be weird if it was supposed to be like that since native scanner data should not be REC709-gamut-like, so maybe something was set weird and it was just a coincidence that it worked closer to the metadata for gamut and the metadata is set wrong) than all new 6.5K samples and scans which all act virtually identically and that the metadata does not let programs give the proper colors for either set, especially not for the 6.5K stuff although doing a lot of manual adjusting of primaries and so on you can get them all looking correct.

On 1/19/2023 at 9:11 AM, Perry Paolantonio said:

I have avoided this thread because, quite frankly, I think you're looking for a problem that doesn't exist and you have yet to post a single image demonstrating the supposed problem.

But I need to correct this:

The ScanStation's resolution modes have nothing to do with color. They're unrelated. The "modes" in a 6.5k scanstation refer only to the pixel dimensions (and subsequently the speed at which the scanner runs, because lower resolutions mean less data).

Inside the scanner, the camera and lens move closer to or farther away from the film, depending on the mode. In a lower res mode, the only difference from a high res mode is that instead of using the entire sensor it's using a crop, and can run at a faster speed. That is, in 2.5k mode, it's using a 2.5k crop of the 6.5k sensor and is capable of running at up to 60fps. In 6.5k mode, the entire sensor is used because the camera and lens are in a different position, filling the sensor with the image of the film, and this runs at a lower speed because - more data. 

The internal processing of the data, the output file formats, all of that is identical regardless of the resolution. The only difference between the various resolution modes is the pixel count and the speed at which the machine can run.

Now, if you're comparing DIFFERENT SCANNERS - say an original 2.5k ScanStation (which used a CCD camera), or a 5k ScanStation (which used a 5k CMOSIS CMOS camera), or a variant with a 4k camera (which I think is still Sony IMX like the 6.5k but a different model), or an Archivist (which technically isn't a ScanStation and uses a different model Sony IMX camera), then potentially you might see some color differences. But those are different machines, not different modes on the same machine. 

Looking at the files more it does seem like the 5k and 6.5k actually were not done exactly the same way. The 5k were exposed a bit more with a slightly different color balance used it seems (although the metadata seems to claim the LED balance was the same so not sure, maybe the color balance difference is because they were processed differently by scanner software at some stage or maybe the metadata doesn't report a difference in how the WB was actually really set or maybe ACR uses a twisted profile or maybe who knows what was done with the early 5K quick demo samples). So until I get some more samples done same time, exact same way for 100% sure, I suppose ignore my last few posts.

That said, if I raise the whites to make them look more or less the same exposure wise and set white balance on same white part to get white balance the same, the 6.5K one needs a huge +38 saturation to look similar (tint and colors are still a touch different). Just messing with color temp and tint sliders in ACR, for this one particular image, it doesn't seem to affect saturation too much at all though. Although perhaps the scanner sensor and such is very sensitive to the balance of it's LED lights and how it sees film and maybe it records a lot less saturation if it sends a bit less light through or something and maybe even for this color it would do that even if that color doesn't act like that in ACR? I wouldn't think it should make THIS much of a difference from a bit of an exposure difference and slight, nothing huge, seeming shift in white balance on the scanner to the images overall saturation, but maybe?

Anyway, since it seems they actually were not scanned the exact same way, I guess it is better to ignore what I said until I can verify it happens when the conditions truly were exactly the same for the scans. I'll try to get a sample 5K and 6.5K frame absolutely done the exact same way at the same time and see what happens. Maybe something was set weirdly or vastly different the 5k demos frommuch earlier on. Again, it's hard to say not running the machine myself.

At the least though there do seem to be the somewhat strange things:

1. metadata suggests the data is more or less REC709 gamut which one would not think should be the case for native scanner output, of course the metadata could be and probably is wrong, but why put in incorrect metadata? data should really be stored in reference to something correctly given so programs know how to interpret it (now sure the data may still need some calibration adjustment after that, but the base starting point should at least be correct) Or if the metadata is correct, how can native RAW from the scanner end up being so ultra close to REC709 gamut? (side note: at least with the way the 5K samples were scanned they sure seem to work and act like they have REC709/sRGB-like data in terms of how saturated the primaries are (although with whatever was done with the 6.5K that doesn't seem to be the case, so it's all strange, trying to adjust exposure to equalize didn't match saturation at all, but ACR might do funny stuff with color engine, etc. etc.))

2. how come if the DNG are true RAW files it puts the display rather than scene referred flag in the metadata and tells programs to treat them closer to JPGs and TIFFs than real RAW files (for instance ACR gives a points slider for color temp instead of the Kelvin scale, etc.), you'd think true RAW scanner output should be scene referred flag like it is for DSLR RAW images or for most stills film scanner RAWs

Anyway all I can go by is what I received and what I was told and how the files are loading on my machine in ACR (and seemingly also Resolve, although I didn't use this too much). Maybe I was told incorrect stuff or something got toggled or there is a bug in ACR or something. All I can say is what I was told and what the metadata in the files and how they are acting. I do agree it would seem fairly strange for 6.5K files to be stored in reference to a different gamut than 5K ones (especially if it didn't have to de-Bayer and downscale to 5K but it seems like it just uses a smaller part of the sensor so that does make it quite strange). I don't know why it should be at all.

The 5K (and 6.5K) files have metadata suggesting they are nearly REC709 gamut (although this appears to be wrong for 6.5K files at least, unless the difference is not due to some ACR/Resolve software bug that is for some reason making them load differently) and the 5K samples I got acted like REC709 in processing so that is why I was asking about how you get wide gamut out of this scanner. Now the 6.5K files I got later behaved differently. It does seems strange, but they do. As a bit of a sanity check I was give a tiny new set with 6.5K and 5K and in the new set the two sizes once again behaved differently. Maybe it's a bug in ACR or Resolve. Maybe the scanner somehow hit something different each time for 5k and not for 6.5k. Maybe the scanstation is doing more stuff to the lower file sizes. I dunno. If they truly are both in the same format, than there is some bug in some software on my end somewhere along the line. And then there is the question is the metadata correct or not and is the way they are loading as 5K or as 6.5K on my system correct. In one case, the scanner would not be producing wide gamut output but clipping to REC709. In the other case, at the least, the data output would be stored in some moderately wide gamut format. Loading the 5k samples into RAW software that was told to ignore metadata for color gamut and just pretend the data is stored in reference to REC709/sRGB primaries made them load looking fairly normal (do that to wide gamut format data and it looks all washed out). I didn't try that for the 6.5K files.

I can find and show this demo at least to show the metadata for 6.5K files being wrong part at least.

The 6.5K metadata says to process files like this as default load:

If major changes are applied to the near REC709 gamut by altering the primaries to something like some custom wide gamut mode then people can get it to load more correctly like:

With the 5K samples I got it would be like the above only the first image would have about the same saturation as the second as a default load, but with the color tones still like you see in the first image and because of that that is why I was questioning if it was storing native wide gamut or not, since they seemed to load fully saturated when interpreted as REC709 gamut. The 6.5K files I got don't seem to load fully saturated when interpreted as REC709 gamut.

On 1/19/2023 at 9:11 AM, Perry Paolantonio said:

I have avoided this thread because, quite frankly, I think you're looking for a problem that doesn't exist and you have yet to post a single image demonstrating the supposed problem.

But I need to correct this:

The ScanStation's resolution modes have nothing to do with color. They're unrelated. The "modes" in a 6.5k scanstation refer only to the pixel dimensions (and subsequently the speed at which the scanner runs, because lower resolutions mean less data).

The internal processing of the data, the output file formats, all of that is identical regardless of the resolution. The only difference between the various resolution modes is the pixel count and the speed at which the machine can run.

 

Well all I can say is the 5k DPX and DNG file samples I got appear to be in a different gamut than the ones I got in 6.5K. All done on the same machine with the same settings as far as I've been told. The 6.5K ones load into ACR or Resolve with noticeably less saturation than the 5K ones. I'm not running the machine myself, but that is what I can say based upon the files provided. I mean maybe something got toggled somewhere somehow. Again all I can go on is the files I got and what I was told. I'm not running the machine and was not there when stuff was scanned. The only thing I can think of is that to produce lower than 6.5K they might need to de-Bayer first (depends upon how it creates lower res) and once they are doing that, maybe they do more processing? Although if what you say is how it generates lower res then that would not seem to be the case, it shouldn't need to de-Bayer first.

The metadata for all of them seems to be wrong. They are all stored with a forward color matrix that would make sense for something very close to REC709 gamut. Applying that to 5K files I got looks kind of correct, saturation looks normal, but you do see that there is a weird shift in some colors, red is too orange, some other colors don't balance relative to others (nothing that color temp or tint fix, they can fix one color but then send the other off), maybe subtle at first glance, but quite real in the end. Anyway, the fact that REC709 gamut applied to interpret the 5K files looks remotely normal at all would kinda imply the data is not in native scanner wide gamut but has been pre-processed. Applying the metadata or REC709 to the 6.5K files I got doesn't look correct at all, same as for 5K but also with saturation much too low. If you do large adjustments to Adobe ACR Calibration panel to adjust primary locations you can eventually get it to start doing stuff that looks normal.

Anyway, how do you explain the forward matrix in the metadata being so similar to a forward matrix for REC709? Either it's completely wrong (which actually does seem to be the case for the 6.5K files) but in that case why do they put bogus metadata and not metadata that properly describes the gamut the data is stored in reference to? or it's correct and the data is not in native scanner gamut but processed down (As seems, sort of (not sure about the color twist), to be the case for the 5K files I got).

At least that is what appears to me to be the case at this point.

opps OK even more simply hah:

6.5K scans actually do seem to be stored in a format that allows for wide gamut colors

any scan under 6.5K does NOT seem to be stored in a format that allows for wide gamut colors (all my sample scans when I was first posting here were under 6.5K)

it is not yet know whether the 6.5 scans actually deliver any wide gamut colors, although the data format would make it possible

the metadata for color gamut seems to be somewhat wrong for all files scanned under 6.5K and utterly wrong for all files scanned at 6.5K, it tells you totally wrong info for programs on how to interpretate the colors so you need to RADICALLY custom adjust the primary locations in programs like ACR or use color charts and special software to build up LUTs, not sure why they don't give proper metadata

More simply since my last post I have received some 6.5K scans in addition to my initial 5K demo scans:

It seems that 6.5K scan mode has data stored in reference to some wide gamut format that probably could hold most colors film is capable of while any scans it does below 6.5K seem to be full re-mapped to a non-wide gamut format that seems like it would have to clip some colors film is capable of.

It seems the metadata for color gamut is totally utterly wrong for ALL format outputs from the Scanstation at 6.5K and somewhat wrong for ALL output formats from Scanstation under 6.5K.

Sub 6.5K stuff will load into Resolve automatically as REC709 and look kinda OK, but on close examination some colors are twisted, reds go orange, you get wrong relatively color balance, it can NOT be correct by color temp or tint or most normal color controls, you need to do more advanced adjustments like adjusting primary locations or use custom LUTs. It certainly seems like it would clip or remap down wider gamut colors that film can produce. The data does not seem to be set up to be able to handle wide gamut colors.

6.5K stuff loads in Resolve and ACR looking all dull and weird, since it is treated as something close to REC709 gamut but it is not at all REC709 gamut, metadata utterly wrong. To get proper colors it seems you must do extensive calibration adjustments to Adobe ACR primary locations or feed custom built LUTs into other programs so the data can be interpreted in the format that it is actually stored in which seems to be some custom moderately wide gamut format. These 6.5K scans might contain all colors 35mm film has or might not, the data certainly allows for it, whether it does or still gets clipped, not sure yet.

Sorry I've been out of my own thread for a while.

I've received more files (my original samples were all 5K and my new ones were mostly full 6.5K with a few 5K for additional comparison). I also got a little bit of an IT8 chart scanned.

In the end it seems that full 6.5K scans from it ARE stored in some wide gamut format (contrary to the totally incorrect metadata which claims something similar to REC709 primaries). It's something weird though. Vaguely like P3, but definitely not P3. It means that you need to painstakingly feed it calibration charts and try to build a LUT or manually adjust primary locations by eye until the data matches what is on the 35mm prints. Kind of weird, the files really should have metadata that lists what primaries they are stored as! What is the point of making someone spend endless hours trying to say tune up adjustments to the primaries in Adobe ACR compared to what the metadata falsely claims they are? Or spend time trying to build some program to make a custom LUT out of chart data and so on. Sure you may need to fine tune to adjust how the scanner sees a certain stock or this or that but we are not talking about that but simply finding out what format the data itself is stored in reference too. Yeah, DSLR makers don't give you the proper metadata for the primaries of the RAW output of the cameras which is silly too but at least there the big stills processing software companies all test the sensors and figure it out and provide a RAW interpretation file for each camera model. It doesn't seem any of them do that for things as esoteric as motion picture film scanners.

Now whether or not these stored in reference to some custom wide gamut format 6.5K scans actually store any wide gamut colors or the engine actually clips them down first and then stores the data is not yet known for sure yet. So far it seems a little suspicious that things seem to stay within REC709 triangle for the most part (and what doesn't might possible just be getting pushed beyond by grading choices). But then I may not have the proper stuff scanned yet, maybe the DI were done in REC709 before printout to film, not sure.

So whether the scans can ever actually ever contain any colors beyond REC709 or not, as motion picture print film certainly can, not yet sure, but the data format CAN hold such colors though, contrary to my initial claims, BUT that is only because the new scans are full 6.5K scans and these seem to use a different reference for the data than any sub-6.5K scans from the Scanstation do.

So, as just mentioned, on the other hand any scans done under the native 6.5K seem to NOT be stored in a wide gamut format, but some regular gamut format. It's vaguely like REC709 gamut, the saturation is about the same, but the primaries seem a bit off, they almost seem to have the same hue as for the full scans and yet the saturation remapped to be something like REC709/sRGB, kind of odd, not sure if that is a bug or not. It definitely doesn't seem capable of storing any colors beyond REC709 (beyond maybe a little twist) in the data format used for the 5K and under scans (which is what my original samples were done in).

So it is seeming like I was correct and not correct. The scans 5K and under, which my original demo samples were, do NOT appear to allow for wide gamut color and seem remapped to REC709 max saturation (although with an odd twist to certain colors, almost like they left the primaries hues in some wide gamut format). But it seems that once you do full 6.5K scans that suddenly the data is stored in reference to a different color gamut, one that is definitely stored with saturation levels that seem more like something wide gamut, maybe vaguely along the lines of something could allow for P3-ish levels or just a bit less max saturation. Tentatively, the blue primary seems quite different from P3 though in particular.

Unfortunately and shockingly, it's starting to seem like I was correct.

One scanner doing experiments and someone from another forum (apparently someone contacted some expert at film-tech and he seemed to come to the conclusion that it surprisingly outputted only REC709 color primary referenced data and simply clipped away all the extra rich, beautiful colors that 35mm film can record that go beyond that) seem to be reaching the same conclusion, that the Lasergraphics Scanstation 6.5K is absurdly appearing as if they remap the wide gamut output the hardware produces to the limited sRGB/REC709 color gamut in their software before outputting files. If they force the data to use REC709 primaries they seem to agree that the colors then look most natural and if they force the primaries to be interpreted as various wide gamuts it seems to come out over-saturated and/or odd. Nobody can seem to get a straight answer from Lasergraphics and they seem to be very evasive and stonewalling even to owner's of the machine who pay $$$$$$$ each year for service contracts. Honestly, they could simply add an option to allow for full native scanner gamut output and do it in less than a single afternoon but it seems they have no intention and maybe are afraid to do that since then they have to go tell everyone that all they scans they ever did, all the films archived by film archives and museums and so on had the color gamut clipped? It would seem a horrible shame to think of many films that got their one and only scan done without anyone realizing that the color gamut was clipped and that maybe for all time the original colors will be lost as the original film rots and fades and the digital archive failed to capture all the colors. I dunno how how a tens of thousands of dollar machine could do that when a $500 stills scanner over a decade ago could easily do wide gamut. And it's nothing to do with Bayer, almost every single DSLR has a Bayer sensor and even the 1st Canon 10D DSLR did wide gamut color. So does the camera inside of the Scanstation 6.5K.

I dunno, I wish I was wrong, but it is starting to seem like I was likely correct. I mean maybe I and some others now are wrong and I'm saying very unfair things about Lasergraphics, don't take this as 100.000000% proven gospel, but it's sadly sure looking like I was correct. I'll be able to judge more myself when I get some full scans back soon and also get a color chart scanned. Hopefully I'll be proven wrong, doesn't seem like it. We'll see. If so, it's kinda frustating. I think even Cintel likely allows full wide gamut color capture. But the Scanstation seems to be clearly better in every other regard so kind of a shame, why have it crippled for now reason (if it really is)?

On 9/28/2022 at 8:32 PM, Robert Houllahan said:

I can ask some high end colorists who work on Marvel films etc. but in general there is no setting color space from a film scan the color space is set really by the display device and space you are grading in, not by the scanner.

Thanks!

(The thing is you need to know what the scan data is stored in reference to to be able to properly translate it into whatever working space you chose. You could chose P3 and set a display to P3 but then if the data is NOT in P3 the software needs to know what it is in so it can translate it to P3. What if one (or all) of the primaries are in very different locations than the ones in P3. If you just treat it like P3 data the color balance will be all twisted in weird ways and the saturation for each channel will be way off. I just don't yet see how a motion picture scan can somehow magically not need this initial translation step that all other visual data seems to. From DSLRs it does (which is sort of what is inside of the modern motion picture scanners), from digital motion picture cameras it does, from stills film scanners it seems to. So far it almost seems like Lasergraphics already pre-does this step and just puts it in reference to REC709 color primaries (although not REC709 curve) at least for the Scanstation, perhaps not for the Director, or maybe it is actually closer to P3, but what exactly is it then and why do they not put it in any metadata and why does Resolve seem to default to loading DPX16 and DNG from it looking like it assumes it is REC709 and not some custom scene (scanner) referenced color gamut?)

On 9/28/2022 at 8:32 PM, Robert Houllahan said:

Well actually not really, especially on "real" RGB scanners which shoot each color with a monochrome sensor and RGB+IR LED lamp pulses, these are mapped into a Cineon Log curve to make 10bit RGB DPX frames. So there is no "Raw" file to be demosaiced as each channel is a full scan record. With a 16bit DPX or Tiff the linear data is mapped into the 16 bits from the sensor, the Arriscan for example uses a 14 bit ALEV monochrome sensor and in 2-Flash HDR it is a full 16 bits of data per channel for each RGB color.

That has nothing to do with what color primaries things are in reference to though. I mean some of the Fuji DSLR were true RGB and they still had to have their RAW files opened with something that knew what color primaries to reference. And my Nikon 9000 ED stills scanner is true RGB per pixel and it's output all appears to need to be in reference to some primaries.

I mean think in 1D and think of money. Say you give me $100,000 USD and then I say 'scan'/'convert' it to $100,000 Digital Dollars. And then Digital Dollars are not USD, not Pounds, not Lira, not Euros, etc. But you can't say oh you can take the $100,000 Digital Dollars and put them in whatever 'format' you want directly. I mean you can, but if I then called it Lira and gave you $100,000 Lira I think you might be a bit upset. You need to know what the Digital Dollars were in reference too. They were actually in reference to USD in this case, but if you don't account for that and then apply a conversion rate, it doesn't work out so well for one party.

On 9/28/2022 at 8:32 PM, Robert Houllahan said:

 

So it pulses the lamp R+ R- G+ G- B+ B- (and IR if a dirt map is to be made) in 2-flash HDR 16 bit mode, which runs at about 3 FPS scanner speed. The important thing is not to clip the file, i.e. to get all the density range on the film into a digital container without losing any detail in the shadows or hilites.

Scanners with CFA cameras (Scan Station Kinetta etc) basically mimic to the best degree possible the operation of a "true" RGB scanner by setting each RGB LED lamp pulse to just below clipping on the clear part of the film base, then they do either a Matrix or 3D LUT in the scanner to fix the color channel cross-talk from the sensor's CFA dyes and that is generally then mapped into a RGB file like DPX or ProRes4444.

On 9/28/2022 at 8:32 PM, Robert Houllahan said:

I can ask some high end colorists who work on Marvel films etc. but in general there is no setting color space from a film scan the color space is set really by the display device and space you are grading in, not by the scanner.

But supposing the scanner camera+filters had primaries really close in to the center of the CIE XY chart and you then viewed the data on a P3 or REC2020 monitor directly. It would like insanely over-saturated and the color balance would likely have a very strange twist to it.

Or suppose the scanner camera+filters had primaries pretty far out, beyond P3, around REC2020 or something and you then just viewed it on a monitor set to REC709 gamut, it would look pale, washed out and likely with a strange twist to the color balance.

That is what happens if you just take RAW data from a DSLR and don't apply any knowledge about what the camera's color primaries were. So that is not done. The software companies find out what the primaries are and then treat the RAW data in reference to those primaries and then convert it to a standard intermediate color management CIE XY D50 space and then from there they translate it to whatever working space you want to work in and then that is translated once again to whatever your display is set to.

On 9/20/2022 at 10:02 PM, Robert Houllahan said:

I think this assumption is incorrect the color space for a film deliverable grade is chosen and the film scan is a RGB value (linear or log) with 0-1024 values for each color in 10bit. As long as the scan is not clipped either in the shadows or the hilites the color balance can be set to the desired look of the colorist. Same with Arriraw or other formats they are color space agnostic.

The data has to be stored relative to something though doesn't it? I'm still not understanding how now. Either simply whatever the filters on the scanner's camera are or something else if they decide to remap it. At the end of the day, it's basically just a DSLR inside the scanner in a sense, taking a digital picture of each frame. Any DSLR RAW photo stores it's data relatively to the raw performance of the sensor + color filters. Any program like say ACR in Photoshop that loads it properly goes to a custom table that the software maker creates to look up what they found the proper primary locations to be (would be nice if the camera makers just gave it in the metadata but they seem to wanna be all secretive).

It would seem strange and annoying if the motion picture scan machines were to scan it relative to the scanner's primary locations and then not put that data in metadata and then you have to hope some software happens to have the proper internal table to handle it (which would even be fine, although it kinda seems like this is not something the softwares have in these cases, perhaps they do for Resolve and Cintel since that is their own scanner, it seems like maybe they do understand that scanner and load custom wide gamut primaries for that, maybe, haven't tested it yet, hard to find info, after tons of searching a little bit of info makes it seem like maybe that is the case) one just randomly apply color gamuts and see which one sort of looks semi-reasonable as the starting point. And most grading tools are not really meant to correct for wrong assumptions of where color primaries are located I don't believe and I think might be tricky to get to match exactly if you have to pick something that isn't exactly right at all. Maybe they figured since motion picture scanning is so obscure compared to DSLR work or digital motion picture work or stills scanning that a lot of software won't have the proper look up info so they just internally scale it all to REC709 gamut so anything can just load it as default with no mistakes, but that would seem quite a shame to lose those colors that print film can handle that are beyond REC709.

Or maybe they expect one to have some test sample of colors for every stock they use (not easy if it's old stock in particular) or to use a spectro to measure various colors on a projected print and find out what they are and then compare to values in the scan and then custom build your own color table and apply your own custom input LUT to the scan data. But that's a helluva an undertaking and not even so easy to do super well, when it seems it should be able to be handled so much more trivially easy. I thnk I've heard of studios doing stuff like that back in the early days, but it would seem like we should be way past that.

Maybe for a negative if you didn't ahve the primaries chosen eactly, it's not quite so bad as no final look has been chosen (but it would still help! and could still introduce a lot of tricky issues) but for a finished print with all the colors already locked in it's really strange seeming.

But also strange is that if you use Resolve and set Custom under part of the color management panel and then you can select the input color space, which I think is you manually telling Resolve in what color gamut+curve the data is stored as (you select both separately if you wish) and it allows you do this when loading DPX16 format and you select something really wide like Arri Wide it just looks insane, wayyyy insanely over-satured and twisted in color. If you select a more mild wide gamut like P3, which should be somewhat close to a print gamut, I have to say it just kinda looks oversaturated to me. I don't have the print on hand to compare with, but it seems hard to believe it looks quite THAT intense. OTOH, if I tell it to treat the data in the DPX16bit linear Print file as REC709 it kinda looks about like what I believe the saturation on the print actually is. If it's stored in wide gamut I'd have thought that would make it look quite bland and maybe noticeably twisted in color and that P3 or something larger might make it look more ballpark. So the data just keeps seeming like it's awfully close to REC709 gamut. I'll check into more in case I made a mistake and somehow it turns out to be closer to P3, which would be good, but it seems like otherwise.

12 hours ago, Robert Houllahan said:

I question your workflow from this.

Motion Picture scans generally do not have any color gamut assigned, you take a DPX or TIFF scan and put it into Resolve or Baselight etc. and then work in the color space you want to work in, the scanner does not assign a color space. So you can take the 16bit TIFF sequence and run in ACES or BT2020 etc. and off to the races you go....

But whatever loads them needs to know what the R, G, B values stand for. Where on the CIE plot is the red primary that the data is in reference to, to make it easy assume an 8bit per channel (and yes I know it's more), so what would 255,0,0 represent? It totally depends upon on what reference primary red it's based upon. To properly work in a display referred or standard type colorspace the program needs to know how to translate it into that. With a RAW file from a DSLR it will be stored with reference to whatever the sensor+color filter array end up making it be. If you try to load it into a program that doesn't recognize the camera it was shot on, it won't load properly. It needs to know the exact camera model and then go to a look up table (since for strange and unfortunate reasons the DSLR makers don't just list the info publicly and don't store it in the metadata) that they had to come up with themselves by studying the properties of the sensor and more it's color filter array and then treat the data with reference to the primary locations for that particular camera model. They transform it from that into CIE standard color management base D50 XYZ space. And then in the programs you can chose whatever working space you want, for stills, it's more stuff like sRGB or ProPhotoRGB (for video more often REC709 or P3 or REC2020 or use an ACES workflow) and then it transforms that data into that space and everything is done in that space (it may need a further transform to the display's exact space if not the same as the working space, in video people often try to set the display to the same as the working space, in stills that is less often the case and the working space is often larger than the display's max space).

But the thing is a program needs to know how to interpret the data. If not it will almost certaintly end up showing the image with weird tints and either under or oversaturated.

12 hours ago, Robert Houllahan said:

DNG is not really a format that any motion picture post uses to work in, so I have a few questions.

1. Are the cDNG files directly from the Scan Station?

I believe so. That was the impression I got, but I was not there.

12 hours ago, Robert Houllahan said:

2. Are the cDNG files from a Negative scan or Print positive?

Finished, balanced, print positive.

12 hours ago, Robert Houllahan said:

As far as I understand the Scan Station can make cDNG files but they are just the unprocessed data from the Sony Pregius IMX342 sensor so a negative scan will not be encoded into LOG nor will the cDNG file appear loaded into a piece of software as a positive so you will have to do that transform.

Yeah negative scans have to be inversed.

There did seem to be a PrintLOG option as well, it definitely used a different sort of tone curve.

But the actual goal in my case was just the regular Print mode.

12 hours ago, Robert Houllahan said:

The LED Lamphouse is set to the just below clipping balance of the film stock per color channel which is your primary light source for scanning.

The AnalogBalance 3 entry matrix in the metadata I'm guessing instructs how to handle this and get the data treated with channels in the original balance, taking advantage of the extra DR storing it this way allows, I believe.

12 hours ago, Robert Houllahan said:

The Sony Pregius IMX342 sensor has a CFA (Color Filter Array) built by Sony and the color dyes are not something a scanner manufacturer can choose, these are off the shelf machine vision cameras used by LaserGraphics / Xena / Kinetta / VarioScan etc. So the scanner manufacturer has to do some math in the Debayer and encoding and if you scan from a color sensor system to cDNG you are likely missing any color science the scanner manufacturer does to write to DPX or TIFF or ProRes and also the cDNG precludes using a 2-Flash HDR process etc.

I would have thought so, but surprisingly, it appears to not be the case.

The cNDG doesn't seem to preclude 2flash HDR. It even sets the metadata to claim that the DNG is stored in display referenced form and not scene referenced form (which is what DSLRs and digital video cameras do).

12 hours ago, Robert Houllahan said:

So if I were trying to figure this out I would drop all the still processing apps and work in Resolve or some other system for motion picture work and go from DPX or some other motion picture file format.

It could be that some software recognizes the IMX342 and does have the native primary locations and uses that.

But the thing again is, if I simply force a more still oriented program to treat the color gamut as if it was sRGB/REC709 then it loads in looking with the same color tint/saturation as it does in Resolve or PremierePro for the DPX16 samples. If Resolve had a special table for how to handle Scanstation and IMX342 and the data was in wide gamut reference of the native capability of that camera then telling Photoshop or whatnot to act as if the data is stored in reference to sRGB/REC709 primaries should make it load looking pretty undersaturated or at least more undersaturated, and very likely with a noticeably different tint as well, but instead it loads with almost identical saturation and the tint shift is very minor. I mean it ends up looking extremely close, almost identical to how the DPX16 automatically loads into Resolve when doing that. If I do the same thing with a wide gamut image from a DSLR it loads way differently when one program assigns it the wrong color space reference.

21 hours ago, Gabriel Devereux said:

Question,

I'm not familiar with CinemaDNG file types. This 'forward' matrix, if it's baked in the metadata... are you sure this isn't a standard XYZ to 709 linear transform -

 

I thought it was for taking the raw scanner/camera data and converting it to the CIE XYZ D50 intermediate space where most color engines seem to process everything.

If it was just taking the raw scanner/camera data and converting it straight to sRGB/REC709 then that would be another matter and the RAW data could then still be wide gamut and that certainly would be nice! I hope I am just being dumb and interpreting that matrix at the wrong end.

One weird thing is that the regular color matrix values in the metadata are just the CIE XYZ identity which doesn't seem correct at all. And the programs that default to using those matrices first and reverse engineering a forward matrix from them display the image in beyond wild crazy colors insane saturation and tint so I don't think those can be correct.

I thought I was looking at it properly though to read the RAW primary locations of out of, which were quite close, but just a trace off from sRGB/REC709. Unless it was meant to be crunching it down to that. But when I look at the ForwardMatrix someone showed for a DSLR it was very different than the one in this film and when I did the same thing to it, it popped out very wide gamut primary locations.

Also, the fact that if simply told Photoshop to assign sRGB to the DPX16bit linear scan and it looked what seemed to be reasonably as expected made me also think the data was stored in reference to sRGB/REC709 primaries. If I tell Photoshop to assign sRGB to a DSLR jpg that I saved in say AdobeRGB or even more ProPhotoRGB then it looks very off when it loads. Same if I say load a DNG into RawTherapee from this, well first it uses the COlorMatrix and gets crazy results, but if I tell it to assume the data is stored in reference to sRGB/REC709 primaries it looks kinda normal (unless the film is more saturated than I think and this is jsut some undersaturated somewhat off look).

Another problem is if say the files really did have wide gamut primary referened data in them, then if the DPX16 files has zero metadata about the color gamut at all then how in the world would any program know what to do with it? I mean for DSLR that actually is the case but then each raw converter software has to have a custom database, look up the camera name in the metadata and then apply whatever they found by testing the proper primary locations to be. But the DPX are not even really raw files and Photoshop has no idea what to do with them. And in ACR for the DNG, if I load in images and use working space ProPhotoRGB, wide enough to handle anything it could reaosnbly have in the data, and then raw proof it down to sRGB/REC709 primaries it doesn't show any clipping warnings, which does happen if I do that with a known wide gamut image (granted many frames won't ahve wide gamut info, but a few of these were pegged all straight along the edge of sRGB/REC709 which means the film should have gone beyond and so should the file if it really was in there I'd think).

21 hours ago, Gabriel Devereux said:

 

as if it's in the metadata and applied on ingesting in a 32-bit floating point engine if you used a linear transform from 709 to AP0/1 or XYZ it would have no loss - I think. 

The reason why I say this is that there is no standard 709 transform. A colour correction matrix from RGB primaries is, of course, partially dictated by RGB primaries and partially dictated by XYZ/709 RGB values of the inherent illuminant (I assume D50 in your use case? - not sure why it wouldn't be D65?). 

yeah it is slightly off from a pure sRGB/REC709 D65 matrix which I was guessing perhaps because it is D50 based although I didn't do that math to see if that accounts for the little bits of difference or not

21 hours ago, Gabriel Devereux said:

Could you post the forward matrix? The ideology of reviewing a matrix and assuming it's compressing X to Y is convoluted - at least in my head, it is. 

CFA Pattern 2 : 0 1 1 2
Linearization Table : (Binary data 23879 bytes, use -b option to extract)
Black Level : 0
White Level : 65535
Color Matrix 1 : 1 0 0 0 1 0 0 0 1
Analog Balance : 1.03499997 0.9990000131 1.210325574
As Shot White XY : 0.3456999958 0.3585000039
Calibration Illuminant 1 : D50
Colorimetric Reference : 1
Forward Matrix 1 : 0.4360747041 0.3850649001 0.1430803985 0.2225044967 0.7168785933 0.06061689931 0.01393220016 0.0971044973 0.7141733173
CFA Pattern : [Red,Green][Green,Blue]

21 hours ago, Gabriel Devereux said:

I'm unfamiliar with scanning film completely. But, let's say for a minute, negate the primaries of film and let's assume that the scanner is using a standard D65 illuminant with an adequate SPD. 

Let's also assume that the scanner is using a bayer filter - that doesn't abide by the Luther-Iver condition (I would've thought personally a scanner would, as the downsides of abiding by the CIE 1931 CMF colour target filtration scheme are noise and latitude - I would've thought that scanning a negative itself would partially negate that and then just throw a shit ton of light at it, but ah well maybe they couldn't be bothered designing a new sensor? - #FilmIsDead, I'm joking... maybe).

So with this, we have the RGB primaries of the Bayer CFA in the scanner, our standard illuminant (D65) and our output Primaries XYZ/709. 

This matrix shouldn't match any other matrix. It shouldn't look like any matrix you've seen (unless you're a colour scientist used to calibrating cameras - for which I apologise) as it's partially - primarily dictated by the RGB primaries of the scanner - I've spent some time calibrating phone cameras to XYZ - even though the final spectral locus/primaries are defined due to the variance in input RGB primaries the values are never the same - usually drastically different!

it looks pretty similar to what I think is a matrix taking data stored in reference to maybe D50 sRGB/REC709 primaries and putting it in XYZ D50 space

it looks very different from the few DSLR matrices I've seen

21 hours ago, Gabriel Devereux said:

Also if the debayer has taken place prior to ingest - I believe (taking reference from SMPTE RDD31:2014) that the colour correction matrix from the tristimulus RGB primaries of the bayer CFA would've already been executed. 

 

 

 

 

On 9/14/2022 at 3:01 PM, Robert Houllahan said:

Yeah i am a bit confused too.

As far as I understand it film scanners generally do deliver wide gamut and I don''t see how in the imager/lamp or lin-log or demosaic math the gamut would be intentionally limited.

This is particularly true with a "True RGB" scanner that uses a monochrome sensor and multi flash R,G,B IR or a 3-line array.

It wouldn't be. At that stage it would surely be in some very wide gamut referenced form.

On 9/14/2022 at 3:01 PM, Robert Houllahan said:

It would seem to me that any gamut limitation would be in the area of the spectral response of the LEDs used in the lamp and the color dyes used in the CFA Bayer mask on the sensor. I doubt any scanner manufacturer is choosing LEDs with limited spectral response. The color cross-talk between channels on a Bayer sensor can be considerable and a matrix or profiled 3D Lut is probably in the pipeline to deal with the characteristics of the color sensor used and there may be some limitations there in comparison to a True RGB scanner.

 

Yeah, it would be very surprising they would use such limited filters though and yeah I doubt they'd use some weird low spectrum LEDs in it.

The only thing I could think of is that, for some reason, they decided to take the native scan data referenced to the primary locations of the scanner (which are surely very wide gamut) and then convert it down to sRGB/ REC709 primary locations maybe solely thinking of targeting the old broadcast standard or making it automatic that any program will load them normally without havinbg to have color management? But it would seem crazy restrictive to not have a toggle option somewhere to just leave it full wide gamut. Of course, maybe there is and it is just well hidden or strangely labelled and the scan operator just didn't have it set to allow wide gamut even though it can deliver that.

Either that or somehow there is something very different about these files that I am somehow missing being used to handling digitally shot RAW or stills film scanner RAW or that the print is way more saturated than I recall and it's actually NOT loading in correctly into programs and they are just defaulting to sRGB/REC709 since the DPX16 have no colorspace metadata and the DNG, for whatever reason, where given an sRGB colorspace data and the sRGB/REC709 that looks fairly normally is actually both somewhat undersaturated and with the colors somewhat twisted and tinted.

On 9/14/2022 at 10:53 AM, David Mullen ASC said:

I'm not sure many real-world colors fall far outside of P3/Rec.709 unless you are shooting fireworks and neon signs, etc. Wasn't P3 based on film print color gamut anyway?

P3 would be plenty decent enough (although it does still clip some real world stuff, although it's not too bad, although some flowers still handly clip it noticeably, but few displays other than a few really high level pro ones can even display more to much more than P3 yet anyway, so only really matter in a long term preservation sense for more mainstream future tech and not practically relevant at this moment) and certainly for typical print film that is good enough, since as you say, yeah P3 was based somewhat on typical motion picture print film gamut so if you have P3 or something very close, you more or less have all to almost all your print film could contain no matter what was shot on it and how.

If I thought it was giving something along the lines of P3 that would be perfectly fine, since that would be able to deliver everything I expect from the film scans.

The problem is that it seems to be REC709 and NOT P3 or something else wide. And yeah certainly for plenty of stuff even that won't matter too much, but if you get into fall foliage, tropical waters, emeralds, flowers, 80s clothes or any clothes with rich saturation, fireballs, some birds, brightly colored sports cars, sunrises, sunsets, etc. then it can easily get noticeably clipped off by REC709.

Heck even some animated stuff if they wanna go wild (it depends a lot doesn't some does) some colors look easily beyond REC709, wild reds, oranges, purples, cyans, yellows, greens, etc. in some of that stuff.

Just on an orange/red bit on some of the frames I see the gamut pegged all along the edge of the gamut, hard clipped along the edge, so those frames obviously had more than REC709 on them. On other frames it is all easily within REC709. It all depends on what is in the scene.

On 9/14/2022 at 12:54 PM, Tyler Purcell said:

That's what I'm also very confused. 

I guess his idea is that because digital cinema imagers can deliver a wider gamut, why can't film scanners.

Not really, I know that film can deliver a wider gamut than REC709 and I know that even my own personal film stills scanner can deliver a wider gamut than REC709 and I know that some even far cheaper totally avg Joe consumer level stills film scanners can deliver a wider gamut than REC709 so I'd naturally be surprised if it were to turn out that there is no way to set something like a Scanstation 6.5K to get greater than a REC709 gamut (hopefully there is or somehow there is something about the files I am somehow missing or not understanding).

On 9/14/2022 at 4:03 AM, Jon O'Brien said:

I've noticed that some in the stills world can concentrate on technical aspects of image that in the cine world could be considered overly fine and rarefied details of image science that few have the luxury to contemplate or study in-depth. After all, the individual pictures flick by at the rate of 24 or more per second. There's much to be done and considered and only so much time to give to the niceties of the single image. I'd say in motion pictures we tend to go with what 'just looks good' and if that means a simple colour science and a simple means of acquiring the end result then that's the way it just is. If I'm way out of line that's fine, just tell me. But I was struck with this thought while reading this thread.

Yeah, it's not that sort of stuff that I am talking about though. Color gamut IS the sort of thing you could easily notice at 24fps. I mean in the extreme form, if you slide each color primary over to the white point you get B&W and that surely looks different than something shot in color with REC709 primaries or whatever.

Lots of other stuff has been mentioned but that stuff is all NOT what I was getting at.

22 hours ago, Robert Houllahan said:

I would just say that high end “finishing” scans from a Scannity / Arriscan / Director / Xena are actual RGB scans either with a Tri- Linear RGB sensor or a Monochrome Area sensor and pulsed RGB LED lamp. 

Scanners with Bayer mask sensors are fast and can make very good scans but as far as I know they are “dailes” or “archival” use machines for more critical film and tv work.

There is some tradeoff with a color sensor, the dyes used in the silicon mask are not perfect and there is crosstalk between color channels. Any scanner maker has to do some math to figure out how to deal with this.

Color sensors are capable of wide gamut and it may just be a choice engineers made building the Scan Station and a possible feature request to LG as how this in interpolated is just how the software works.

Have you asked LaserGraphics? 

 

I suppose for some reason Lasergraphics might have limited native scanner primary reference wide gamut scans to the Director, but that would seem surprising. Even cheap stills film scanners, even some with Bayer method, allow for wide gamut scans and even full color scanners like Nikon 9000 Coolscan cost like 20x-100x less and came out years earlier and allow wide gamut film scans. So I wonder if maybe there is some weirdly name option that the scanner isn't seeing and has it set to REC709 color gamut when maybe some little setting change could allow for wide gamut. I mean maybe there is something about the scan files I am simply not interpreting correctly, but so far I can't see anything I am doing wrong in that regard, but open to suggestions.

Yeah Bayer sensors are not perfect, but almost all DSLR use them and virtually all DSLR have put out wide gamut RAW files or JPGs from day 1. You lose a bit of color resolution and precision and get more metamerism and so on and so forth, but in terms of primary locations, most of them have them set quite wide indeed, way beyond sRGB/REC709.

I did just ask Lasergraphics the other day whether the 6.5K machine can output data referenced to color primaries wide gamut than those of sRGB. I will see what they have to say, if anything. I've heard rumors that unless you own a machine or appear ready to buy one that they might not talk much, if at all, so I'm not 100% sure I will even get a response.

Someone else using one of the machines asked them a few questions somewhat along these lines and said they got a kinda vague answer that didn't really answer anything and it was sort of an "it's all proprietary secrets" kinda response that didn't make anything clear. I'll see if they get a more clarifying answer to a couple more exactingly specific questions.

In the meantime I was sort of hoping that in one of these forums someone would just be like oh yeah you just have to make sure that so and so setting is set/not set and then you'll get wide gamut reference output. Or, although it would be a shame, a nope, that one limits it with reference to REC709 color primary locations there is nothing the scanner could have done using this machine but so and so brand and model can do that. It's almost impossible to find info on exactly what any of the models from any maker can do regarding color gamut. I found a few vague hints that maybe the Cintels can actually give wide gamut color, but it is not entirely clear (they also seem inferior otherwise to stuff like Scanstation, etc.). It's surprising since in the film stills world it's trivial to find this info for any stills film scanner and people are talking all over about standard and wide gamut scans. But the motion picture world is a much smaller world. I do see that sort of talk a lot in purely digital work flows, but haven't had luck finding a lot when it comes to motion picture film scanned sources, only for digitally shot productions where, in recent times, you can find a lot of talk about wide gamuts and such (or back in the day, when the first digital motion picture cameras appeared when you'd hear some bash them for having small REC709 palettes and poor DR and lacking in the DR of film and it's more rich color possibilities).

5 hours ago, Robert Houllahan said:

Again did you have a LAD (Lab Aim Density) or TAF (Telecine Alignment Film) or some other SMPTE control film scanned to reach this conclusion?

It wasn't a TAF. The little sample bit I got back didn't include any of the color or gamma tracking frames at the very start of the print, just a few seconds of the film itself. A TAF doesn't seem to have very saturated colors so not sure you could see much from that. Perhaps if it had one of the fuller color checker type charts, as some prints do, can't recall exactly what this one has on it at the start, some of the colors would be beyond sRGB/REC709 primary locations. It would be good to get those frames included in the full final scan to see what tweaks need to be made to scans from the machine to get them closer to perfectly matching the print, but that seems like a separate issue.

I'm not sure why a TAF scan would tell you any more about the color gamut than just scanning a few seconds of a final print and seeing what the metadata in the DNG output says or seeing how the DPX react assigned different gamut profiles. Maybe one of the more comprehensive patterns would have a few patches out of REC709 gamut and one could try to see if they looked a touch muted relative to the others or see if any program or any profile assignment could bring them back, but that is a bit subtle.

You can easily tell roughly what sort of gamut any jpg image was stored in reference to so long as it has any reasonable amount of color in the scene, apply the wrong assignment of gamut and it will look normal or way oversaturated and twisted or way undersaturated and twisted (and for that file type the metadata would tell you the reference color gamut as well even if the image which 100% white or pitch black).

19 hours ago, Tyler Purcell said:

Why would you think it's a Rec 709 color space? What makes you assume that? 

Also I should clarify, I don't think it is that all the output samples are in REC709 color SPACE, they just all seem to be in REC709 color GAMUT (solely with regard to the xy locations of the color primaries on an CIE xyY gamut plot).

4 hours ago, Robert Houllahan said:

I am not able to scan to DNG direct with our Scan Station but maybe that is in a later software release.

I generally think the 2-flash is 14bit precision as the Sony Pregius 6.5K sensor is 12bit.

I do not know exactly what Lasergraphics is doing under the hood on the Scan Station but on the Xena scanner there is a Bayer mask transformation matrix to overcome some of the (quite large amount) of color channel cross talk. It is possible that the Scan Station uses a similar matrix or a 3D Lut to manage this aspect of using a Bayer mask sensor and the color dyes Sony used on the silicon.

I could do a comparative test on our Arriscan in 2-Flash which is 16 bit and true RGB.

Thanks.

Yeah, the Scanstation 6.5K also has a Bayer sensor like most DSLR so it also has to debayer the file (or in the case of DNG output let the software the user uses do the de-Bayer). From what I read, the Director and Scanity are true RGB per pixel, etc.

The real issue is whether the Scanstation is taking the native color data and transforming it from the native color gamut of the scanner, which is surely pretty wide, and converting it to a small sRGB/REC709 primary locations (long time broadcast standard, but far less than film's color palette and far less than the new broadcast and home video standards) before writing out the data to files. It would seem shocking for them to do that, baffling, but it seems like the samples I have received have had that done to them. I'm not sure if maybe the person doing the scan had some setting somewhere set that is forcing that or if the Scanstation 6.5K simply is really being restricted to that the smaller color gamut or if I'm somehow misinterpreting the files (but then how come the DNG have an sRGB forwarmatrix and so on?).

How are Hollywood studios preserving the full color gamut of 35mm/70mm films they can in and release on UHD if the scanners are clipping stuff to a small gamut on output? Yeah they can still get true HDR out of them, but what about the colors? I know for Wizard of Oz they simply did like a B&W for each of the three Technicolor strips and they probably measured the spectral response for what each strip recorded and then custom ended up capturing the full, rich, huge color palette of three-strip Technicolor. But what about regular films like on single strip full color Eastman or Fuji or whatnot?

If you look at people doing scanning of stills from stills film cameras, you see all sorts of talk about scanning into wide gamut formats and how one should not use sRGB/REC709 if you care about preserving all the possible colors in your slide or negative.

6 hours ago, Tyler Purcell said:

I'm confused. How are you testing the output gamut of a file without knowing the input value? Are you just testing on white light? Any tool showing you gamut, would show you "actual" image gamut, which is always compressed coming from the scanner anyway. 

In an image file based upon the normally used tristimulus model, the data has to be stored with in reference to some primary color locations otherwise how in the world do you even know how to interpret the data? You need to know where on the CIE XY chart the R, G and B color primaries are located.

The DNG output file has a ForwardMatrix in the metadata that tells programs how to interpret the data. Color management modules are standardized work in CIE D50 XY and that ForwardMatrix is multiplied by the R,G,B color value to convert it from whatever reference it is in to D50 CIE XY. The matrix in the sample files I received is extremely close to the matrix one would use to convert from data stored in reference to the sRGB/REC709 color gamut (the slight difference might be because the scanner light source was called D50 and sRGB/REC709 use a D65 white point).

And then again, if I load in a DPX16 and tell a program to assume that the data should be interpreted as if was stored in reference to sRGB/REC709 primary locations, it ends up looking pretty much the same as the file loads into PremierePro or Resolve or what the REC709 ProRes output looks like.

6 hours ago, Tyler Purcell said:

You can't convert a gamut from one type to another, that doesn't ever work. 

It's done all the time. If you have an image in say AdobeRGB and are viewing on a screen set to sRGB any color managed viewer will conver the gamut from AdobeRGB to sRGB so it looks normal on that screen.

6 hours ago, Tyler Purcell said:

Theoretically you'd "add back" the missing gamut information in post when you work in an expanded range. 

You can't add back colors that were clipped to begin with. If the data was clipped away, it is clipped away. You can boost some saturation and saturation at bright levels and sort bring some back to an extent, but it won't be the same and it's hard to not get other stuff artifically boosted looking. Also the tints might go all wrong since the color primary location it was interpreted as might not be on a straight line out in saturation from th white point but might be shifted one way or another.

6 hours ago, Tyler Purcell said:

Where I understand your reservations about this process compared to shooting on a DSLR in RAW, those imagers and imaging systems are very different than a film scanning system. You run out of dynamic range super fast on a DSLR. Film has WAY more dynamic range. So to compensate for this, we compress the film information and then expand it in post. So it shouldn't matter what the compressed gamut is on the DPX files. Once expanded it should fill in the missing data. 

 

I'm not talking about dynamic range though. Dynamic range and color gamut are not the same thing (although more dynamic range can make the gamut larger in a way, but that is not the part I am talking about). I'm not talking about like SDR vs. HDR. I'm talking more along the lines of color gamut portion of say sRGB vs AdobeRGB vs P3 vs REC2020 etc.

I'm talking about stuff like:

https://www.eizo.com/library/basics/lcd_monitor_color_gamut/

https://webkit.org/blog-files/color-gamut/

https://nick-shaw.github.io/cinematiccolor/common-rgb-color-spaces.html

https://www.benq.com/en-us/business/resource/trends/understanding-color-gamut.html

https://www.displaymate.com/Display_Color_Gamuts_1.htm

and I'm not talking about stuff like being able to see details inside of a room as well as stuff out the window in bright sun at the same time and I'm not talking about whether you have large steps between each tone or super fine gradations, etc.

Camera RAW files tend to not include the color primary locations since the manufacturers get all secretive, but that also is why you keep needing RAW processing updates when a new camera comes out. The software company has to go measure the color filters and sensor and see what the camera captures in terms of where the color primaries should be located and then it knows what the values in the RAW file mean. If we are say 8bit just make it simple, what does say an R,G,B of 255,200,10 even mean? if you don't know where the R, G and B color primaries are you have no clue what 255/255 red and 200/200 green and 10/255 blue means. If the green is located where it is in sRGB/REC709 it means one thing, if it is way, way up there on the CIE XYZ/xxY chartthen it means something else. If the data was in reference to the former but you treated it like it was in reference to the latter then you'd get absolutely nuclear intense looking greens and they wouldn't just have the green component way oversaturated it would also be shifted in tint most likely as well and have too much of a blue or red tint to it as well. That is the sort of thing I am trying to get at.