A sampling scanner for film archives

[Daniel D. Teoli Jr.]

A good scanner to make for film archives would be a film sampling scanner. Dan Baxter touched on it in another thread. A rough scan to see what you got then work on it later for a better scan if it warrants it. The sampling scanner could be set for scanning every 8, 12, 14, or 24 frames. It would then spit out a MP4 that could be put in the folder with the film record.

The point of it all is, the scanner could be made cheaper than your normal scanner would cost, and allow film archives to see what is on a film and keep a small sampling for the record.  In the past if I wanted a sample done, I'd have to look at the individual frame scans and take out frames by hand to make a sampling of the film.

Maybe the sample scanner could have a setting for speed as well when exporting the MP4. Say 50% to 250% range. Just don't get too anal with all the crap where you need remote learning for 2 days to run it. Sure, you can do all this stuff in post. But the point of a sampling scanner is to make the work fast and easy to do right then and there and not let the work pile up to be revisited. Revisiting work is a big problem. Work piles up and new work comes online, you forget what needs to be done with the old work, or at least waste time trying to figure out where you left off. 

I don't know how this would work; it is just an idea that may make life easier for the film archivist. I'll have to do the same thing I tell the people here to do...test it out to see what the results looks like. I've made a lot of films with still photos for the entire film, but never with the idea of sampling a film. 

<><><><>

New Year's Eve - Skid Row Bar 1971 (Candid)

Photo: D.D.Teoli Jr.

I shot it when I was 17. I looked old for my age and was able to sneak into the bars. Now that I am in my 70's, looking old for your age loses some of its luster.

Daniel D. Teoli Jr. Archival Collection
Daniel D. Teoli Jr. Small Gauge Film Archive
Daniel D. Teoli Jr. Advertising Archive
Daniel D. Teoli Jr. VHS Video Archive
Daniel D. Teoli Jr. Popular Culture Archive
Daniel D. Teoli Jr. Audio Archive
Daniel D. Teoli Jr. Social Documentary Photography
 

Edited April 9 by Daniel D. Teoli Jr.

[Daniel D. Teoli Jr.]

OK, here is a proof-of-concept type of thing...aka a testing of my 'sampling scanner' idea. 

.3 Second Sampler Of Breast Reduction Film D. D. Teoli Jr. A. C. : D. D. Teoli Jr. A. C. : Free Download, Borrow, and Streaming : Internet Archive

Surgery of breasts being reduced.

I tested various speeds and .3 seconds seemed to be the sweet spot between .3, .5 and 1 second for the test times. I had all these photos from when I scanned this film with a Retroscan years ago. I can't remember how many photos were in the scan, maybe 25,000 to 30,000. Whatever it was, it took a long time to grab stills. And these are not regulated as far as number of frames between images - just grabbed some frames at random.  

Well, too bad film is a dying area or maybe some company/s would make such a sampling scanner and make it affordable.

[Daniel D. Teoli Jr.]

You can see that you don't need that many frames from a film to get the flavor of it. 

Retroscan output: DDTJRAC

This sample section could be summed up in 2 frames.

Maybe a sampling scanner could be set via footage if it is easier than counting frames. 6", 12", 18", 24" 36"

If you're like me (short on time and overloaded with work and films) and don't want to bother with a Zeiss viewer any longer, just set up the film on the rewinds and loupe every 10 - 20 feet or so. 

[Dan Baxter]

On 4/9/2024 at 10:16 PM, Daniel D. Teoli Jr. said:

A good scanner to make for film archives would be a film sampling scanner. Dan Baxter touched on it in another thread. A rough scan to see what you got then work on it later for a better scan if it warrants it. The sampling scanner could be set for scanning every 8, 12, 14, or 24 frames. It would then spit out a MP4 that could be put in the folder with the film record.

This already exists. I don't understand what the point in purposefully dropping frames would be given you have to transport the entire length of the film anyway.

On 4/9/2024 at 10:16 PM, Daniel D. Teoli Jr. said:

The point of it all is, the scanner could be made cheaper than your normal scanner would cost, and allow film archives to see what is on a film and keep a small sampling for the record.  In the past if I wanted a sample done, I'd have to look at the individual frame scans and take out frames by hand to make a sampling of the film.

How can it be made any cheaper than what's already available??

Evaluation scans can be done on existing scanning machines, also Filmic is pretty much designed exactly to give you this as it comes with a condition report on the film.

On 4/9/2024 at 10:16 PM, Daniel D. Teoli Jr. said:

Maybe the sample scanner could have a setting for speed as well when exporting the MP4. Say 50% to 250% range. Just don't get too anal with all the crap where you need remote learning for 2 days to run it. Sure, you can do all this stuff in post. But the point of a sampling scanner is to make the work fast and easy to do right then and there and not let the work pile up to be revisited. Revisiting work is a big problem.

Well there you go, you've hit your first major limitation. Most scanners don't go direct to MP4.

What's wrong with using the existing technology to achieve the sampling scans as you call them?

[Perry Paolantonio]

51 minutes ago, Dan Baxter said:

Well there you go, you've hit your first major limitation. Most scanners don't go direct to MP4.

The ScanStation does. The majority of the scans we do include a second MP4 access copy in HD with a one-light grade. The file h is more or less blu-ray spec and will just play on most televisions off a USB thumb drive, at its native frame rate (much nicer than making optical discs and having to do frc and all that). We would never scan to MP4 only unless the purpose of the project was to create a digital reference library of a film collection. It'd be a waste of the client's money to pay us to do a scan to such a lousy format. But it's great for just seeing what you have. 

On 4/9/2024 at 11:36 AM, Daniel D. Teoli Jr. said:

You can see that you don't need that many frames from a film to get the flavor of it. 

I get your point, but as Dan Baxter said, it's kind of weird when the scanner has to go past every frame anyway. Why not just capture them all and make a low res access copy you can watch, which will tell you far more about the film than a page of thumbnail images? That is kind of what he Filmic is made for. 

About 6-7 years ago at NAB, MWA partnered with a company that used a modified version of the MWA Spinner to output essentially what you're showing - it was designed to evaluate the film: it looked for defects, it generated thumbnail images, it  figured out shrinkage percentages, etc. I forget what the software was called but it was basically an add-on to the MWA Spinner and I think the company that made it (the software) was also German.  The result was a PDF report, I think. 

Of course, one could generate similar reports from other software working on full scans of the film. That's something that could be done in software.

[Daniel D. Teoli Jr.]

I tested out this concept in 2 other speeds - .10 seconds & .20 seconds per sample frame.

It is something. You can reduce a 15 or 20 minute film down to a few seconds and still get a decent feel for it.

.10 seconds per frame:

Breast Reduction Surgery Sampler Test .10 Sec Per Frame 22 Sec Total D. D. Teoli Jr. A. C. : D. D. Teoli Jr. A. C. : Free Download, Borrow, and Streaming : Internet Archive

.20 seconds per frame:

.20 Second Sampler Of Breast Reduction Film D. D. Teoli Jr. A. C. : D. D. Teoli Jr. A. C. : Free Download, Borrow, and Streaming : Internet Archive

Now the only problem is how to do it on a fast and affordable basis. This is especially useful plan if you have many hundred 'pig in a poke' home movies as I do. You can only put so much info in the description. And words can never take the place of seeing it.

Stock footage companies could probably benefit from this technique as well. Both for their own internal use and for their customers use. It would make it easier for their clients to sift through the material to buy. Once the customer finds films with potential they can view the entire reel.

Call it the 'Synopsis of Film' or 'Visual Synopsis of Film.'

In the end we both do the same thing; it is just that I archive films to donate to the historical record, whereas they do it to sell by the second. 

 

 

Edited April 16 by Daniel D. Teoli Jr.

[Perry Paolantonio]

My point is that you shouldn't do this in the scanner. It's not saving you any time. 

If you're only scanning 1/3 of the frames in a film in order to make thumbnails, you still need to move from the beginning of the film to the end. That is, all frames have to pass through the scanner even if all frames aren't being scanned. The time it takes to shuttle from frame one to frame 5 on a scanner like the scanstation is longer than the time it takes to simply scan frames 1-5, because the scanner has to stop, switch to shuttle mode, shuttle, take an image to make sure it's in the right place, adjust if not, then take the image. rinse. repeat. All that takes more time -by a lot- than just scanning straight through. You can do what you want (grab every X frames) in software on the resulting scan of every frame. then you have both things. 

 

[Daniel D. Teoli Jr.]

When you do your tests, always go the distance...from the alpha to the omega. I tried a few other times with this test, but didn't post them. I found 1 second is too long and .05 seconds is too short. It looks like .20 or .30 of a second per frame is the sweet spot. 

[Daniel D. Teoli Jr.]

 

This is an easy way to grab stills for a sampler film. Highlight rows, copy and you are all done. This sample show a frame selected every 13 frames. If you want less, do 1 frame per row.

Edited May 15 by Daniel D. Teoli Jr.

[Robert Houllahan]

On 4/16/2024 at 2:43 PM, Perry Paolantonio said:

My point is that you shouldn't do this in the scanner. It's not saving you any time.

 

Also especially with archival materials this nitwitted scanner would necessitate more scanning for a finished scan, essentially scanning the film at least twice and maybe more times.

I would think that the best practice would be to scan the film once in high res DPX or ProRes and make multiple other viewing copy files like MP4 and DNxHD etc at the one time you scan. Easily done with the Scan Station internally or other scanners like the DFT Polar from the DPX files and Resolve or Baselight etc.

If you scan a very delicate film once on this and it ends up breaking multiple times you then have broken film and a bad scan.

[Daniel D. Teoli Jr.]

On 5/18/2024 at 2:33 PM, Robert Houllahan said:

Also especially with archival materials this nitwitted scanner would necessitate more scanning for a finished scan, essentially scanning the film at least twice and maybe more times.

I would think that the best practice would be to scan the film once in high res DPX or ProRes and make multiple other viewing copy files like MP4 and DNxHD etc at the one time you scan. Easily done with the Scan Station internally or other scanners like the DFT Polar from the DPX files and Resolve or Baselight etc.

If you scan a very delicate film once on this and it ends up breaking multiple times you then have broken film and a bad scan.

 

Well, that is true if film is delicate. You want to handle the film the least. But a sampling scanner would treat film easy and just give you a fast snapshot of the reel. It would be something like running the film through a Zeiss viewer. Most of my films are well worn; (muti-gen dupes, with scratches, etc.) a sampler scan is not going to hurt them one way or another Robert. 

You are not an archivist Robert, you just scan films for $ and that is it. You don't have to look over and after thousands and thousands of films and the digital output in your collection over the years. And if I am wrong, then please share your cine' film archive with us. A sampling scanner would be a most welcome tool to get a fast snapshot of a film.

And it is not only film I have to work with, I work in a huge number of areas of collection Robert. Here these are some of the A's and B's in the Archive to give you an idea of the scope. (Some of the proprietary files are removed.)

I'm a 1-person Archive Robert. So my time is always short. Beside archival work I am a social documentary photographer. Having too much to deal with is why I have a hard time getting back to these threads. Here is an old scope of the Archive. It is much, bigger now, Robert.

Collection Scope Of The Daniel D.Teoli Jr. Archival Collection : D.D.Teoli Jr. : Free Download, Borrow, and Streaming : Internet Archive

As a non-professional archivist, I say a sampling scanner would be a useful tool for the film archivist. I'm not guessing at this...I've tried the concept, Robert. And if you don't want to use a sampling scanner, no one is forcing you, Robert.

I use a sheetfed scanner on a lot of my paper scans. It is tougher on originals and the scans are only about 85% - 90% as good as a flatbed scanner. But the choice is not sheetfed or flatbed...the choice is to use the sheetfed scanner or not scan at all Robert. 

SHOOTOUT…Flatbed Scanner vs. Sheetfed Scanner vs. Copy Stand Photography – Daniel D. Teoli Jr. Archival Collection – II (home.blog)

Sure, I'd like to have a Phase One camera for my copy stand. But the reality for the copy stand is a 16mm Fuji and not a 100mp Phase One. We just have to work with what we can successfully use Robert to get the work done. 

 

Edited May 19 by Daniel D. Teoli Jr.

[Dan Baxter]

I still don't really understand the purpose of what you're proposing. Here's what you'd call a "sampling scanner for film archives":

And you can listen here (or rather read the subtitles) as to how it's actually used by one particular archive customer (Deutsches Filminstitut & Filmmuseum )

One of their main uses is to use it to inspect damage after they lend out a film and it comes back to them. Basically that reduces manual work having do be done doing it on a rewind table manually. It makes an automatic condition report which includes finding new damage like new lines in the print or new splices and missing frames etc. New lines in particular it can find because you can just compare your previous condition report to the new one, and if the line was in there the first time it was inspected it should be in the report the second time. Most of the professional restoration scanners won't work if you try to make all the base damage visible - if you do it on a sproketless LaserGraphics for example the sproket detection doesn't work and it looses all registration.

So again, how does dropping frames help the Archives do what they need... and wouldn't you think the companies making these things know the needs of the Archives they're making them for? Film scanners aren't high speed rewinds, their speeds are normally capped at what the manufacturer thinks is safe, so if you really need to drop frames for some reason why not just do it afterwards?

I'd note by the way, that they don't even call the device a "scanner". Quoting from the subtitles: "we have deliberately not bought a scanner, we have bought a device that we have acquired for our archiving purposes, which also now includes digital restoration. It's not a scanner, but a device with which we can see information edge-to-edge. We need this information." He also gives examples of how it can assist them with restoration - they can use it to make an access copy of a print as a reference for color timing the negative, they can also use it to compare the condition of different materials of the same film so they know which ones they should send for restoration and save money not wasting their time on something in poor condition.

Edited June 16 by Dan Baxter

[Daniel D. Teoli Jr.]

I haven't given up with the film sampler scanner idea. Although I don't have much hope with seeing an affordable sampler scanner sold on the market. Even so, I've been doing some experiments with using speed for sampler scans. The problem with this is you have to scan the whole film then process it, whereas a sampler scanner would give you a fast snapshot of the film, hopefully faster than trad scanners, to see what the film is about. 

Here is a film sampler made at 1250% speedup of 'Breast Self-Examination' (1950)

Speed experiments were made at 1000%, 1200%, 1250%, 1500% and 2,000%. 1250% seems to offer the best bang for the buck with time reduction vs. content viewing. The original film was reduced from 15 minutes 22 seconds down to 45 seconds. File size is reduced to 26.9mb.

 

My software only goes to 1000%, so I had to Macgyver it and run it though twice. Once at 1000% and once at 250%...which is a lot more work than having a sampler scanner to spit out a rough synopsis scan. 

 

https://archive.org/details/breast-self-examination-1950-sampler-1250-d.-d.-teoli-jr.-a.-c.

 

<><><><>

 

Daniel D. Teoli Jr. Archival Collection
Daniel D. Teoli Jr. Small Gauge Film Archive
Daniel D. Teoli Jr. Advertising Archive
Daniel D. Teoli Jr. VHS Video Archive
Daniel D. Teoli Jr. Popular Culture Archive
Daniel D. Teoli Jr. Audio Archive
Daniel D. Teoli Jr. Social Documentary Photography