Jump to content

Low-Budget Scanning For Indie Features


Thomas Beach

Recommended Posts

3 hours ago, Robert Houllahan said:

frankly there is allot of fetishistic tendencies towards camera gear and what scanner and all kinds of stuff that does not necessarily show up on screen but people want on set. 

If the writing is good and the cast can act you can make a film with a olde aaton some vintage glass bubble gum and duct tape.

Correct me if I'm wrong but could I be forgiven for thinking that this tends to be a certain type that I've come across?

A certain age bracket. Beard. Slightly intense type. Unpleasant. Really into digital ie usually something with a bl----y skull on it or similar. Sooo professional, and real industry people ... you know?

Edited by Jon O'Brien
Link to comment
Share on other sites

On 6/8/2022 at 9:50 AM, Thomas Beach said:

Is anyone doing their own feature film scanning using the MovieStuff Retroscan Universal mk-ii?  I realize it is only a 2k mximum scan, but am wondering if this product has found a market with the low-budget indie film community preferring to shoot on 16mm/S-16mm or Ultra 16mm? Any good discussions to be found relating to the pluses and minuses of bypassing the Rank-Cintl or Da Vinci big boys? Thanks.

The Retroscan Universal MkII needs a number of changes to make it usable (build a light, put in a proper camera, etc) by which time it's now a DIY scanner suitable only for the technically capable hands-on operator - which most of their users are not. The Retroscans are not designed as a dailies scanner nor for professional-grade work. They're designed for archives where the quality isn't essential, and for the home-movies-to-dvd people who don't know any better in terms of quality (they're often comparing their work to companies that "scan" by literally projecting the image onto a wall like this company does). It's also not going to save you money overall. If you want to buy anything look at buying the Lasergraphics Archivist as an all-around 16mm scanner.

On 6/8/2022 at 11:08 AM, Robert Houllahan said:

The Blackmagic Cintel scanner will do a better job and those are pretty "cheap" at $30k and maybe allot less if you can find a used one. It is also limited in resolution to about 2K wide for the whole 16mm frame.

Cintels can't scan Ultra16, there's a physical strip behind the perfs.

On 6/12/2022 at 2:57 PM, Tyler Purcell said:

The HDS needs to be run by a pro. The Cintel II is WAY easier to operate. Even I struggle to get a good image out of the HDS, it's very tricky to nail the color.

Thank you for pointing this out. This is the same for most scanners honestly, the operator needs to be properly trained, know how film should look, and instantly recognise when there's an issue with the settings. Just because something is easy to operate doesn't mean you'll get consistent quality out of it if it's not in the hands of a pro.

Link to comment
Share on other sites

  • Premium Member
4 minutes ago, Dan Baxter said:

Cintels can't scan Ultra16, there's a physical strip behind the perfs.

And they're not even 1080p in 16mm mode so not even worth discussing. 

4 minutes ago, Dan Baxter said:

Thank you for pointing this out. This is the same for most scanners honestly, the operator needs to be properly trained, know how film should look, and instantly recognise when there's an issue with the settings. Just because something is easy to operate doesn't mean you'll get consistent quality out of it if it's not in the hands of a pro.

I mean the Cintel is nearly automatic. One button and it perfectly white balances the image and you hit "start" and it works. There is nothing an operator can do really. Maybe screw up the threading. 

Link to comment
Share on other sites

On 6/12/2022 at 11:22 PM, Tyler Purcell said:

Which is $20,000 US dollars. I have no idea how you can get it for less, but that's what they quoted me last year when we got the HDS+. 

Just a quick follow up here:  I got the license for $10,000.  Here's how:  I helped them make their product better.  I provided a lot of feedback to them.  I bought two Dustbuster+ licenses.  Dustbuster+ is their starter product.  Kinda like Diamant Jr.

I rewrote the English copy for their Dustbuster+ web page.  They immediately published it.

I helped them make a better product, and they returned the favor by reducing the price of a Diamant license by 50%.  Maybe you could do the same thing for them.

On the topic of HS-Art software, I just bought a Mac Studio Ultra, and rendering Dustbuster+ timelines on it is blistering fast!  It takes 1/3 the time it used to take on our PC workstation and about 50% less time than on my old iMac Pro.  If time equal money to you, then using these film restoration tools on a Mac Studio will save you even more time/make you even more money.  

I can't wait to move our film restoration efforts off of the PC and onto a Mac.  Speed:  so much better.  Maintenance:  almost nonexistent.

  • Upvote 1
Link to comment
Share on other sites

  • Premium Member
5 hours ago, Todd Ruel said:

I helped them make a better product, and they returned the favor by reducing the price of a Diamant license by 50%.  Maybe you could do the same thing for them.

Still $10k isn't very feasible. We also rarely do restoration to a level where we need it. Resolve's tools are also getting better and we can also get MTI DRS Nova for $1000/month if we need something badly. The wet gate on the HDS+ does help enough, that Resolve can usually fill in the rest. 

5 hours ago, Todd Ruel said:

On the topic of HS-Art software, I just bought a Mac Studio Ultra, and rendering Dustbuster+ timelines on it is blistering fast!  It takes 1/3 the time it used to take on our PC workstation and about 50% less time than on my old iMac Pro.  If time equal money to you, then using these film restoration tools on a Mac Studio will save you even more time/make you even more money.  

Right, but the problem is our scanner is Windows only. We have a shared storage solution as well that's 10G. However, copying DPX files to the storage, can take overnight sometimes. So by the time we open it up on my Mac workstation (which will soon be an M2 Ultra Studio when it's released), it's been at least 12hrs since scan. The key would be to process on the same system we scan, on that super fast SSD raid we have directly connected internally. But as you pointed out, the Mac's are just faster! Currently our turn around time is very fast, it's how we stay in business. We're sometimes 24hrs turn around from raw unprocessed film, to finish scan handing off to the client. 

Edited by Tyler Purcell
Link to comment
Share on other sites

  • Premium Member
35 minutes ago, Daniel D. Teoli Jr. said:

Thanks for all the info Tyler. 

If you are having that much trouble getting a stable image with the HDS with new film...what are you getting with warped / shrunken film?

 

Works the same no matter what film you use. The stability issue is directly connected to the gate not squeezing the film at all. There is no friction on the film in the gate, so it can move left to right. 

Link to comment
Share on other sites

  • Premium Member
On 6/15/2022 at 10:00 PM, Tyler Purcell said:

Still $10k isn't very feasible. We also rarely do restoration to a level where we need it. Resolve's tools are also getting better and we can also get MTI DRS Nova for $1000/month if we need something badly. The wet gate on the HDS+ does help enough, that Resolve can usually fill in the rest. 

Right, but the problem is our scanner is Windows only. We have a shared storage solution as well that's 10G. However, copying DPX files to the storage, can take overnight sometimes. So by the time we open it up on my Mac workstation (which will soon be an M2 Ultra Studio when it's released), it's been at least 12hrs since scan. The key would be to process on the same system we scan, on that super fast SSD raid we have directly connected internally. But as you pointed out, the Mac's are just faster! Currently our turn around time is very fast, it's how we stay in business. We're sometimes 24hrs turn around from raw unprocessed film, to finish scan handing off to the client. 

I have often resolved this kind of problems by having TWO external raid arrays which have compatible connections for both mac and pc. Then I will just connect the whole raid to the different computer instead of transferring files back and forth. I used to often have 10tb or more of files to move between systems at a time so it was a no brainer to do it this way because it takes only about a minute to connect the raid to different system instead of waiting for a day for files to copy to another storage ?

File system not compatible with both mac and pc?  just use Paragon plugins in either one system to make it compatible and you are good to go ?   

Link to comment
Share on other sites

  • Premium Member
1 minute ago, aapo lettinen said:

I have often resolved this kind of problems by having TWO external raid arrays which have compatible connections for both mac and pc. Then I will just connect the whole raid to the different computer instead of transferring files back and forth. I used to often have 10tb or more of files to move between systems at a time so it was a no brainer to do it this way because it takes only about a minute to connect the raid to different system instead of waiting for a day for files to copy to another storage ?

File system not compatible with both mac and pc?  just use Paragon plugins in either one system to make it compatible and you are good to go ?   

this way you can also continue scanning with the system when you are processing the previous file because no time is wasted in file transfers or rendering. Just swap the raid connectors between computers and you can continue scanning when rendering the previous files. It doubles the capacity of the system

Link to comment
Share on other sites

  • Premium Member
4 hours ago, aapo lettinen said:

I have often resolved this kind of problems by having TWO external raid arrays which have compatible connections for both mac and pc

Not fast enough for the scanner sadly. We have many external arrays, best we can do is around 1500Mbps, the scanner needs 2500+ to work at any reasonable speed. It's a bug within the windows OS which isn't very fast opening and closing individual files.

Edited by Tyler Purcell
Link to comment
Share on other sites

On 6/12/2022 at 8:51 PM, Giray Izcan said:

It is not worth it. Shooting film is expensive as it is so you should stick with getting your scans done professionally. I guarantee you that you will not get to shoot real projects more than a handful of times a year so you might as well get it scanned properly. If you are Mr deep pockets then maybe you can shoot as much as you'd like but for average earthly beings it is expensive. Besides, if you have to think about cutting corners when it comes to scanning, I'm sorry but you have no business shooting on film. You'll be happier with digital and spend the money elsewhere like production design and rentals. Film only makes sense if there is budget in place where you can get all the coverage you need without cutting corners otherwise you are slave to the format. 

 

Yes, true. But artists have to dream and explore. Artists need room to work for growth and development. 

Joan%20Miro%20Artist%20File%20D.D.Teoli%

Joan Miro in studio...click to view.

 

RomareBeardenD.d.TeoliJr.A.c..jpg?cnt=0

Romare Bearden talked about the importance of being able to dream as an artist. In short, if you can't dream, you won't do much as a creative.

Inside New Yorks Art World Romare Bearden 1979 : D.D. Teoli Jr. A.C. : Free Download, Borrow, and Streaming : Internet Archive

Unfortunately for the broke bohemian, cine' film is one of the most money sucking arts you can get into. I love film,  but have deep regrets for ever getting into cine' film. You have to be able to do justice to your work and not just say 'hey it is film' and the project turns out shitty because of not being able to afford to do it right.

This is not to say, don't do a project unless it is 100% the best. But the tech should be 'good enough' to not distract from the subject matter.

 

 

 

Edited by Daniel D. Teoli Jr.
Link to comment
Share on other sites

On 6/14/2022 at 12:36 AM, Robert Houllahan said:

I have run allot of feature length films with small budgets through Cinelab over the years and there are ways to make it work even on very small budgets.

The cost of scanning has dropped allot which kind of offsets the increase in the cost of film stock.

Really depends allot on the subject matter and creativity of the people making the film.

Also frankly there is allot of fetishistic tendencies towards camera gear and what scanner and all kinds of stuff that does not necessarily show up on screen but people want on set. 

If the writing is good and the cast can act you can make a film with a olde aaton some vintage glass bubble gum and duct tape.

 

Robert, you should know about scanners. And have samples ready from various scanners for the film practitioners to review before ordering scans.

As far as gearheads?

Dunno, I'm not on set. But sure, you got camera fondlers, fanboys and gearheads of every variety. For me, I'm not much of a gearhead. I just want the finished project. I can appreciate fit, finish and construction. But in the end, all I want is the finished product and not a hard-on for the gear.

For instance, I'd be perfectly happy buying hi-grade or even decent scans of films to work with and not have to bother with the physical film and scanning. But, many times to need the experience of handling things like film to learn. You can't learn via shortcuts and be well rounded. And people may learn from the lusting and fondling of gear.

There have been times in the past where I bought a thing just to look at it, see how it worked and how it was built, along with taking photos of it. If you want to look at it...you have to buy one!

Edited by Daniel D. Teoli Jr.
Link to comment
Share on other sites

Tyler, it is very troubling they would send out the machine with a gate that scratches film. Do you think it was a QC fluke? 

That's basic shit right there. You got a brand new camera or scanner, it should not scratch the film.  

How did you recover from that loss?

Link to comment
Share on other sites

Todd, if you haven't done so, you should be making a blog about your work with Diamant. Showcasing options you can deliver with your stock footage / restoration gig. 

Same with Robert. He talked about making things work for low budget indie films. Well put out some of the budget plans to show how it works in black and white. 

I mean, you guys are in biz. You want biz. So, spell out all the options to get more biz and show how it can be done before someone gives up on it due to worries about costs. Breast feed them in info!

Edited by Daniel D. Teoli Jr.
Link to comment
Share on other sites

12 hours ago, Tyler Purcell said:

Works the same no matter what film you use. The stability issue is directly connected to the gate not squeezing the film at all. There is no friction on the film in the gate, so it can move left to right. 

 

Then it wont work for warped film. Sounds like a Retroscan gate where the film just runs between two edges...flat or warped. Warped films should be flattened in the gate or you can't get any steady focus.

Edited by Daniel D. Teoli Jr.
Link to comment
Share on other sites

1 hour ago, Tyler Purcell said:

Not fast enough for the scanner sadly. We have many external arrays, best we can do is around 1500Mbps, the scanner needs 2500+ to work at any reasonable speed. It's a bug within the windows OS which isn't very fast opening and closing individual files.

So how do you handle it?

Can you scan at slower speeds?  Will your cache keep the files until they can be saved to the drive or are some dropped?

 

Link to comment
Share on other sites

You would never think in a million years a scanner would scratch film. What a mess. People don't care what kind of product they put out nowadays.

Listen up...and I'm only going to tell you this once...

When you get some new piece of equipment that runs film, run some white poly leader through it first. Don't use acetate leader. Poly leader is very sticky from static electricity and easily picks up problems. Below is leader that was run through a projector with deteriorated rubber rollers. 

 

16mm-test-leader-damaged-by-projector-d-

Photo : D.D.Teoli Jr.

Acetate leader may be better for showing scratches and dents to film. This leader below happens to be poly, so it can also work.

 

16mm-projector-damage-to-film-d-d-teoli-

Photo : D.D.Teoli Jr.

 

I gave up on acetate leader. Just too crazy priced now. I got a few hundred feet left, but am mainly poly leader now. I like acetate leader best. Poly leady is one big dust magnet. In the dark, shine a flashlight on it sideways and see what I mean. Terrible. But acetate is only for the well financed cine' devote, if you are talking about thousands of reels of film like I have to deal with.

Edited by Daniel D. Teoli Jr.
  • Like 1
Link to comment
Share on other sites

  • Premium Member

Thanks for the tip! 

I actually have some black camera negative I run through everything first and I did on our scanner but I didn’t notice the damage upon inspection it was so fine. Our wet gate removed it instantly. 

Link to comment
Share on other sites

  • Premium Member
7 hours ago, Daniel D. Teoli Jr. said:

So how do you handle it?

Can you scan at slower speeds?  Will your cache keep the files until they can be saved to the drive or are some dropped?

 

Slower speeds yep, but then you’re making less money. 

Link to comment
Share on other sites

On 6/18/2022 at 11:29 AM, Tyler Purcell said:

Not fast enough for the scanner sadly. We have many external arrays, best we can do is around 1500Mbps, the scanner needs 2500+ to work at any reasonable speed. It's a bug within the windows OS which isn't very fast opening and closing individual files.

What format are you scanning to? 4k 16bit DPX (a ridiculous format to use, especially if you're scanning color neg) is about 1700MB/s at 24fps. We easily get those speeds on our SAN, which is home-built using off the shelf WD Red drives. But even with the network removed, one could get enough speed to do 4k DPX on a local array with just 4-5 drives in RAID 0 (and RAID0 is fine if the files are only going to be there temporarily).

There is no issue with Windows slowing down DPX sequences. You've brought this up before, but it's simply not the case. Back in Windows XP it was a problem yes, but hasn't been for many, many years. Working with a DPX sequence on an NTFS RAID 0 is pretty straightforward stuff and is done all the time, even at high bandwidth. We have local raids for caching in all of our Windows machines - some are NVME or SSD caches, but some are just simple 4x 7200RPM spinning disks. All are NTFS, and all can handle image sequences.

 

Link to comment
Share on other sites

  • Premium Member
3 hours ago, Perry Paolantonio said:

What format are you scanning to? 4k 16bit DPX (a ridiculous format to use, especially if you're scanning color neg) is about 1700MB/s at 24fps.

We mostly do 10 bit 4k DPX. We can also do 16 bit 4k Tiff as well. 

3 hours ago, Perry Paolantonio said:

We easily get those speeds on our SAN, which is home-built using off the shelf WD Red drives. But even with the network removed, one could get enough speed to do 4k DPX on a local array with just 4-5 drives in RAID 0 (and RAID0 is fine if the files are only going to be there temporarily).

I mean, I have a bunch of 5 disc and 4 disc arrays, mostly thunderbolt and some USB C. Max speed on them is around 750Mbps with Western digital Black drives and/or Seagate EXOS drives. They just aren't fast enough sadly. 

So we built an internal 4 disc SSD array, which gives is around 2500Mbps, that got our speed up to 12fps with 10 bit 4k DPX on the FF, which is nice. 

Part of the problem of going faster is multi-fold, part of it is the USB-C camera, the other part is the poorly written software. 

3 hours ago, Perry Paolantonio said:

There is no issue with Windows slowing down DPX sequences. You've brought this up before, but it's simply not the case.

Seeing as one of my close friends actually works for Microsoft and we've pushed the issues up the food chain, yes there is a bug with windows, even the most modern version. Obviously Film Fabriek told us about this issue AFTER we purchased, thinking the system would do advertised 24fps in 10 bit 4k DPX, which it won't. This problem shows up mostly in the Pro Tools and Avid community with lots of cache files. Avid and Pro Tools never really solved it, but they did a work around by having a maximum file count per folder. This helps, but slow-downs on Pro Tools and Avid on Windows are generally still attributed to this same bug. It's never an issue on Linux or Mac OS. 

So how do scanner companies get around this issue? Simple, they pre-cache in ram. When you're thinking about GPU based stabilization, it needs a place to do that processing and it uses the ram of the system. So the actual image off the camera is going directly to ram, being processed and the final file is dumped. This is not how the Film Fabriek works, it doesn't touch the ram at all. Each frame is directly written to disc, then verified. It's the verification process, which is an OS process, that's gumming up the works. 

Our Microsoft friend, spent hours here futzing with our system and we were able to see the problem first hand. He installed a little utility that along with task manager, showed system resources and why the drive's resources were exponentially increasing even though the FPS was stable. It's because there is NO ram buffer. The file verification process is done at the OS level similar to a "copy" and not back end as an already verified file drop. Where the right way to do it is actually take a buffered file from ram and dump it on the drive. This way, the drive speed, doesn't mean much, as long as you have a big buffer. Even a 64gb ram machine, can store A LOT of frames in memory and a decent GPU will have enough added buffer to write the file no problem. 

We can capture 4k CRI files on a single USB C SSD from a Blackmagic Cintel II scanner no problem. That's exactly how our system to work, but due to poor software coding, it just can't. 

 

  • Like 1
Link to comment
Share on other sites

1 minute ago, Tyler Purcell said:

We mostly do 10 bit 4k DPX. 

This is nowhere near the 2500MB/S you claim it requires. 10bit DPX needs about 1200MB/S, plus you want a bit of overhead. If you have RAIDs that can do 1500MB/S consistently (and I don't mean testing with BMD or AJA disk speed tools, because those barely generate enough data to get past the caches on most hard drives), then you should have no problem with this. 

3 minutes ago, Tyler Purcell said:

Part of the problem of going faster is multi-fold, part of it is the USB-C camera, the other part is the poorly written software. 

I think you're conflating things here. You started off talking about the disk speed needing to be 2500MB/s and that's not true. 1500 should be sufficient, though a bit tight.

Now you're talking about the speed of the camera. But the camera being USB-C. The camera almost certainly isn't outputting a DPX file, it's pumping out the RAW data and then the DPX is being made by the frame grabber (in this case, it seems that might be a software element rather than hardware. DPX isn't a format that machine vision cameras really understand. They can do things like pass the raw sensor data (which is pretty lightweight if it's not debayered yet), or they can convert to an RGB bitmap file and pass that back to the software. But not DPX.  And if you're looking at the specs for the camera on the camera manufacturer's site, the top speed they rate is always based on an 8 bit image. That's the industry standard for machine vision cameras - to use the 8bit speed as the advertised speed of the camera. So you need to dig deeper to find out what the camera's speed really is, for a higher bit depth image. That should be in the datasheet for that camera. 

9 minutes ago, Tyler Purcell said:

So how do scanner companies get around this issue? Simple, they pre-cache in ram. When you're thinking about GPU based stabilization, it needs a place to do that processing and it uses the ram of the system. So the actual image off the camera is going directly to ram, being processed and the final file is dumped. This is not how the Film Fabriek works, it doesn't touch the ram at all. Each frame is directly written to disc, then verified. It's the verification process, which is an OS process, that's gumming up the works. 

What you are describing here is not a bug in how the OS handles the data, it's that this is a very inefficient and inherently slow way to do the work. If it has to write the data to disk and then verify that data, it's Writing and reading a big file. That's going to slow things down by adding a bunch of unnecessary I/O.  But that's not the fault of the OS, that's just a matter of the way the scanner software is handling the files. There's no reason they can't do the verification of the data in memory and then write the file. GPU has nothing to do with it.

 

12 minutes ago, Tyler Purcell said:

We can capture 4k CRI files on a single USB C SSD from a Blackmagic Cintel II scanner no problem. That's exactly how our system to work, but due to poor software coding, it just can't. 

Since the camera in this system is a bayer sensor, you would probably be better off writing the raw data to the disk and then converting it from Resolve. 

 

What camera is used in the scanner you have? Exact manufacturer and model? It should be easy enough to tell in the System Information window in Windows, if you can't see it on the physical camera. 

Link to comment
Share on other sites

  • Premium Member
3 hours ago, Perry Paolantonio said:

What camera is used in the scanner you have? Exact manufacturer and model? It should be easy enough to tell in the System Information window in Windows, if you can't see it on the physical camera. 

Flir GS3-U3-123S6C-C

 

Link to comment
Share on other sites

30 minutes ago, Tyler Purcell said:

Flir GS3-U3-123S6C-C

There are a lot of things that could cause slowdowns, but my bet is the interface. 

That camera has a USB3.1 Gen1 interface, which is only 5Gb/s. (gigaBITs - note the small 'b'). That's 625MB/second, assuming you're getting full saturation, which you almost certainly are not, because it's USB. A 4k 12bit file is a bit over 18MB/frame. So the USB 3.1 connection can move 33 frames per second, theoretically, assuming it's an RGB image off the camera onto the disk.

I would bet you're getting maybe 50-60% of your available 5gbps bandwidth if everything is perfectly configured, which might get you near 20fps max in bursts. But if the PC is doing other stuff (even background stuff you're not aware of), that's going to affect performance too. That could include any image processing they're doing on the computer, whether that's GPU or CPU based. This is why professional scanners don't use USB cameras. To get 4k files off the camera to the PC at speed, you need a proper interface on a PCIe card: CameraLink (though nobody is really using this anymore), or Coaxpress, or 25GbE (5x faster than USB3.1 gen1). And you need a motherboard/CPU combo with enough bandwidth to allow all that data to move back and forth.

Also, that camera has a tiny image buffer onboard - it can only hold about 6-7 frames of 12bit 4k, so if the FPGA on the camera is doing any processing that could be a choke point as well. 

The problem isn't your disk speed.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...