DIY 2K scanning

[Scot McPhie]

Working on something else I?ve found it?s relatively easy to move files around with simple PERL scripts which made me think about Work Printers and DIY telecine methods.

 

Imagine a set up like this - a high quality digital still camera is set up to take pics from a film - the still camera would be connected to a PC by a USB port - with it?s memory card appearing as a hard drive in the PC.

 

A controller programme tells the camera to take a pic and to then advance the film transport mechanism - this goes on for a hundred times or so (however many till the memory card is full) and then the PERL script is called to move the files off the memory card and on to a physical hard drive in the PC and the whole process starts again until length of film is fully telecined.

 

Scans could then be done at 2K or higher resolution - and once they?re all in the computer as a series of numbered stills they would be relatively easy to manipulate in Premiere or Photoshop or other applications.

 

Making the camera take the photo might be the hard thing - you would need some kind of mechanical method I guess - a camera with a remote control function might be the easiest, if such a thing exists.

 

Any thoughts on this whole process? I?d be quite interested in making one - and since I?d like to shoot alot on 16mm it would be worth my while to put a few thousand into it - it would definitely save in the long run.

 

 

A quick search on google for digital cameras and remote control revealed some interesting links - including some with gui interfaces

 

http://photopc.sourceforge.net/

http://www.yk.rim.or.jp/~mamo/Computer/DS-7/index-e.html

http://www.math.ualberta.ca/imaging/

 

 

 

Actually the more I look at it - the better it gets! heres a few pics from the last site

http://www.math.ualberta.ca/imaging/

plus the instructions

 

 

 

 

 

From http://www.math.ualberta.ca/imaging/photopc.html

PHOTOPC

Section: User Commands (1)

Updated: 15 Jan 2001

Index Return to Main Contents 

NAME

 

photopc - utility to control digital cameras based on Sierra Imaging firmware

 

 

SYNOPSIS

 

photopc [-h] [-V] [-v[v]] [-q] [-z] [-t] [-f timeformat] [-s <speed>] [-S <pseudo-speed>] [-l <device>] [-u] [command [params] ] ...

 

 

DESCRIPTION

 

This is a command line tool to manipulate digital still cameras that use certain control protocol, namely Agfa ePhoto line, Epson PhotoPC line, Olympus D-xxxL line, Sanyo and Nikon (at least CoolPix 9xx) cameras. It can set camera parameters, download and erase pictures, e.t.c. It was originally developed for Epson PhotoPC 500, now the author uses it with Olympus D-600L, results with other models may vary.

 

 

OPTIONS

 

-h

    This option produces a help screen of all options and commands.

 

-V

    This option prints software revision information.

 

-v

    This option enables verbose output. Using two -v options increases the level of verbosity. This is primarily meant for debugging.

 

-q

    This option sets quiet mode. If it is not specified, the program will show running download progress indicator.

 

-z

    This option sets no time-zone processing mode. Use this option if the camera's clock is (should be) set up shifted by the local timezone offset.

 

-s speed

    This option allows setting the data rate to use when communicating with the camera. Valid rates are 9600, 19200, 38400, 57600 and 115200 (the default).

 

-S speed

    On some UNIX systems with some serial hardware/driver combinations you need to "lie" to the driver about the communication speed that you want to use at startup and during the session. Physical startup speed is always 19200. E.g., if on your system speeds up to 19200 are translated one to one, but in order to use 57600 baud you have to specify 50 baud, and you want to run at 57600, you'll need to specify photopc -s57600 -S19200,50 <commands ...>

 

    Another example: if your driver will use physical speed four times as high as you told it to set, and you want to run at default 115200 physical speed, specify photopc -S4800,28800 <commands ...>

 

-t

    Set timestamp of generated files to the time of snapshot.

 

-f timeformat

    This option allows to specify the format for automatically generated image file names; the value is used as the format string for strftime (3) function. By default, names of the format "MMDD_NNN.jpg" are generated, where MM is a month, DD is a day, and NNN is a count starting from 001. A good starting point for the timeformat value could be "%Y_%m_%d-%H_%M_%S.jpg". If the format string contains context "%%XXXd" where XXX are digits, it is taken for the count, i.e. if the file with such name is present, next count is tried. If the format string does not contain "%%XXXd" part, the count field, if necessary, is appended at the end of the name.

 

    A few single-character formats are used as shortcuts. "x" makes the name of seconds since the epoch in hexadecimal format (8 characters + ".jpg"), "d" - the same in decimal format (10 characters), "1" stands for default format "%m%d_%%03d.jpg", "2" stands for "%y%m%d%%02d.jpg", "3" stands for "%Y_%m_%d-%H_%M_%S.jpg" and 4 stands for "%Y/%m/%d/%H_%M_%S.jpg". The program attempts to create intermediate directories if necessary (and possible).

 

-l device

    This allows specifying the serial device to use for communicating with the camera. The default is /dev/photopc (which is typically created as a (symbolic) link to the real device, e.g. /dev/ttyS0) in UNIX, and COM1: in DOS and Windows. Note that the program does not use UUCP style locks so you need to be certain that no other program, such as uugetty, uucico or mouse manager, is trying to access the same device.

 

-u

    Use Universal Serial Bus (USB) if available.

 

 

COMMANDS

 

After the options, one or more commands can be given in the command line.

 

Some commands accept frame-list as a parameter. Frame list is a comma-separated list of tokens (no whitespace allowed), each of the tokens being a word all, or a single number, or two numbers separated by a dash (-), or a number followed by a dash and a word last (without whitespace).

 

query

    Display all current camera settings.

 

count

    Output the number of pictures stored in the camera. This is intended for use in shell scripts.

 

list

    List information about frames stored in the camera. Fields displayed are: frame number, size of image file, protection state, resolution mode, timestamp and filename if available. Faster than llist but may not work with some (older) models.

 

    WARNING: this command changes current directory on models that support directory tree on the flash media.

 

llist

    List more information about frames stored in the camera. Fields displayed are: frame number, protection state, size of image file, resolution mode, flash, shutter and timestamp. Slower than list and does not work with models that use Exif file format.

 

    WARNING: this command changes current directory on models that support directory tree on the flash media.

 

cd <path>

    On the models that support multiple directories on the flash media, change current directory as specified.

 

    The program chdir's to the first non-empty folder at the beginning. So, your first cd command must specify full path starting with a slash. If it where unix you could also cd ../something but here we do not have a way to move up one level.

 

protect <frame-list> On | Off

    Change "protected" status of frames. Frame numbering starts at 1. The command will fail if the frame number is omitted, not numeric, or does not correspond to a stored frame. "Protected" frames cannot be erased with "erase" and "eraseall" commands or with the "erase" button on the camera.

 

erase <frame-list>

    Erase frames. Frame numbering starts at 1. The command will fail if the frame number is omitted, not numeric, or does not correspond to a stored frame.

 

    To avoid unexpected results, list of frames specified for this command is sorted and processed in reverse order.

 

eraseall

    Erase all stored frames.

 

eraselast

    Erase most recently taken frame.

 

snapshot

    Take a picture.

 

off

    Switch off the camera. This command only has this effect on some models (notably Olympus D-600L). On other models, it just terminates the session immediately (otherwise the camera would be active several seconds after the last command). On models with big timeout, such as Photopc 600, it may be wise to always issue "off" at the end of command row, to decrease power consumption.

 

    All commands going after "off" are ignored.

 

image <frame-list> <file-or-dir-name>

    Download an image or series of images from the camera to disk file(s) on the host computer. Frame numbering starts at 1. The command will fail if the frame number is omitted, not numeric, or does not correspond to a stored frame. Files are stored in JPEG (JFIF or Exif) format. Image file also contains a thumbnail portion, and set of parameters like the shutter speed, date and time, I.D. string and more. Directory must be specified if frame list is requested. If directory name is specified, downloaded file(s) will get unique names starting with month and date of snapshot, unless -f option is used, in which case unique names according to -f option setting.

 

thumbnail <frame-list> <file-or-dir-name>

    Download a thumbnail image or series of thumbnails from the camera to disk file(s) on the host computer. Frame numbering starts at 1. The command will fail if the frame number is omitted, not numeric, or does not correspond to a stored frame. Files are stored in JPEG (JFIF or Exif) format. If the camera does not keep files in EXIF format, then the thumbnail file also contains a set of parameters like the shutter speed, date and time, I.D. string and more. Directory must be specified if frame list is requested. If directory name is specified, downloaded file(s) will get unique names according to -f option setting.

 

audio <frame-list> <file-or-dir-name>

    Download an audio notice file or series of files from the camera to disk file(s) on the host computer. Frame numbering starts at 1. The command will fail if the frame number is omitted, not numeric, or does not correspond to a stored frame. Files are stored with ".wav" extention but the format is not RIFF WAV format. It is compressed in some way, probably with some variation of ADPCM. The program stores the raw data as provided by the camera, it is not playable as a normal .wav file. Directory must be specified if frame list is requested. If directory name is specified, downloaded file(s) will get unique names according to -f option setting.

 

preview <file-name>

    take a `preview' snapshot and save image as a <file-name>. File is in JPEG (JFIF or Exif) format, and does not contain any additional data, just plain (very low resolution) image.

 

upload <file-name>

    Upload image file into the camera. This command is not enabled in LOWMEMORY version (i.e. under MS/DOS).

 

    WARNING! You cannot upload arbitrary JPEG file. Depending on the camera model, such attempt may result in the image not uploaded, not viewable, or even the camera "hung" needing to remove the batteries to make it work again. File previously downloaded from the same camera model is always uploadable. Otherwise, your milage may vary. photopc program does not perform any checks, so it's up to the user to provide the file in proper format.

 

mode Record | Play | Thumbnail

    Set specified operation mode on certain models.

 

resolution Hi | Lo | Ext | <number>

    Set the camera to high, low or extended resolution mode. Exact meaning of the modes is model-dependant and may specify different image dimensions and/or JPEG compression rate.

 

clock

    Set the camera's internal real-time clock using the current date and time from the host computer. Time of taking a snapshot is included inside the JPEG files produced. Note that the models which display the time on the LCD will then display GMT rather than local time. With -z option, time shifted by the timezone offset is set, and the LCD will display local time. Using unshifted (GMT) time in the camera seems a proper way to go.

 

shutter <speed>

    Set the camera shutter speed. Units can be specified in microseconds or in traditional form like "1/250". Valid interval depends on the model. If a value outside the interval is specified, the camera will set the speed to the nearest margin of the valid interval. Special speed value auto will set automatic shutter. Note that this only works while the camera is connected to the host, and automatically turns the flash off. While disconnected, shutter speed is always set automatically.

 

aperture Auto | Lo | Med | Hi

    Set the lense aperture on models that support it.

 

flash Auto | Force | Off | AntiRedeye | SlowSync

    Set the camera flash mode.

 

id <string>

    Set the camera identifier string that is included inside the JPEG files produced. Only few models support this.

 

autoshut-host <seconds>

    Automatically shut off the camera after that many seconds of inactivity while connected to the computer (on the host).

 

autoshut-field <seconds>

    Automatically shut off the camera after that many seconds of inactivity while disconnected from the computer (in the field).

 

lcd-autoshut <seconds>

    Automatically shut off the LCD monitor after that many seconds of inactivity.

 

lcd-brightness <value>

    Set LCD brightness to the value (valid values are 1 to 7).

 

focus Macro | Normal | Infinity

    Set the lens focusing mode. As far as I can tell, on fixed focus models macro only means force minimum aperture, not change the focal length. Only has effect in the same invocation with "snapshot" command.

 

adjust Normal | Contrast+ | Contrast- | Brightness+ | Brightness-

    Set adjustment of picture brightness/contrast.

 

metering Center-weighted | Spot | Matrix

    Set auto exposure metering mode.

 

whitebalance Normal | Sunny | Incandescent | Fluorescent | Flash | Preset | Cloudy

    Set white balance.

 

color On | Off

    Set color or B/W mode.

 

dzoom On | Off | <number>

    Set digital zoom on or off. Note that low resolution must be set prior to turning on digital zoom. Numeric values have model-dependant meaning. Only has effect in the same invocation with "snapshot" command.

 

zoom <mm>

    Set optical zoom value on models that support it.

 

 

EXAMPLES

 

% photopc query

 

% photopc -f "image%H:%M:%S" eraseall snapshot image 1 .

 

 

OTHER NOTES

 

The latest version of this tool can be found at ftp://ftp.average.org/pub/photopc/; see also WWW page at http://www.average.org/digicam/.

 

Trademarks are properties of their respective owners.

 

 

KNOWN BUGS

 

On some models, powersaving timeouts are measured not in seconds but in 1/10 sec units. The program reports them as in seconds.

 

Llist command relies on the format of thumbnail image that does not contain necessary information on some models (e.g. Nikon CP 900), this command does not work for these models. List command, on the contrary, do not display useful data on models that do not return it in the register 47 (e.g. Agfa 780).

 

The program is known to have some timing problems talking to Nikon CP900 when the camera is "napping" in powersaving mode. Until the problem is solved, a workaround is to run the program this way:

 

$ photopc && photopc <command list>

 

Nikon seem to have some other problems too, especially when doing "actions" (e.g. snapshots, erase).

 

Another Nikon problem: if 32Mb (or bigger?) memry card is inserted, the program cannot get the number of frames taken, and if "all" or "NN-last" range is specified for downloading, the program fails. Specify explicit range of frames. Also, with any card, number of frames taken is not updated immediately after the "snapshot" command (the same has been reported for other models). If you find how to find the correct number of frames, please let me know.

 

 

COPYRIGHT

 

The program is copyrighted by Eugene G. Crosser <crosser@average.org> and freely distributable. See README file for details. DOS and Windows support copyright Bruce D. Lightner <lightner@lightner.net>

 

 

CREDITS

 

The man page was originally written by Jeff Tranter <jeff_tranter@pobox.com>, who also contributed a few bugfixes. Some information about the serial protocol obtained from Jun-ichiro "itojun" Itoh <itojun@itojun.org>, Thierry Bousch <bousch%linotte.uucp@topo.math.u-psud.fr> and other contributors.

 

 

SEE ALSO

 

epinfo(1), protocol description. protocol description.

 

 

Index

 

NAME

SYNOPSIS

DESCRIPTION

OPTIONS

COMMANDS

EXAMPLES

OTHER NOTES

KNOWN BUGS

COPYRIGHT

CREDITS

SEE ALSO

 

o This document was created by man2html, using the manual pages.

Time: 01:43:19 GMT, January 26, 2001

 

 

The relevance for telecine work is quite obvious - although it appears this camera may only take jpeg's though

 

Scot

[GeorgeSelinsky]

Some problems to think about, even if you solve the mechanical and software issues:

 

1) Time it takes to scan each frame. You have to not only think about the time to take the picture but to process it into the computer and into the file.

 

2) The image quality. A CCD/CMOS based digital still camera isn't going to capture all the range that a $20,000 photo tube can, and I question what some manufacturers mean by "2K resolution". How 2K is it? Then what is the quality of the optics that are being employed to grab this image?

 

3) Storage space. 2K takes up a lot of room.

 

and finally...

 

4) Output. Now that you have all this great data from a device you've built yourself, you're going to do what with it? Make an HD file? That's cool I guess, if you can get the software to agree on everything. But you're certainly not going to make your own laser film recorder (although I have no idea, maybe you can?).

 

There is a guy who made a Super 8 and 16mm home telecine projector that advances frame by frame and lets a DV camera of your choice do the capturing, frame by frame. He wrote software that integrates into Premiere. His website I think is http://www.moviestuff.tv

 

Good luck anyway,

 

- G.

[DanStewart]

This is more like it -

 

http://www.jkcamera.com/digital_printer.htm

 

Use a nice fast high quality Nikon tethered to a PC, and use Nikon Capture software to record directly to the PC hard disk. In any case get a DSLR with a nice macro lens - for a list of cameras and software control options:

 

http://www.steves-digicams.com/

 

Jeff Kreines hypothesised this could achieve 1fps @ oversampled 2k.

I'd be interested in how it works out if you try it.

 

Cheers

Dan

[Kevin Zanit]

You would be better off using a flatbed scanner with a photo adapter.

 

It just so happens that most flatbeds are 24 frames (of 16mm) wide.

 

I read somewhere about a guy who did this and used an old Dot matrix printer for the film advancement.

 

Talking to the printer port as a lot easier than talking to USB. The same goes for working with TWAIN.

 

Then simply write an Algorithm to detect the frame lines and cut up each frame to a single image (this guy had the software on line, cant find it though).

 

Registration will still be a problem, so a stabilizing program would need to be run.

 

 

Kevin Zanit

[GeorgeSelinsky]

You would be better off using a flatbed scanner with a photo adapter.

I don't think that's a good idea. Number one, it takes really long to scan. Number two, the scanning mechanism must make each trip and with motion picture film, it's going to get a LOT of use, to the point it may wear out. Number three, the quality of scanner optics from what I've heard and seen is low. I scanned a frame of 35mm off of a positive contact print using a fairly decent Microtek scanner. I was unpleasantly surprised to see a lot of color bleeding in and noise. I downrezed the file from 2K to NTSC size and compared it to a frame grabbed off of a telecine transfer done to mini DV (from a Rank). While it's a bit unfair to compare a negative to a positive transfer (so I'd have to disregard the extended shadow range), I still found the rank to be superior in color reproduction - even in a 4:1:1 color space. Like I said, if a $400 device equalled or surpassed a $20,000 phototube, everyone would be doing it.

 

It's only half the battle to get the resolution, the rest of it is color reproduction, dynamic range, and eliminating noise/distortion. Especially in the case of the latter, it's not much use to have 2K resolution if there is noise and various chromatic issues.

 

Still, I think that it's a worthwhile option to explore and play with. I'm interested in any developments.

 

- G.

[Luke Prendergast]

Most of the optical issues could be avoided with a direct contact scan, but there would be film wear concerns. A standard 16 frame is just slightly bigger than a 2/3" chip, as is super8 to 1/3". You might get all of a Super16 frame onto a 1" chip. As for color fidelity, a red, a green and a blue exposure for each frame would be the way to go. 75Mbit using DV25 compression over firewire?

Consider that a video camera hooked to a PC via firewire will quite happily throughput 25 (30) frames a second so data transfer for SD should be a non-issue. Premiere captures full progressive stills in single-frame or time-lapse modes, not at real-time frame rates, but certainly many per second. Just need to drill a hole in your keyboard and add a trigger from your intermittent movement telling it's ready for another frame grab...

 

PS - I haven't tried it yet

[Luke Prendergast]

Here's the CCD vs film frame size comparison

[Kevin Zanit]

That is why I said he had to add a film scanning add on to the scanner.

 

It magnifies the image slightly, and back lights it. This lets the scanner work with the film much easier.

 

I feel pretty confident that this would look better than re-photographing it with a digital still camera.

 

Also, if time is of the essence then I really dont think anything you home make is a good idea.

 

That said, I have seen demos of people that made the thing based on a scanner it it looked fantastic.

 

 

Kevin Zanit

[GeorgeSelinsky]

Most of the optical issues could be avoided with a direct contact scan

Well, there is no such thing as a real "contact scan", i.e. something like a contact print. The CCD element would have to be physically in contact with the emulsion side of the film for that to happen, and we can see that this is obviously not a realistic option. From what I understand scanners also have lenses that focus the image of what they are scanning onto the linear CCD. Those linear CCD's are black and white, one for each primary color. They are not by any means expensive or for that matter as accurate in their reproduction as a photo-electron tube. They exhibit more noise and this will be a serious issue where you have a dense area in the negative. There is also the possibility of more noise in one color channel than the other.

 

The data transfer rate is an issue with 2K scans. Try moving around uncompressed 2K video files on your computer, you'll see exactly what I'm talking about.

 

Concerning time, what's the point of spending the time on creating such a machine if it takes forever to scan your negative? I mean, if it takes 15 seconds to scan each frame and write it to disk, that's one hell of a time eater. You're going to have to leave the machine alone and that's one computer you're not going to be able to use for a while.

 

Anyway, just some thoughts. I can't imagine that someone couldn't invent an interesting and flexible DIY telecine device. If the Russians could invent the PPSh submachine gun in the heat of warfare, using preexisting parts for Mosin nagant rifles, then it's possible.

[DanStewart]

Which made slightly less noise than a 2c :P

 

It seems to me you need a steady movement, lens, sensor and fast disk.

 

Do you really think you're going to do better than a 35mm size sensor in a DSLR, nikor macro lens and optical printer movement combo? I think you're getting into some fairly advanced engineering to compete with those guys. A SCSI RAID can handle the data, though the camera won't do better than 1fps @ 2k. For faster - build more.

 

Of course its way short of a proper film scanner but then you're talking about competing with the big boys - but do that for less than 100k/unit and you're in business...

 

Cheers

Dan

[Scot McPhie]

The real attraction in something like this was to save money - but the time factor is quite a worry - I did a few calculations and this practically negates it's usefulness:

 

200 feet of 16mm is about 5.5 minutes (can't remember exactly but that will do for now)

 

If shooting at 24 fps

 

5.5 mins x 60 secs = 330 seconds

 

330 x 24 = 7920 frames

 

Now if it takes 5 seconds to take the picture, advance the film and get the picture off the camera onto the hard drive

 

7920 x 5 = 39600 seconds to scan 200ft of 16mm

 

39600 / 60 = 660 minutes

 

660 /60 = 11 hours

 

Kind of daunting!

 

Now if you were shooting a 90 minute feature film with a 4:1 ratio

 

that's 360 minutes

 

360/5.5 = 65.454545

 

65.45 x 11 hours = 719 hours

 

or 29.99 days of continuous use

 

 

 

Rather than try and save money and make one of these - the effort's probably better put into raising money for a proper transfer!

 

Scot

 

 

(Correct me if any of my maths is wrong!)

[Phil Rhodes]

Hi,

 

Sure, but it would cost many thousands of pounds to have that transfer done, and I don't make many thousands of pounds a month even if you sit there and stare at it!

 

Phil

[GeorgeSelinsky]

While I know that it's often a fruitless task to try and make all of your film equipment, I can understand the desire tohave your own gear.

 

For those who are primarily cinematographers, the situation is different. You're relying on someone else to provide the tools for your job, and the money ofcourse is their problem (after all, you're working for hire in most cases). It's all the more natural to desire the best tools for the job.

 

But if you're a filmmaker and you're self financed, like I am for example, every savings does count. Of course there are foolish ways of "saving" money, but sometimes the only way to get a project do ne on your own resources is to take certain short cuts, i.e. shooting on short ends. If I did things "properly", shoot on fresh stock, shoot in sync, etc. I seriously doubt I'd get even half of my film shot on Super 16 with the resources I have.

 

It's easy to tell someone "just raise the money" but some people's ideas, like mine for instance, are aimed at a small audience and raising money for such projects isn't easy. Plus, it's nice as a filmmaker to have a sayin what you do instead of worrying about decisions by committee.

 

If someone told me "sure, here's a supervised spirit transfer and film dailies, now how about making the female lead my cousin's daughter, who's going to be a terrific actress... ", I'd start looking at a filmchain wi th some respect again. Sure ,the DP will be happier with the spirit transfer and film dailies though - and when I'm in the position of that DP I completely understand.

 

The key thing is to have the proper expectations and understand when to draw the line. You can't expect to build a telecine device for under $1000 that will compete with the latest six figure machine developed by a huge team of engineers. Otherwise everyone would be doing it. There are also foolish things like trying to shoot a feature with a windup 16mm magazine camera. Sure it's cheaper, but how much more is an Arri St when you're spending thousands already on film? If you're going to make a 2K scanning device, and scan your film and then take it to an expensive laser recording facility, are you going to get acceptable results? Etc...

 

If the possibilities are reasonable to achive what you set out to do, I say go for it - as long as you have the time and energy, and have reasonable expectations and are well aware of the obstacles you have.

 

- G.

[Marc Wielage]

In addition to the comments from others, there are a lot of problems with your idea that I don't think you take into consideration:

 

1) keeping the film clean as it gets scan; just getting an ultrasonic cleaner is more trouble than you can imagine.

 

2) color space, LUT specifications, and a 1-light setting that retains the full dynamic range of the film. Some of these are jealously guarded by companies in the scanning business.

 

3) the need for a large, fast drive array capable of dealing with 13 megs a frame (for a 2K file), which is about 400 gigs per 20-minute reel of negative. You also have to be able to backup these files, do fixes (when necessary), and delivery the files in different file formats to different companies.

 

4) the ability to digitally conform the negative to match an EDL.

 

Scanning negative can be a nightmare of complexity, because there's so many obstacles to hurdle. If it were easy, everybody would do it, and it'd be fast and cheap (which it isn't -- yet). I suspect by the time negative scanning becomes dirt-cheap, half of all features will be shot on a digital format.

 

--Marc W.