VOP - Video Optical Printer - A Raspberry Pi based project for slit scans and step printing

Johan Malmsten · May 15

As the title suggests. I have, for the past few month's been tinkering away at a project where I use a Raspberry Pi 4B 4GB to control a Raspberry Pi Camera HQ and a HDMI monitor. I point the camera at the monitor and during long exposures I can move the flat image in the monitor in 3D space to make smeared images. And if you target an image file in storage that already exists. That results in a new exposure being added to it. Rince and repeat using an exposure sheet.

What you may recognize this as, is probably an emulation of how slit-scans and backlit animations were made. As in 2001 a Space Odyssey, The Superman 1978 title sequence and other examples. Only, instead of moving the camera, we move the image. Because the image can be moved in 3D space however we want with full repeatability without need for expensive and advanced motion control gear and software. Of course, it does however get limited by the resolution and bit-depth of the monitor (mine is 1920x1080 in standard 8 bpc), the black-levels of the monitor, and the smoothness of the smears need to take the refres rate of the monitor into account. But for the price of camera+monitor+Pi. It's good enough to tinker with for me.

Right now. It can do:

- 2 layers of images per exposure, The Projection Magazine and its Bipack. For the exposure, they are multiplied together to simulate two film strips laid ontop of each other. I am currently working on making a third layer possible so I can do PatternA in one image, Pattern B in another image and a Hold-out matte to do moire pattern animation in select areas.

- Both layers have the ability to ingest video to do simple step-printing.

- It has two modes, Simple Smear System (SSS) where you move the artworks around and it can make motion blurs that trail far longer than a 360 degree shutter allows. And MultiDimensional Smear (MDS) where each keyframe can describe its own path for the artwork to take that doesn't have to follow the main keyframes path. Classically that can make for fun transluscent extrusions for example.

- The projection side and camera side can both use single colored tinted virtual gels for colorizing kodalith-like artwork mattes.

- We can also put the system into a monochrome mode that runs the image render in monochrome, But it uses the camera side virtual gel to tint it in camera to still get that color for the exposure. Providing a cleaner image.

- There are also other fun things like virtual anamorphics and such. And more is being worked on, But I am a lone "developer" here so please be gentle.

- ... And since it's an optical path, we can mess with what the light does between the monitor and the camera lens.

It's all controlled with a web application that the Pi runs so you can access it through any chromium-compatible web browser in your network.

Pricing?

It's FOSS. AGPL 3.0 - Repo hosted on Codeberg: https://codeberg.org/jmalmsten-com/VOP

blog: https://www.jmalmsten.com/category/vop

youtube playlist:

Right now, it's not in a stable state. Documention is sparse and somewhat outdated. I just wanted to get this post up in case there's interest in this project and I am learning as I am doing things. Anyone tickled by the prospect will have to provide their own hardware.

I mean. I can't be the only one who's wanted to do essentially compositing the old needlessly hard way. 🙂

Brandon Paterno · May 16

simply excellent!! good to see more work being done in the film post-on-film space. perhaps cine film shines most as an expressive medium manipulated optically. OP’s are the enlargers of the motion film world, and it’s never been easier to make a nice one. optical printers are killer.

i’m working on a c++ mcu library for constructing film performance, camera and projecting hardware, so on the protocol side i’ll look into how to talk to ur software. my optical printer has independent stepped rail movements for the camera and the lens, and im sort of looking into queueing and tracking punch ins — i’m excited to play around with the 3d stuff you have here and key framing zooms for more granular rolling shutter emulations.

have you looked at mcopy? maybe some stuff on the film out side in that software could be applicable here, or vice versa.

Edited May 16 by Brandon Paterno

Johan Malmsten · May 16

I haven't really heard of the mcopy project. Had to look it up. Bookmarked the blog and the repo for later perusal. Looks like a very interesting project indeed. But as the V in VOP is Video. I haven't really seriously considered actual film on the camera side.

The things to watch out for there, I guess, would be the blacklevels of the monitor used for projection. If you aren't careful. The blacks get milky fast depending on the hardware as exposure times get longer and multiple exposure runs add up the photons. I mitigate it by measuring the black levels in the center of the TIFF frame and remapping the tonal range to make black actually black before compositing it with the latent tiff that may already exist. With actual film, that cannot be done, so you need a monitor with real black from the start. Like a CRT, Plasma, or OLED. In modern days, OLED would be the way to go, but it's been too expensive for me to upgrade to it. If you want good result on expensive film however, I don't think it can be worked around realistically.

Also. Right now, I haven't made a real signal out or API that can talk to cameras outside of the Pi Camera HQ. But I guess it shouldn't be too unreasonable to add in something that just says a generic "open shutter now", "close shutter now" along with "advance/reverse camera by X frames now" along with some kind of timings to allow physical film to move through the unknown camera. I'll have to look into what signals would be expected of mcopy or similar systems. If there are generic command that can be sent somehow that the camera side recognizes.

I'll add it to the issue tracker, but it'll be among the stretch goals beyond v1.0.0. And since I do not have a working motion picture camera, related hardware and money for film stock, processing and scanning, I'd have to rely on others on the internet for real world tests that could get expensive quick. But the current code is in python running on a Pi so you probably can tap into what is sent to the current camera system if you feel adventurous.

And, I should add. I barely know how my own code works. I put "developer" in quotation marks because I so far am very reliant on LLM help to put together things codewise. Most of the VOP is built with the help of Gemini, but I have transitioned over to Claude Chat as it's much better at working with code for me. I do try to look over everything I put in there. I don't personally trust Claude enough to let its full agentic side loose on my pi with Claude Code. And I am learning things as I go along. But if you have qualms about running AI generated code, I feel I need to be up front about it.

I have however wanted to look into stepper motors for a little bit of hardware manipulation of the camera and lens. Just to be able to move the camera in and out a little bit to fill the view of the sensor with the monitor image when full width isnt necessary, and doing repeatable defocusing for glow effects. Right now both are very manually hands on driven.

But... Mostly I'm just delighted to see that at least one person outside of my head finds the project interesting. And remember, until it reaches v1.0.0 I do not consider much of anything in it as holy, so signal paths and commands may be changing wildly between whims and releases. 🙂

Edited May 16 by Johan Malmsten

Johan Malmsten · May 16

I added the rudimentary API thing for communicating with film hardware to the issue tracker so I don't forget about it. And if you know python and have ideas how to do it, let me know. At first glance, I guess, maybe serial commands and/or something with GPIO pins?. It is an open source project after all, so I am open to suggestions and feedback and collaborations. Film cameras are outside of my budget as I mentioned in the above ramble, but others might be able to prioritise things I can't 🙂

https://codeberg.org/jmalmsten-com/VOP/issues/180

Brandon Paterno · May 16

ohh, i misunderstood. i thought you were using a film camera as a means to capture and process slit scans. i’ll take a look at the code later. existing printing tools use serial, sending out a command (8 bit unsigned integer) and then waiting for a confirmation of the completion of an action — the pi camera benefits from talking directly to the main board but id probably have to change some of those communications to see how it handles the slower call and response time.

Johan Malmsten · June 8

I just want to add in general (outside any potential film camera interface). That the third layer (calling it BiPack2) has been added and loading both projection side and camera side with video has been simplified. Along with fully implemented non square pixel aspect ratios (digital anamorphic squeeze in both projection and camera side) to optimize pixel counts, code to detect EDID information like monitor resolution to enable monitors of all sorts of resolutions and aspect ratios (I have upgraded my prototype setup to a 3:2 screen with 2160*1440)

There's also been a whole wood grain styled makeover using images from my own home lumber walls and tiling work in Krita. No AI has been allowed to touch images anywhere in this project. And if you don't like the color, these textures, along with everything else is styled with regular .css so you can tune it to the way you like.

I also added a calibration page with live view for focusing and lining up of camera and monitor.

There's also the ability to trigger ProRes444 renders and low bitrate h264 Workprints manually. So you can have a quick motion preview or a full video ready for your NLE of choice.

And there's a bunch more things added or changed.

I am planning on making a few videos showing the whole workflow in action. But I keep getting into ye olde Columbo mindset of "Oh, there's just this just one more thing" mentality. and as a 10 second 240 frame animation can easily take an hour to expose one pass, a livestream can get rather uneventful. Maybe I should rig up an old webcam to just capture the movement of the projection monitor for explainer videos and livestreams? Dammit. Another thing for the issue tracker.

Follow the development in the repo at Codeberg

https://codeberg.org/jmalmsten-com/VOP

And my blog at

https://www.https://www.jmalmsten.com/category/vop.com/

Edited June 8 by Johan Malmsten

Pilvari Pirtola · June 14

Can the resolution scale up how high?

One project I've been wanting to try is to shoot on S8 from a screen with resolution so high that film could not separate the pixels. Trying to play with the digital/analog look.

Do you show the different layers to camera at once or separately?

Johan Malmsten · June 14

1 hour ago, Pilvari Pirtola said:

Can the resolution scale up how high?

One project I've been wanting to try is to shoot on S8 from a screen with resolution so high that film could not separate the pixels. Trying to play with the digital/analog look.

Do you show the different layers to camera at once or separately?

As I said to Brandon before. Right now, I have no immediate plans to put in interfaces for film based cameras. It's not so much a matter of me wanting to or not. But more about pure personal expenditure limitations. I only have the means right now to develop and test for using the Pi Camera HQ. So it's right now limited to that one digital camera. The project however is open source and free, so if you know how to talk to your camera using intervallometer/animation boxes. Then feel free to go into the python code to add in that functionality in a way that doesn't break it for my digital camera system. Then do a pull request and I'll probably merge it in. Butdo know that any issues regarding that module, I'd direct to you. 🙂

But. More about the actual questions. Theoretically, I guess the resolution is mostly limited by the screen, cable and the circuits in the Pi. As long as the VOP system gets a valid EDID handshake it should be able to run it. Having googled around, the Pi 4 is a bit limited for UHD/2160P. By that, I mean, in terms of refreshrate. As the smear system is relying on virtual movement of the image during exposure. The refreshrate becomes a bottleneck for the smear smoothness. At UHD you may be limited to 30Hz instead of 60Hz which means you'll need to double the exposure times to get the same smooth smears. And with longer exposures you need to be careful with the blacklevels of the screen. Probably dialing in some NDs if the lens starts diffracting when high fstops are utilized.

For Super8. The resolution question is quite a controversial one. I, myself doubt you'd see much of stairsyepping even with a 1080p screen as the projection monitor the camera is pointed at. After the grain blurring is added that is and whatever the lens does to the details. At a certain point, the higher scan resolutions only get you finer grain detail rather than image details. If you use a1080p and zoom in to fill the frame you'd at max be around 1440x1080 screen resolution. If you get your hands on a 3:2 screen at 2160*1440 like I did and fill the screen you'd have 1920x1440 to play with. What would make a bigger difference is probably the distance between the screens pixels.

My suggestion there would be to do some tests with monitors yourself. Point your S8 camera at them to test out if you find the results adequate for your needs. Try out both filling the image and fitting it inside with effectively letterboxing and see how long exposures you can do before you see the image breaks. With multiple exposure jobs those black levels will start adding up.

On the question of layers. That's a yes... and no kind of answer I'll give:

The VOP system uses three layers (named Projection Mag, BiPack 1 and BiPack 2 for convenience, but they are all three functionally identical). They are positioned in their own 3D space, then rendered as 2D images and those rendered 2D images are multiplied together to finally be shown on the projection monitor. The effect is basically like stacking three strips of film or Kodalith cels. The pixels get darker depending on the values. They "block the projector bulbs light" like in a real Optical Printer, so to speak.

However. As the system is built around being able to add new exposures onto the same "latent" tiff indefinitely, you can run through a pass with one three layer combo. Then do another pass with another three layer combo... Rince and repeat until you get all the elements you need onto the TIFFs. The latent tiffs can only add more light. So if you want to make it darker. You need to first make the latent image in the cam mag inverted, burn in the darkness, then invert it again to get back so you can burn in light again.

Again. With multiple exposures using film as the recording media, you'll need to be careful with black levels as a projection screen that looks black on a 3 second exposure can go milky grey after 4 exposures as that adds up to 12 seconds. I cheat with my digital camera by pulling down the exposure in post to clip out the noise floor so that pixel values are 0 so 0.0*0.0 always = 0.

That's my rambly response.

Tldr:

Resolution depends on your hardware (I think 4K 60Hz could be easier on a Pi5 but my test rig is currently a Pi4). And how high resolution is enough is not a discussion I can afford to answer without some expensive tests. The EDID handshake SHOULD permit whatever the Pi can communicate with. But I have so far only tested with 1920x1080 and 2160x1440 at 60Hz.

The layers are multiplied together and you can burn in as many if them as you want into the latent TIFF sequence.

And I welcome code contributors. If you have the money to experiment with film cameras, go ahead, but I cannot verify things myself.

Brandon Paterno · June 14

6 hours ago, Pilvari Pirtola said:

Can the resolution scale up how high?

One project I've been wanting to try is to shoot on S8 from a screen with resolution so high that film could not separate the pixels. Trying to play with the digital/analog look.

Do you show the different layers to camera at once or separately?

pretty much any monitor above 1080P will get you there unless you're shooting with 50asa. resolution this resolution that but the crystal formation is a fairly fixed thing between sizes of film that you can't cheat. you could mess with your processing a bit to hide it a bit more (heavier agitation?). A friend of mine did titles for 16mm off of his MacBook Air screen and it looked excellent projected via an Eastman 25 so I think most laptops with a high resolution screen should work a treat. Though for motion you'd want to either hand single frame it all or use some sort of call and response method between computer and camera. mCopy and film out is the most straightforward way to do such a thing, despite its limitations and weight.

Serial controls, similar to those that mCopy uses, can probably be "vibe coded" (or whatever it's called) into this program. on the hardware end you need a basic motor or solenoid and a microcontroller that listens for a command and then confirms the completion of a shutter release. In VOP, the camera capture mechanism is pretty barebones as is, so for hardware control all the RPI-cam stuff in camera_hardware.py needs to be replaced with an equivalent serial command handler that sleeps based on whether or not a command has been received (pyserial-asyncio?).

Johan Malmsten · September 2

Just thought I'd bump the thread once to say that the project has moved from Codeberg to GitHub. So new link to the repo is at: https://github.com/jmalmsten/VOP

As for what has changed. Quite a few things. But top billing would be Animation Desk Mode (ADM) where you now can step through the exposure sheet one frame at a time and trigger the exposures manually. That way you can animate things between the camera and projection monitor. Think of it as a "Virtual Production" rig for 2D cels, cutouts, even claymation and other objects. Your imagination sets the limits.

Another fun addition is a filter sweep mode. With it you can during an exposure filter through the image with a Low Pass, Band Pass, or High Pass. The simplest result can be basically a 3D distortion effect based on height maps. Again. Only user imagination sets limits here (outside technical limits of course). You can use for example one image that has a depth map and another that has the hero lighting rendered to it. And if you run them together with the same amount of depth smear and a band pass. You'll have a pseudo 3D object projected from a single source.

And I am currently running through tests to see what I can get away with regarding projecting flat video frames and rephotographing them using the VOP. Apparently, if I use rec709 footage and bake it with additional gamma of 2.33 I can get the result to be fairly close after the VOP has crunched it back down. If anyone wants to do these things. I would recommend doing your own gamma correction tests to find out where your projection monitor lands. I cannot promise that a gamma of 2.33 is anywhere near good for your theoretical hardware. And... when doing video reprinting with the VOP, the result is a bit flickery. I am not completely sure if it's because the prototype is currently fully exposed and not in a black matte box. Or if I am having some issues with timings for the exposures themselves. Further testing is needed.

Top Technical Features (from the repo README.md)

16-bit Pipeline: All processing is done in 16-bit linear color space in TIFF sequences for maximum math accuracy and dynamic range.
Motion Blur: Move artwork on-screen in 3D space during exposure to create physical light smears.
Smear: Smear and extrude like an 80's title sequence.
Virtual Gels & BiPacks: Use digital mattes and color overlays to simulate traditional optical effects.
Three Layers Per Exposure: for each exposure, up to three layers of artwork can be used, they live in their own 3D space, and the 2D planes transforms are rendered into 2D images that are then multiplied together for exposure.
Multiple Exposures: To combine passes, the VOP uses LIME (see the wiki: https://github.com/jmalmsten/VOP/wiki/LIME).
Anamorphic Workflow: In order to maximize the pixels recorded on the camera, the VOP can use non-square pixels in it's math that can then be accounted for at playback or in your NLE/Compositor.
Animation Desk Mode (ADM): A rudimentary function set aimed to make the VOP able to be run manually frame by frame for shooting animations of things between the camera lens and the projection monitor.
Filter Sweep Mode - by doing a sweep of the brightness values of pixels as a mask during a smear, we can use depth-maps to get 3D displacement effects.
Video I/O: Video files can be uploaded to the VOP both on the Camera and Projector side. When a compatible video file is ingested using ffmpeg, it's converted into a sequence of TIFFs. And you can use that sequence with the step printing functionality of the VOP. When a job is finished, a low bitrate h264 workprint is generated for quick review, and a high quality ProRes4444 can be generated as well for easier post work.
Notifications: As a single job can take multiple hours, a ntfy server is included in the deployment script so you can subscribe to it inside your LAN with a free phone app and be notified when a job has finished or failed.