Jump to content

Mechanics of digital video capture.


Bob Speziale

Recommended Posts

For those of you with technical knowledge, maybe you can answer this question. How does a DSLR with a 16 or 20 or 50MP sensor capture 2MP digital video frames? Are most of the pixels turned off? Are the 2MP images captured by the same pixels on every frame, or after each frame is captured are another 2MP on the sensor used? Or is the whole sensor used used and the frames then downsized? Or something else? On my BMPCC original camera, it has a 2MP sensor, so the same pixels get used for every frame. But how about on a 16MP , 21Mp or even 50MP DSLR?

Link to comment
Share on other sites

Early DLSR's like the 5D M2 used line skipping and or pixel binning - resulting in issues such as aliasing or moire. 

These days cameras are better able to do in camera mathematical scaling or record at higher resolutions allowing a 1:1 ratio between sensor photo-site and recorded pixel

Link to comment
Share on other sites

So it's just the 2MP in the center of the sensor that is used, with all the other pixels turned off? But isn't the lens projecting the image over the whole sensor (or most of it if the video is cropped a bit to 16x9)? But if most of the sensor is used, is the image downsized by the camera to 2MP before the frame is captured?

Edited by Bob Speziale
Link to comment
Share on other sites

It's complicated and inconsistent and even manufacturers won't say exactly what they're doing.

Some cameras might be cropping in, others might be skipping pixels, others might be binning pixels, others might be reading out a lower quality signal of the full sensor, and then they're all scaling up or down to get to 2MP, like you'd scale in Photoshop.

It's worth knowing what's what so you can anticipate which cameras do well at what. You might get a sharper image, more dynamic range, or more aliasing with one method compared with another. But getting too caught up in that hasn't served me well, just catered to my curiosity. 

Link to comment
Share on other sites

22 minutes ago, M Joel W said:

It's complicated and inconsistent and even manufacturers won't say exactly what they're doing.

Some cameras might be cropping in, others might be skipping pixels, others might be binning pixels, others might be reading out a lower quality signal of the full sensor, and then they're all scaling up or down to get to 2MP, like you'd scale in Photoshop.

It's worth knowing what's what so you can anticipate which cameras do well at what. You might get a sharper image, more dynamic range, or more aliasing with one method compared with another. But getting too caught up in that hasn't served me well, just catered to my curiosity. 

I was just curious because I couldn't find any easy answers anywhere. I know my 50MP 645Z and my 2MP BMPCC both shoot 1080P video. The BMPCC image seems the best.

 

Link to comment
Share on other sites

Quote

Some cameras might be cropping in, others might be skipping pixels, others might be binning pixels, others might be reading out a lower quality signal of the full sensor, and then they're all scaling up or down to get to 2MP

Some cameras even use different strategies depending on the frame rate, e.g. using the full sensor width with a high quality downsizing algorithm at lower frame rates (24-60P), then switching to a mix of lower quality or sensor cropping algorithm to reduce CPU workload at higher frame rates (120P and above). It really depends on the model. With the increase of embedded horsepower, using the full width of the sensor seems to become more and more common amongst recent DSLR and mirror-less.

Quote

The BMPCC image seems the best.

Well, "best" in terms of what ?

Edited by Nicolas POISSON
Link to comment
Share on other sites

Reading reviews here and there, it seems that the 645Z was already not that great for video when it was released in 2014. It has a medium format sensor and crops only a little. So the area used for video remains very wide. But there is much more to image quality than the algorithm used to output 2K video from a high resolution sensor. Video on DSLR and mirror-less has improved a lot in the recent years. So it is not really surprising that an "old" dedicated video camera delivers better image than an "old" medium format camera.

Link to comment
Share on other sites

7 hours ago, Nicolas POISSON said:

Some cameras even use different strategies depending on the frame rate, e.g. using the full sensor width with a high quality downsizing algorithm at lower frame rates (24-60P), then switching to a mix of lower quality or sensor cropping algorithm to reduce CPU workload at higher frame rates (120P and above). It really depends on the model. With the increase of embedded horsepower, using the full width of the sensor seems to become more and more common amongst recent DSLR and mirror-less.

Well, "best" in terms of what ?

The BMPCC shoots 12 bit raw dng files which contain about 10x the data that downsized dslr video frames do. To me it just looks better. Here's a single frame from a video I shot at iso-1600 in daylight coming through blinds at f3.5. Here's a single frame (downsized to post here) and a crop of that frame. Colors and skin tones just look better to me.

I was thinking this over last night and realized dslr's usually offer a choice of image size, high, medium and low quality images that will reduce the file size, eg. for emailing. It seems likely that this may be the same process used to ceate the 2MP digital video frames, whatever that process is.

bob-desk-rs.jpg

bob-2-2020.jpg

Edited by Bob Speziale
Link to comment
Share on other sites

  • 2 weeks later...

I would not categorize all DSLR in a single set and make generic conclusions based on two models that are 6 years old. All I can say is that the 645Z delivering lower quality video than a BMPCC is not really surprising. Concerning the images you posted, well I cannot tell which camera has better skin tones since there is only one sample. But I would not link skin tones to any debate like pixel binning vs line skipping.

Quote

dslr's usually offer a choice of image size, high, medium and low quality images that will reduce the file size, eg. for emailing. It seems likely that this may be the same process used to create the 2MP digital video frames, whatever that process is.

I do not understand what you mean. Whatever video device you use, there is always a compromise between the amount of data, image size, frame rate, compression ratio, and image quality. That is true for a smartphone as well as an Alexa. But this is a different story than the algorithm used to create a 2K stream from a high resolution sensor.

Link to comment
Share on other sites

On 8/5/2020 at 3:45 AM, Nicolas POISSON said:

I would not categorize all DSLR in a single set and make generic conclusions based on two models that are 6 years old. All I can say is that the 645Z delivering lower quality video than a BMPCC is not really surprising. Concerning the images you posted, well I cannot tell which camera has better skin tones since there is only one sample. But I would not link skin tones to any debate like pixel binning vs line skipping.

I do not understand what you mean. Whatever video device you use, there is always a compromise between the amount of data, image size, frame rate, compression ratio, and image quality. That is true for a smartphone as well as an Alexa. But this is a different story than the algorithm used to create a 2K stream from a high resolution sensor.

My original question was what is that algorithm? DSLRs have been creating HD video streams for about a decade but I haven't seen any articles on how they do it. Do they just use 2MP of the sensor and tuen off the others, or do they use 2MP of the sensor and rotate to another 2MP for the next frame, or do they use the whole sensor and downsize the image to 2MP?

Link to comment
Share on other sites

  • Premium Member

It varies by camera, some pixel bin, some line skip, a high-end camera would downsample (but I'm not sure many did that because of the processing power needed), some window the sensor, and sometimes it was a combination of windowing, pixel binning, etc.  Some sensor cropping is always involved since most still cameras have a 3:2 sensor, not a 16:9 sensor, some do a lot of cropping first (like from full-frame to APS-C area) others crop a little so that the math works out better for whatever process they are doing to get down to HD.

 

Link to comment
Share on other sites

1 hour ago, David Mullen ASC said:

It varies by camera, some pixel bin, some line skip, a high-end camera would downsample (but I'm not sure many did that because of the processing power needed), some window the sensor, and sometimes it was a combination of windowing, pixel binning, etc.  Some sensor cropping is always involved since most still cameras have a 3:2 sensor, not a 16:9 sensor, some do a lot of cropping first (like from full-frame to APS-C area) others crop a little so that the math works out better for whatever process they are doing to get down to HD.

 

Thank you David. Are there any published articles or video on the process? I was surprised that this was one subject where Google failed me completely.

Link to comment
Share on other sites

53 minutes ago, David Mullen ASC said:

I’d keep Googling, that’s how I would look it up. Some reviews of cameras discuss how they are generating their video, as well as user forums.

Problem with google is it will bring up a million hits related to dslr video specs, nothing on how the video actually records, other than mentioning the crop factor (eg. full frame video on some models, or the video crop factor). That is, they will describe the output, all aimed at potential buyers and users,  but nothing on how the output gets produced. This post is the only time I have ever seen the actual mechanics of creating a 2MP video frame discussed anywhere. Maybe it's some sort of trade secret. 

Edited by Bob Speziale
Link to comment
Share on other sites

  • Premium Member

You just have to search deeper into subjects like pixel binning, for example searching "pixel binning canon r5" got me this:
https://www.eoshd.com/8k/a-closer-look-at-the-canon-eos-r5-lower-quality-4k-mode-to-avoid-thermal-cut-off/#:~:text=In order to achieve the,60 megapixel Sony a7R IV.

Link to comment
Share on other sites

Thank you David. So the magic words for google and youtube are pixel binning for video and line skipping for video. Much appreciated. The article and video answer my questions. Your original answer and some of the answers in the thread were above my knowledge of the terms to sink in. But I have it now.

Edited by Bob Speziale
Link to comment
Share on other sites

for an even greater understanding of how video chips transform light into pixels, i went down the google rabbit hole with these terms:

- cmos, photon, electron, hole
(it might return this for starters http://spiff.rit.edu/classes/ast613/lectures/ccds_kids/ccds_kids.html)

and your youtube rabbit hole may start at somewhere like this https://www.youtube.com/watch?v=_ag0iSs5vhs

happy googletubing!

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...