Does the complexity of the scenbe background increase camera workload for RAW the way it does encoded formats?

[Nicholas Mirro]

Hi.  I started shooting RAW footage recently and hit a brick wall.  It seems hardware rated for 4K120 often cannot reach that benchmark reliably.

For our Canon XF605 (can't output RAW), AVC and MP4 image quality is in some way tied to the scene complexity.  Panning through a complex scene like an audience or 50 feet from trees drastically increases the 605's compression workload.

Regarding 4K120, I first thought the Ninja+ was overrated and could not reliably handle RAW at 4K120.  But now I'm beginning to think it is the Z Cam E2-M4's problem.  Lots of juddering when panning through complex scenes at this high framerate.

But... does in camera RAW processing bottleneck when the scene gets complex the way encoded footage does?  or is RAW image processing workload (output to HDMI) independent of content?

 

[Nicholas Mirro]

Both the SSD and HDMI cable used are high end.  They are definitely not the bottleneck.

[Tyler Purcell]

Wait, you bought a camera that can't capture it's entire list of frame rates internally? 

[Nicholas Mirro]

Externally by HDMI.  My prior 5 camcorders and 6 camera bodies all met their HDMI output specs.

[Tyler Purcell]

1 hour ago, Nicholas Mirro said:

Externally by HDMI.  My prior 5 camcorders and 6 camera bodies all met their HDMI output specs.

According to a quick search of the specs, it doesn't say the camera can create higher than 59.94 FPS on the HDMI output.

I assume it can only do high speed in camera when set to HD mode, per the manual. 

[Nicholas Mirro]

Thanks again.  I see why you say that.  I spent last hour stressing over your finding.  The camera allows the 4K120 RAW setting and the Ninja V+ records, but with juddering. Then after almost canceling my Atomos support request, I finally found my source.  Please look at this Atomos video...

 

Edited April 24 by Nicholas Mirro

[Aapo Lettinen]

I have not used that combo but often HDMI RAW has most issues with the hdmi cable not being reliable on the very high data rates (frequencies and bandwidth) on the highest settings. it is a bit hit and miss with the cables, the expensive ones may be better but not always. I tend to blame the cable if some settigs can't be reached and the ssd does not drop frames. sometimes the ssd drives can be the issue too but you can spot it easily because the not-fast-enough ones will fill the buffer and start to drop frames. with the normal NinjaV the Angelbird drives have been best for me with raw as they are designed to support fast continuous bitstreams instead of the normal computer drives which are good for data bursts but suffer on continuous data stream like video recording, you can see this on computer use too and not just on video recorders. but  don't know which ones you use on the V+ and they are probably fine if you can't see the Ninja indicating frame drops.

 

[Nicholas Mirro]

Here's what the juddering looks like...  It's 4K so lots of detail.  I was thinking that much panning detail might overwhelm either the camera or Ninja's processor.  But I found some footage showing mostly sky and it had the same problem.

 

 

Edited April 24 by Nicholas Mirro

[Aapo Lettinen]

4 minutes ago, Nicholas Mirro said:

Here's what the juddering looks like...  It's 4K so lots of detail.  I was thinking that much panning detail might overwhelm either the camera or Ninja's processor.  But I found some footage showing mostly sky and it had the same problem.

 

 

seems it drops complete frames but l would suspect it is the camera and not the recorder. kind of like shutter/framerate mismatch but probably something else. how it looks like when you inspect the jitter frame by frame, is there a repeating pattern and how it behaves when counted the frames?

[Nicholas Mirro]

Quote

how it looks like when you inspect the jitter frame by frame, is there a repeating pattern and how it behaves when counted the frames?

There are two types of jutters.  For the red markers, the frames seem to follow this sequence...  1, 2, 3, 2, 3, 4, 5

For the purple markers, there is a partial frame at the top. The top ~5% of the image is from another frame, or maybe the bottom 5% stuck on the top.  

Easier to see this screenshot if opened in a new tab.  The shot is a continuous vertical tilt (pretty fast) over this fabric.  It's shot at 4K120 and played back at 30.

 

[Nicholas Mirro]

One more thing.  During the rookery shoot, there were a few skippy icons.  Since I've been secure erasing the drive it does not do that anymore.  None for the above test.  Do you think that clears the Ninja V+?

[Nicholas Mirro]

Be nice to edit posts 🙂

I did just try a new ZILR cable and no difference.

[Aapo Lettinen]

4 hours ago, Nicholas Mirro said:

One more thing.  During the rookery shoot, there were a few skippy icons.  Since I've been secure erasing the drive it does not do that anymore.  None for the above test.  Do you think that clears the Ninja V+?

erasing the drive clears the partition table so that the data on the drive is not fragmented anymore and it can potentially record faster data rates. So a formatted drive is usually a little bit faster than a drive which has something on it (especially compared to a drive which has clips recorded, then some of them removed without full reformatting, then recorderd more on the same drive. this causes lots of fragmentation on the new data), depending on if the storage space is "continuous" or if there is only fragmented free spots available to store more data for the existing files taking the rest of the space, then the "drive needing to continuously jump from one spot to another during recording" which slows it down. This is very common and if formatting seems to partially help with the footage problems then it may indicate that your ssd is not fast enough and you need a much faster one. I personally use Angelbird drives for 5.8k and 60fps 4.1k proresraw recordings and for under 30fps 4.1k I use WD blue line 3dnand drives, don't remember the exact model but they are often just fast enough to store 4.1k raw at normal framerates and sometimes 60fps 4.1k might work if the drive is newly formatted and one dares to risk one or two drops every now and then.

About the clip test, are you sure your interpret footage settings are correct and the editing software is not doing any pulldown on it? I can't imagine the camera adding pulldown on 120fps footage, there is something else going on. When importing slow mo material which is meant to be played back at the "native slow motion rate", you should set the project fps to your final output delivery fps (like 23.976 or 24.00) and then in the interpret footage settings of the clip select that same framerate for the clip's playback speed and ensure there is no additional settings anywhere else which could mess up the footage.

Edited April 25 by Aapo Lettinen

[Nicholas Mirro]

Aapo thanks so much for taking time here.  I am grateful!  Definitely no pulldown since the project framerate is 29.97 and footage is 29.97 VFR 120.  As I understand it, the frames are 1:1 as 120 is a multiple of 30.

I think I've figured it out partly.

Something I missed.  The Atomos YouTube demo does not advise 4K30 VFR 120.  It suggests either 4K23.98 VFR 120 or 4K25 VFR 100.

I tested the VFR 100 setting and it recorded the same fabric tilt footage (playback 25fps 2 minutes and 11 seconds) without a single error.  Finally some light at the end of the tunnel!

I just tested the suggested 23.98 VFR 120 setting and its a no-go.  Lots of artifacts so unusable.

So my project framerate needs to be 25 vs 29.97.  Destination is Vimeo so I guess no issue.  Now only need to add all the 29.97 footage to a new 25 fps project 😄

[Aapo Lettinen]

3 hours ago, Nicholas Mirro said:

Aapo thanks so much for taking time here.  I am grateful!  Definitely no pulldown since the project framerate is 29.97 and footage is 29.97 VFR 120.  As I understand it, the frames are 1:1 as 120 is a multiple of 30.

I think I've figured it out partly.

Something I missed.  The Atomos YouTube demo does not advise 4K30 VFR 120.  It suggests either 4K23.98 VFR 120 or 4K25 VFR 100.

I tested the VFR 100 setting and it recorded the same fabric tilt footage (playback 25fps 2 minutes and 11 seconds) without a single error.  Finally some light at the end of the tunnel!

I just tested the suggested 23.98 VFR 120 setting and its a no-go.  Lots of artifacts so unusable.

So my project framerate needs to be 25 vs 29.97.  Destination is Vimeo so I guess no issue.  Now only need to add all the 29.97 footage to a new 25 fps project 😄

ok so it was Atomos framerate vs camera framerate mismatch, great that you found the issue!

 

When you are shooting slow motion and there is no sync sound, your destination framerate vs the recorded file framerate does not matter at all.  You will just change the rate in interpret footage settings (premiere, ae, etc) or clip settings (Resolve) to match the playback framerate to the project framerate you are finishing with.  The end result will be the same you wanted if you recorded 120fps stored to 25fps file vs 120fps stored to 48fps file etc. when the playback speed of the clip is set to your target (for example 29.98) when editing the project so that all the frames are played back intact and no frames are skipped or interpolated or blended (which happens if the playback framerate does not match the project framerate).

Playback speed of the recorded clip is just metadata, it is not set to stone and on most video files it can be altered on the actual raw material too by altering the metadata of the file. the actual video data stays always the same, you just tell it to play back at different fps.

same thing in editing programs, you just tell the program that "this clip which the camera marked to be 59.94fps is wanted to playback at 24fps" and that's it. The clip becomes longer in seconds&minutes (because it plays back less frames per second) but there is no artifacts because no one is altering the actual video frames, just playback speed

[Nicholas Mirro]

Yes!  The problem being the Z Cam cannot produce at least one of its claimed framerates when exporting RAW out of HDMI.  So mismatch of capabilities.

So far, only one completed project has a slow-motion clip, with diving ducks running on the surface of the water before flying.  Since we couldn't pick up their splashing sound, we used simulated sound and stretched it over the timeline.

The distorted (stretched) sound worked really well.

How does one sync slo-mo sound being cameras and TC devices peak at 30 fps?  I realize Tentacle desktop apps can recalculate timing for at least 50 and 60 fps footage.  Further dev of the Windows version was discontinued in 2021 for some reason.

We have four Tentacle Sync Es, one for a recorder and three for the cameras.  This is our first attempt to record slo-mo audio and I don't know where to start.

I understand the idea that playback speed is metadata.  e.g. variable framerate

Edited April 27 by Nicholas Mirro

[Nicholas Mirro]

Wow, I have a Zoom F6 and just now see it can capture at 192 Hz, so it can be slowed to 48 Hz.  That would give 1/4x and match the 100 fps slo-mo.

So I would switch the recorder to 192 when filming 100 fps.  Then sync to a clapper.  Does that make sense?