...
- https://codecalamity.com/encoding-uhd-4k-hdr10-videos-with-ffmpeg/
- https://codecalamity.com/encoding-settings-for-hdr-4k-videos-using-10-bit-x265/
- https://support.frame.io/en/articles/4305241-creating-hdr-files-for-frame-io
VMAF
I did explore using VMAF - Video Multi-Method Assessment Fusion as a way to quantify the compression, the notes for setting this up are below, however I think we are going with a fairly high compression factor , so I think this is probably not really going to help us much.
...
...
- questions/69251960/how-can-i-encode-rgb-images-into-hdr10-videos-in-ffmpeg-command-line
- https://brandur.org/fragments/ffmpeg-h265
HDR
...
...
...
...
...
...
...
...
...
...
...
Color Preservation
Testing Methodology
Converting SMPTE color bars to the compressed movie, using ffmpeg to expand and then compare with OIIO. NOTE, for compression schemes that are not 444 we may need to mask the transitions.
...
- threads/open-source-video-testing-calibration-patterns.2944378/
- https://github.com/test-full-band/tfb-video/releases
- https://trev16.hatenablog.com/entry/2021/07/23/145725 – good site for HDR test sites.
ProRes
There are four Prores encoders, Prores, Prores_ks, Prores_aw and now with ffmpeg 5 VideoToolBox Prores, which is a hardware based OSX M1 encoder/decoder.
From https://trac.ffmpeg.org/wiki/Encode/VFX the recommendation is to use Prores_ks with -profile:v 3 and the qscale of 11
Options that can be used include:
- -profile:v values can be one of.
proxy(0)lt(1)standard(2)hq(3)4444(4)4444xq(5)
- -qscale:v between values of 9 - 13 give a good result, 0 being best.
- -vendor apl0 - tricks the codec into believing its from an Apple codec.
Using this with the usual color space flags, seems to work well with the exception of ffmpeg itself, which needs the flags:-vf scale=in_color_matrix=bt709:out_color_matrix=bt709 added to the command to ensure the right input colorspace is recognised, e.g.:
ffmpeg.exe -i INPUTFILE.mov -compression_level 10 -pred mixed -pix_fmt rgba64be -sws_flags spline+accurate_rnd+full_chroma_int -vframes 1 -vf scale=in_color_matrix=bt709:out_color_matrix=bt709 OUTPUTFILE.png
However, other encoders seem to be recognised correctly, so there is clearly some metadata missing. I did try using the prores_metadata filter to try adding some additional parameters, but it didnt seem to help.
ffmpeg.exe -i ./chip-chart-yuvconvert\basicnclc.mov -c copy -bsf:v prores_metadata=color_primaries=bt709:color_trc=bt709:colorspace=bt709 chip-chart-yuvconvert\basicnclcmetadata.mov
TODO:
- Figure out the missing metadata.
- Wedge qscale values
- Do some colorspace tests with different qscale values to see where color breaks down.
VMAF
I did explore using VMAF - Video Multi-Method Assessment Fusion as a way to quantify the compression, the notes for setting this up are below, however I think we are going with a fairly high compression factor , so I think this is probably not really going to help us much.
https://github.com/Netflix/vmaf
https://jina-liu.medium.com/a-practical-guide-for-vmaf-481b4d420d9c
https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652
https://ottverse.com/vmaf-ffmpeg-ubuntu-compilation-installation-usage-guide/ - building VMAF on ubuntu.
Color Preservation
Testing Methodology
Converting SMPTE color bars to the compressed movie, using ffmpeg to expand and then compare with OIIO. NOTE, for compression schemes that are not 444 we may need to mask the transitions.
Testing loading the compressed movie in to RV, Firefox, VLC, Avid, resolve, , to compare the resulting color transformation - not sure if there is a procedural way to run this?
...
ffmpeg -y -i ../sourceimages/chip-chart-1080-noicc.png -sws_flags spline+accurate_rnd+full_chroma_int+full_chroma_inp -vf "scale=in_range=full:in_color_matrix=bt709:out_range=tv:out_color_matrix=bt709" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -pix_fmt yuv444p10le -qscale:v 1 -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 ./chip-chart-yuvconvert/spline444out_color_matrix.mp4...
1 ./chip-chart-yuvconvert/spline444out_color_matrix.mp4Using the libswscale library. Seems similar to colorspace, but with image resizing, and levels built in. https://www.ffmpeg.org/ffmpeg-filters.html#scale-1
This is the recommended filter.
Color Metadata NCLC/NCLX
The above gets the underlying data stored correctly, but there are additional metadata flags that can be set that are interpreted by some players, these are the NCLC color tags for color primaries, transfer function and conversion matrix. This is defined as a ISO spec here (see https://www.ffmpegiso.org/ffmpeg-filters.html#scale-1
This is the recommended filter.
Color Metadata NCLC
The above gets the underlying data stored correctly, but there are additional metadata flags that can be set that are interpreted by some players, these are the NCLC color tags for color primaries, transfer function and conversion matrix. This is defined as a ISO spec here (see https://www.iso.org/standard/73412.html). The numbers below are part of the definition.standard/73412.html). The numbers below are part of the definition.
NCLC stands for Non-Consistent Luminance Coding, a brief overview of its history is here. For MP4 files, its also known as NCLX. Additionally this metadata can also be represented in the h264 metadata stream in the video usability Information (VUI) block.
You can read the metadata using mp4box.js which is a visual browser of the mp4 metadata, and look at moov/trak/mdia/minf/stbl/stsd/avc1/colr
NOTE: None of the flags below affect the encoding of the source imagery, they are meant to be used to guide how the mp4 file is decoded.
...
| Numeric Value | String Values | Description |
|---|---|---|
| 1 | bt709 | |
| 9 | bt2020 | |
| 11 | DCI P3 | |
| 12 | P3 D65 / Display P3 |
| Multimedia | ||
|---|---|---|
|
Color Transfer Characteristic aka color_trc
...