Attendance:

Guests:

• Pierre-Anthony Lemieux, Sandflow Consulting

• Michael Smith, Intel

• Tilman Schmidt

• Li Ji, ILM

• George Tattersall

Discussion:

• Efficient Spatial access considerations with JPEG 2000 - Pierre-Anthony Lemieux & Michael Smith

  • Slides: https://academysoftwarefdn.slack.com/files/U02F67JU83D/F07UBKBFV1T/openexr_ht-v5.pptx

  • Granular spatial access is hard-coded currently.

  • RGB and Kimball noted that tiling affects this though.

  • Kimball: you can go vertically down or up. But for standard 2d image file, this is true.

  • Parallel processing - send each chunk.

  • Each codec has a native chunk size.

  • HT256, HTK256 - 256 line chunks; overhead

  • HT, HK - full image is chunk, does not benefit from access granularity.

  • Allowing the API to request a particular area and let the codec figure out the best way to access it.

  • Currently codecs have to decode the entire image when they only want part of the image.

  • Peter: can’t tell API not to give me the overscanned area

  • Kimball: not true in the core library. Possible to get a subset of a scanline.

  • Peter: idea is to update the slice, tell what you've allocated when you set up the frame buffer

  • Michael: 3rd access pattern - resolution requests, not full-resolution. Enabled by full image but more awkward

  • RGB: doesn’t have that unless it exists already within the file.

  • P-A : API could abstract it so you can always get a lower res image

  • Peter: implies special cases for different codecs, currently that’s not true.

  • P-A: codecs tailored to specific tasks.

  • Peter: yes currently true, single scanline better currently. But if you introduce low res capabilities needs to be supported in all codecs.

  • P-A: client could be aware that some modes are not possible.

  • Peter: that’s unfortunate, has to go back to the user. Gets into ambiguities of the mip-map support already in OpenEXR. Supporting low res might introduce more confusion.

  • P-A: resolution scalability seems to be hard concept. Powerful when accessing over pipe constraints.

  • Kimball: that’s why there’s a thumbnail attribute. Questioning whether we want to think about resolution stuff too much. Actually used in PNG?

  • JPEG used it.

  • RGB: identifying a use case you want people to have access to that will show well because of compression but they don’t do that now anyway. Sort of circular logic. Separate argument.

  • Michael: chugging through OpenEXR sequences in Nuke is onerous, doesn’t have to be that way.

  • RGB: ran into this displaying 8k images on LED wall. Some portion of this we did, we used the mip-map tile trickery plus knowledge about where the camera is. The user knows it only needs part of the data. Positive trait but we already get away with it with tiles and mip-maps. What would be better than that by having the codec do it?

  • P-A: More efficient and scalable.

  • P-A: Low-hanging fruit is efficient line-by-line access. Could contribute to that and demonstrate it with J2K.

  • Kimball: core library does no threading though it can be enabled. Have not contemplated fusing of unpacking with the codec. codec asked to unpack the buffer that is on disk. Whether you want all channels in the end, still have unpack all of them. Info about what is unpacked later is present at time of decoding.

  • RGB: making EXR better is different than making J2K better in EXR.

  • P-A: higher compression ratio.

  • RGB: doesn’t benefit any other codec.

  • P-A: there’s another wavelet based codec.

  • Kimball: things like DWA, could skip unpacking unneeded channels.

  • Peter: metrics on different chunk sizes? EXR sort of tries to do this by having tiles. A bit like ROI interest. Could you just solve this problem (for lossless case) by just having tiles? What is the difference in read/write speed say using 64x64 tiles vs whole image in one go.

  • Michael: tiles are poor mans way of having spatial scalability. File sizes would be bigger.

  • Peter and Kimball: stats would be helpful

  • Peter: one way to address this as codec, say codec only supports tiles not scanlines.

  • --- missed some parts here due to interruption ---

  • … discussion about adding unpacking feature to EXR …

  • Kimball: as we move to more safety features and future C++ versions…

  • Peter: don’t need to read entire image out if we add this feature

  • Kimball: requires unwinding a bunch of the decoding pipeline
    
    

• https://github.com/AcademySoftwareFoundation/Imath/pull/453

  • Kimball: C++ 11 in Imath - still need to support? This PR fails with C++11.

  • Larry: most projects in our ecosystem have bumped to 17 as a minimum.

  • Kimball: CI failing with new change

  • Cary: issue if we were changing API forcing ppl to support multiple versions it would be an issue.

  • Larry: are you happy with the current solution in the PR

  • Kimball: way of addressing what Alex proposed, making it more generic, done a couple of iterations. Feel like there is a simpler way. Testing in godbolt and codegen was nearly identical or more consistently across compilers than Alex’s proposal. Option to turn it off. Separate routine so you can always get the branchless version.

  • Larry: need more test cases specific to this change. Benchmarking?

  • Kimball: only a little, not much of a performance gain. Don’t have Intel compiler setup though. Could get gcc to unroll the loops, but couldn’t get it to vectorize the loop properly.

  • Kimball: can be a real world impact, compiler should be able to better keep things in the register through the subscript operators. Even if not compiling with C++23, can call a built-in to check if the argument is a constant. In future can say `if consteval` but for now there’s a macro Kimball added.

  • Kimball: need to figure out how to add a test case.

  • Larry: not great that there is all this template magic needed to define this. Nice to put in another file to declutter it.

  • Cary: what would be do if completely writing this from scratch? Would design with an array underneath?

  • Kimball: Not sure. color3 inherits from vec3 but color4 does not inherit from vec4… don’t know history.

  • Nick: because of difference between 3 value element vs 4 value element and normalization or divide was expected. Used to have homogeneous variants of things but not in there now.

  • Cary: is complexity we’re adding inherent to the problem or the legacy? Disappointing thing, why does it need to be so complicated. Can’t be simpler without making API changes that we are not willing to make.

  • Kimball: there is a component of that.