Attendance:

Guests:

• Pierre-Anthony Lemieux, Sandflow Consulting

• Michael Smith, Intel / Wavelet Consulting

• Li Ji, Industrial Light & Magic

• Michael Hickman

Discussion:

Quick Recap

The OpenEXR team discussed implementing a 32-line chunk size option alongside the existing 256-line setting for HTJ2K compression to improve multi-threaded performance, with the decision to add both as separate compression types (HTJ2K_256 and HTJ2K_32) to maintain backward compatibility. They addressed OSS fuzz issues in the HTJ2K code and discussed future API changes, including plans to deprecate the C++ API in favor of the core API and potential enhancements to allow compressors to be stateful across chunks. The team also explored optimization possibilities for multi-channel decoration transforms and reviewed outstanding items before the next release.

Next Steps

• Pierre-Anthony to update documentation to list both 256 and 32 scanline options for HTJ2K compression

• Pierre-Anthony to notify Carrie about the decision to add both 256 and 32 scanline options

• Pierre-Anthony to submit pull request for adding 32 scanline support

• Kimball to deprecate direct access to compression classes in C++ library

• Kimball/Pierre-Anthony to start discussion about enhancing API for compressor/pipeline state management

• Michael to keep team updated on progress of multi-channel decoration transforms research

• Team to consider adding fuzz testing to the library itself

Summary

HTJ2K Compressor Development Updates:Pierre-Anthony mentions that Carrie wants a resolution on the number of scan lines per chunk for the HTJ2K compressor, which is high on his to-do list. Kimball reports submitting two pull requests to fix OSS fuzz issues in HTJ2K, including adding exception handling and addressing corrupt data problems. Kimball also notifies the team that he'll be pushing changes this weekend to prepare for internal modifications to the C++ library, with Peter confirming these are needed for the next release. Pierre-Anthony begins discussing the background on the 32 scan lines issue but doesn't complete his explanation.

HTJ2K Chunk Size Optimization:Pierre-Anthony reports that implementers have provided feedback that multi-threaded performance on scan line files with HTJ2K compression is not meeting expectations on machines with many cores. The issue stems from the current 256 lines per chunk setting, which limits parallelization compared to PIZ and ZIP compression methods that use 32 and 16 lines respectively. Pierre-Anthony proposes reducing HTJ2K chunk size to 32 lines, which would only decrease compression efficiency by a few percent but significantly improve performance on large machines. The team discusses several options including changing the default, adding a new compression type with smaller chunk size, or making chunk size a parameter, with most agreeing that adding a new compression type option would be the least disruptive solution.

Tile Size Options Discussion:The team discusses whether to add a 32-pixel tile size option alongside the existing 256-pixel tile size for image compression. Peter suggests keeping both options would be safest since they're close to release, noting it wouldn't have been a bad idea to have both sizes from the beginning. Kimball initially has no strong opinion but later leans toward having both options, while Michael points out that having two choices might be confusing for users. Larry mentions he has integrated this into open image I/O but it's not in the release yet, so he's not concerned about compatibility issues.

OpenEXR Chunk Size Compatibility Discussion:The team discusses potential issues with changing the chunk size in OpenEXR files, with Peter expressing concern about backward compatibility and error messaging. Kimball suggests adding both 256 and 32 versions with clearer naming conventions (HTJ2K_256 and HTJ2K_32) to avoid confusion. Pierre-Anthony updates the pull request during the meeting to implement these changes, modifying the enum values for consistency. The group also briefly discusses the history of the "IMF" abbreviation in the codebase, with Larry suggesting it likely stands for "ILM Image Format."

HGJ2K API Restructuring Plans:Pierre-Anthony is working on code changes to add support for 32 scan line chunks for HGJ2K 256. Kimball explains that the plan is to deprecate the C++ API in the future, with the core API becoming the stable, version-backwards-compatible API, while the C++ API will function as a wrapper around it. Kimball mentions this approach would help avoid C++ version issues in the VFX reference platform and allow the use of modern C++ features like multi-dimensional ranges from the standard library.

Memory Management API Enhancement Discussion:Pierre-Anthony and Kimball discuss the issue of memory management in the compression pipeline, where currently the compressor allocates buffers but the pipeline deallocates them. Pierre-Anthony suggests enhancing the API to allow compressors to be stateful across chunks, which would be beneficial for hardware-backed compression that could avoid repeated initialization overhead. Kimball expresses concerns about thread safety but acknowledges the potential benefits, especially for GPU decoding. They agree to explore a potential API enhancement that would formalize the contract between the compressor and pipeline while maintaining thread safety, with Pierre-Anthony planning to start with an RFC or pull request.

JPEG 2000 Channel Ordering Discussion:Pierre-Anthony explains that the header in the JPEG 2000 code stream is used to reorder channels so that RGB appears in the correct order to benefit from the built-in decoration filter. Michael mentions they are researching multi-channel decoration transforms that could apply to any RGB or GBR combination without needing channel mapping. Peter suggests conducting a study on possible optimizations when packing everything into one part, noting that correlation between channels in multi-channel images might depend on the rendering method.

OpenJPH Release Review and Fixes:The team discusses the status of a release, with Michael asking if they need to review OpenJPH one more time. Kimball mentions fixing two bugs in the EXR code for OSS fuzz, and Peter notes that the OSS fuzz team is helpful with setting up fuzzing. Pierre-Anthony asks everyone to review the pull request and confirms he has updated the documentation to list the two new compression types separately. Kimball mentions having issues with Windows static compilation, and he and Pierre-Anthony agree to start a chat about other matters.