Attendance:

Guests:

• Vlad Lazar, Industrial Light & Magic

• Eric Enderton, Nvidia

• Mark Leone, Nvidia

• Michael Smith, Intel / Wavelet Consulting

Discussion:

Quick Recap

The team discussed various technical challenges related to OpenEXR library compatibility issues and API changes, particularly focusing on customization conflicts and namespacing problems in Nuke. Mark presented his work on implementing GDeflate compression support for OpenEXR, which includes page-oriented formatting and GPU processing capabilities, while the team explored potential solutions for compression system improvements and build configurations. The conversation ended with discussions about ongoing work, including bug reports and crash investigations, with plans for future development and upstreaming of features.

Next Steps

• Vlad to consider static linking as a solution for the OpenEXR namespace conflict issue in Nuke.

• Vlad to reach out to Kimball for feedback on the OpenEXR core namespace issue.

• Mark to reach out to Kimball to get feedback on his GDeflate implementation code.

• Mark to submit a PR for the GDeflate implementation to OpenEXR.

• Mark to contact the maintainers of the GDeflate repository to rebase atop the current master and discuss upstreaming to Libdeflate.

• Mark to investigate CI issues with his OpenEXR fork.

• Cary to help Mark with CI issues if the PR fails.

• Peter to look at the ARM/Raspberry Pi issue to identify the OpenJPH crash problem.

• Peter to try using address sanitizer to debug the ARM issue.

Summary

OpenEXR Library Compatibility Issue:Vlad encountered an issue with OpenEXR in Nuke, where his custom forked version conflicted with the built-in OpenEXR library, causing assertion errors when compressing with HTJ2K. Peter explained that this was likely by design, as the new core library is intended to be drop-in compatible with older versions, allowing for backported fixes. Larry noted that namespacing was implemented to prevent conflicts between incompatible versions, but the unnamespaced core library presents a challenge for embedding applications like Nuke that require customizations.

API Compatibility and Library Changes:Vlad and Peter discussed API compatibility issues related to Vlad's changes to the core library for deep compression, which broke API compatibility and required recompilation. They explored potential solutions, including using LD preload to manage different library versions, but Vlad expressed concerns about shipping such solutions to users due to potential complications with multiple experimental branches. Peter suggested discussing the matter further with Kimball to address the trade-offs between customization and compatibility.

Core Namespacing and Library Challenges:The team discussed challenges with namespacing and linking in Core, particularly regarding OpenJPH and OpenEXR libraries. Peter and Vlad explored various solutions, including static linking and using symbolic flags, but noted that these approaches could lead to multiple copies of libraries being installed. They also touched on ILM's custom namespace implementation and the potential for different versions of Core to be used simultaneously without conflicting with Foundry updates.

OpenEXR Feature Development Discussion:The team discussed the need for Kimball's feedback on a feature development plan and clarified that the issue involves implementing preview features in OpenEXR for Nuke versions 15 to 17. Peter asked whether the problem was about developing new features or having a custom version, and Vlad confirmed they want to provide artists with access to bleeding-edge features, currently using a complex workaround that hasn't been widely adopted.

GDeflate Compression for OpenEXR:Mark presented his work on adding GDeflate compression support to OpenEXR, which he implemented as a fork of the existing library. He explained that GDeflate is a page-oriented format that allows for parallelization on GPUs, requiring metadata to be written during compression and read during decompression. Mark noted that while the CPU implementation performs similarly to LibDeflate, the next step would be to demonstrate GPU speedups by implementing GDeflate in OpenImage.I/O, though Larry pointed out that OpenImage.I/O currently lacks API calls for getting UN-decoded file chunks, which would be needed for GPU processing.

OpenEXR Compression System Implementation:Mark and Larry discussed the implementation of a new compression system for OpenEXR, focusing on the concept of pages and chunks. Mark explained that the compression is done in 64-kilobyte pages, which are further divided into chunks for processing. Larry raised concerns about the performance implications for smaller data sizes and suggested adjusting tile sizes for optimal results. Mark noted that the implementation aims to have minimal impact on developers and can be enabled through a CMake configuration variable. Cary mentioned that the Bazel build configuration needs further work, which will be addressed by the maintainer.

Upstreaming GDeflate and GPU Compression:Mark has completed work on GDeflate and plans to upstream it to the Libdeflate repository, which he previously discussed with the maintainers two years ago. He will reach out to them to rebase the fork onto the current master and discuss upstreaming. The team discussed the potential for GPU-accelerated compression and decompression, with Mark mentioning a colleague's work on BC formats in another library. Cary expressed a preference for avoiding build-time configurable codecs, suggesting that any new features should be supported in all builds by the time of release.

CI Issues and Bug Backlog:The team discussed ongoing work and issues. Mark agreed to submit a pull request, despite CI failures, so Cary could investigate. Peter offered to examine an ARM-related crash issue, potentially using an old Raspberry Pi. Cary mentioned a backlog of bug reports for Kimball to review, pending his availability over the weekend. The team also discussed the possibility of OpenJPH being the cause of a reported crash.