Attendance:

Guests:

• Pierre-Anthony Lemieux, Sandflow Consulting

• Michael Smith, Intel / Wavelet Consulting

• Li Ji, Industrial Light & Magic

Discussion:

Quick Recap

The team discussed a recent presentation given by Cary that was well-received, with attendance numbers similar to a previous town hall event. Peter and Pierre-Anthony explored performance differences between various compression algorithms, particularly focusing on read speeds and identifying discrepancies between their findings. The group also discussed technical challenges related to decoder replacement in the C API and considered ways to implement custom decoders without modifying underlying code, while briefly touching on upcoming travel plans for team members.

Next Steps

• Pierre-Anthony to run performance tests on the Sparks Aces sequence using the same tools as Peter to verify compression speed discrepancies

• Pierre-Anthony to contact Kimball regarding API design for global decoder replacement functionality

• Pierre-Anthony to explore augmenting the context data object for decoder state management

• Cary to follow up with Kimball regarding items to be included in the upcoming release

• Peter to share details of the test data and code used for performance measurements with Pierre-Anthony

Summary

Cary's Successful Presentation Review:The group discussed a recent presentation that Cary gave, which was well-received with 33 attendees, including Cary who was presenting remotely. Cary shared his experience from the Open Caller I/O town hall, noting similarities in attendance numbers. The conversation briefly touched on Joseph's recent trip to Reykjavik and the high costs there, before transitioning to comments about the presentation's success.

Compression Algorithm Performance Discrepancy:Peter and Pierre-Anthony discussed performance differences between compression algorithms, particularly focusing on read speeds. Peter reported that HTJ2K was 10 times slower than ZIP for single-threaded reads, and up to 40-50 times slower when accounting for thread contention. Pierre-Anthony shared his experience that ZIP was only 50% faster than OpenGPH, which did not match Peter's findings. They agreed to test the algorithms on the same data set to investigate the discrepancy.

Compression Tool Performance Investigation:Peter and Pierre-Anthony discussed performance issues with compression tools, particularly focusing on a 10x slowdown Peter observed with a specific data set. They agreed to investigate this anomaly by having Peter run the Sparks sequence or send a problematic file for further analysis. They also touched on the impact of chunk size on parallelization and the efficiency of different compression methods like HT and ZIP. Michael provided context about the performance measurement tools used, noting that Exr Metrics may not accurately reflect file I/O times. The team decided to analyze the data rate per subband to understand the characteristics of the problematic data set.

Decoder API Runtime Replacement Discussion:Pierre-Anthony discussed replacing a decoder at runtime in the C API, but Peter noted that this would likely need to be done per decoder instance due to potential thread contention issues. Cary suggested that Kimball, who developed the core library, might prefer to keep the API low-level and let applications handle such decisions. Pierre-Anthony considered proposing this as a feature and planned to reach out to Kimball for further discussion.

GPU Decoder Integration Challenges:Pierre-Anthony discussed the need to inject a custom decoder into a library without modifying its underlying code, focusing on the challenge of using a GPU-based decoder across different applications. Peter raised concerns about the reliability of forcing non-GPU-optimized code to run on a GPU, emphasizing potential crashes due to unanticipated changes in execution paths. Cary suggested exploring ways to modify the context data to achieve Pierre-Anthony's goal, while Pierre-Anthony proposed altering the default state setup routine. The discussion concluded with Pierre-Anthony considering the possibility of creating a sub-library for high-level functionality, separate from memory allocation or decoder choices.

Kimball's Vancouver Travel Plans:The team discussed Kimball's upcoming travel to Vancouver and his planned meeting with Cary. Pierre-Anthony expressed interest in a long-term project and mentioned he would follow up with Kimball. Cary inquired about specific details related to the release date and Kimball's plans, but no clear answers were provided. The conversation ended with Cary suggesting they call it a day.