Attendance:

Community:

Discussion:

• Virtual Town Hall

  • Previewing our fall release with Kimball's changes 

  • C API

• Fall release

  • Kimball: C API not pushed yet, reading side complete, writing side started halfway through, goal is to finish it this week. After his vacation, smaller changes to put in -

    • Additions to API, stateless read to avoid threading issues if needed.

    • Change to start getting to a stable ABI, redoing namespaces and change versioning scheme, transparent change. Ensure it is stable enough in Windows and Mac

    • Nick: re: ABI change -there is a mechanism for checking your symbols haven't changed from build to build.

• Blosc compression

  • Kimball: concerned about adding a different dependency

  • Cary: Blosc is main new dependency, Phillippe was having issues with it. Could we do same as we did with libdeflate? Either download and build local option or you find an external one

  • Nick: Blosc is fairly small. Doing same pattern probably makes sense

  • Peter: Fuzz test has its own build system, hard to test

  • Kimball: you can follow their directions locally to validate it but it's a pain

  • Peter: could provide our own build file. They did a multiscanline format which doesn't work with deep since multiscanline not implement. Did get multiscanline working in a separate PR.

  • Kimball: basically use Blosc library, remove deflate library?

  • Larry; libdeflate was better performance, if move want to be sure we are not introducing a performance regression

  • Nick: don't think Blosc provides deflate

  • Kimball: zstd may provide it. Can do same for Blosc as we do for deflate - inline inclusion. Linux distros don't like that though. Blosc now includes zstd even though both are compressors. Blosc uses original zlib not libdeflate... so maybe we shouldn't support blosc. We should investigate further. zstd is on its own. Blosc is a superset of zstd and also its own separate compression format.

  • Nick: Blosc rolls up lz4, fastlz, BloscLZ, zlib and zstd. Concerned about introducing all of these options into OpenEXR when not feeling the need.

  • Kimball: if it makes its smaller, it's a gain. Faster not as important.

  • Nick: trading our maintenance time for smaller files.

  • Pete: blosc did original testing on deep files. Seems strange to add a compression type not customized to work with images.

  • Kimball: rather do a custom spin of BC6h stuff - silicon to decompress on tile on the GPU

  • Pete: making deep smaller would be good

  • Li: looking at floating point compression specifically. Deep is not that. 

  • Nick: blosc is pain on Windows - Kimball: has its own pthreads implementation.

  • Nick: biased against OpenEXR being the "magical toolbox of compressors"

  • Pete: could start depricating our formats saying we don't recommend you use this anymore.

  • Larry: blosc widely used, including by OpenVDB... not in league as something truly untested.

  • Nick: good EXR is tightly focused, having better compression still tightly focused would be preferred.

  • Larry: would like to rely on EXR for textures more.

  • Nick: withdraw objection but something of interest in context of Storm and realtime renders - not using OpenVDB in USD build. 

  • Pete: going to test how well blosc compresses.

  • Cary: not thrilled with how this has unfolded, submitted last December, now up against deadline to include for next year. Ideal way would be studio made a custom implementation and used extensively then go from to sharing. But no compelling story as to whether this is better and how much better. 

  • Peter: whether upcoming changes affect the API - whether that's wise since there is a fair amount of change just switching to the core. Changing the file format may be too big of a change.

  • Kimball: headers are distributed but probably shouldn't be. We should review before signing off on the release - decide which things we want to depricate and start removing.

• Version bump?

  • Cary: would these changes be a bump to EXR 4 ?

  • Kimball: if compression include should be 4. if only C  API core internal rewrite then could be 3.3. In switching to stable ABI, would it be good to reset that to 4.  

  • Nick: do we need to get in under VFX ref platform timeline and declare a 4.

  • Kimball: in theory, these are all non-breaking changes so it's up to us 3.3 or 4.

  • Pete: there is a file format version as well. EXR vs EXR 2 files.

  • Larry: when added multi-part , natural to bump version for such a big change.

  • Pete: based on experience, headaches of new versions will have to do with sharing files between facilities.

  • Cary: need to answer the zstd question. Peter to do tests.

• OIIO

  • Larry: have not hooked up Kimball's branch to OIIO yet 

  • Cary: would be good to test this soon.

• Windows

  • Nick: Core API on main - on windows, getting a corrupted header when writing a checkerboard test file. Not sure if issue is on his end or in library.

• Python bindings

  • Cary: working on finishing, almost has deep working, reading tiles at all levels - current implementation reads top level only. Not sure how to review, it is a large change.

  • Larry: deep data - custom thing, if first class OpenEXR binding defined that held thing to be passed around I would switch to that to conform, made something up but it isn't ideal. 

  • Cary: read deep, returns NumPy array whether every entry is another NumPy array. Not working as expected yet.

• Li update

  • Research project of EXR compression - has student committed to the project. Don't have a benchmark dataset. May invite professor and students to an upcoming TSC meeting. 

• Dev days projects

  • Cary: think of any entry level tasks for potential projects.

• Cary gone next 2 meetings, Kimball gone next meeting.