Attendance:

Guests:

• Li Ji, ILM

• Michael Smith, HT-J2K compressor

Discussion:

• Li Ji presented dry-run of “Use cases for benchmarking design” slides

  • Slides: https://github.com/lji-ilm/openexr-notes/blob/main/resources/Use%20cases%20for%20benchmarking%20design.pdf

  • What will be a good test, what are criteria? Needs to reflect real-world usage, be reproducible, scalable, understandable.

  • Joseph: looked at submission from Michael Smith for HT-J2K compressor, trying to determine whether he was doing relevant work. https://jpeg.org/jpeg2000/htj2k.html .

  • Questions about real-world production usage. What common patterns of calls are in real world? Approximate combinations / importance of patterns.

  • When someone tries to commit, benchmarks would be invoked?

  • Larry: might want to think of these items more on a task basis. Trying to get to one metric? Let people look at the cases and decide for themselves.

  • Larry: interesting cases for benchmarking

    • Hardware

    • Patching as add a patch, version to version comparisons are we getting better

    • Application that wraps EXR - how far off the best bare bones EXR case is it?

  • Aras P tests setup as an example: https://aras-p.info/blog/2021/08/04/EXR-Lossless-Compression/

  • Make tests useful for studios

  • Kimball - 4 major cases, covers a lot of ground:

    • Open in editing tool - reading whole window - test exists

    • Playback - OpenRV type access - stream in some portion of image

    • Compositing workflows - Nuke/Shake - very heavily scanlined, diff threads reading scanlines, concurrent scanline access

    • Rendering - test render from OSL could be used, Larry: access pattern is very particular

  • Larry: writing simpler to test, no access patterns to untangle.

  • Larry: content types

    • photographic - samples in DPEL now?

    • renderer output

    • texture input, a few diff patterns

  • HDR Photographic survey http://markfairchild.org/HDR.html

  • Joseph - no command tool to convert from cameras to OpenEXR

  • Michael Smith: standard photographic tests https://dpel.aswf.io/asc-stem2/

  • More reference content: https://opencontent.netflix.com/