2024-06-17

Owned by doug walker
Jun 23, 2024
10 min read

June 17, 2024

Host: Doug Walker

Secretary: Doug Walker

Attendees: 

Nick Porcino
Doug Smythe
Larry Gritz
Thomas Mansencal

Doug Walker

(Sorry, didn't capture the other participants)

Apologies:

  • Carol Payne

OCIO NanoColor Working Group Meeting Notes

  • Core Color Space Names

    • Doug W.: The group made some progress on the set of core color space names at the meeting right before this one, the ASWF Color Interop Forum. That recording is available from the ASWF Calendar web page.

    • Doug S.: When Jonathan is here, we should discuss how we switch over to the new names, do we want aliases for the old ones, how are document updates handled, etc.

    • Nick: I already have a list of names that were stuck with that I'm handling in my implementation for backwards compatibility. In USD we have a script to fix broken schemas. People invoke the migration manually since they may have a mix of new and old tools that want to read a scene.

    • Doug S.: MaterialX has a mechanism when it loads an old document to upgrade to the latest version of things.

  • Best Practices On Color Space Usage

    • Doug W.: Following up on last week's meeting, we were talking about how to handle transforms that are not part of the core set and which may not be represented as a custom color space. The example we were using last week was if a DisplayViewTransform needed to be inverted on the way to the rendering space. The point was made that in order to handle a situation like that, the texture would need to be processed, perhaps using full OCIO and then a copy saved in one of the nanoColor supported color spaces. This need to invert a DisplayViewTransform is one that comes up quite often and Maya, for example, already supports that conversion via full OCIO. In some cases people may not want to have to duplicate their textures in order to save as a USD or MaterialX file. There is a trade-off between flexibility/convenience and making a file that will have universal compatibility. Certainly if you are going to send a file to another facility or archive it, universality is important. But within the confines of a given studio pipeline, flexibility may be more important. I discussed this with my colleagues and we think we should allow users to make that trade-off rather than disallowing any flexibility, with the proviso that if they are going to use arbitrary color spaces from their OCIO config, that will limit the portability of the document.

    • Doug S.: When you say that there are cases where full OCIO is needed, are you talking about during a render?

    • Doug W.: In principle yes, however, in many cases the renderers are creating a cache. Doug S.: That's the big asterisk. One of the main concerns of this project is not necessarily the flexibility vs. portability argument, but rather the performance argument. We don't want to have slow operations during every texture pixel access. The idea is you convert to something that is a very quick transform away from the rendering space. In my experience, it is indeed common to run into images in a display-referred space that need the more complex transform, but there is always a preprocessing step that does the transform prior to the render. The nanoColor restrictions are aimed at the image used in the render, but I think it's reasonable to maintain file pointers to both the original and the processed version in a USD file, for example.

    • Thomas: What about the case where you have a blackbody simulation or a physical sky but you're not using a spectral renderer, so it's producing an RGB value that's based on the model you're implementing. Who would take ownership of getting that to the rendering space? Because the physical sky is being calculated on a per pixel shaded basis.

    • Doug S.: As long as that shader is identifying it as being in a certain color space, then that would be a case where nanoColor would be invoked to convert to the rendering space.

    • Doug W.: I fully agree that we don't want the renderer to need to apply some complicated color transform every time it needs to access a texture. But it seems the renderers all have their own mechanism for solving that. For example, some renderers run OIIO's maketx to use full OCIO to apply the color transform to convert to the rendering space and generate mipmaps. And that's basically a cache that could be deleted when the render is done. But the main reason I brought up the topic was to verify that everyone is ok with the notion that people should have the flexibility to create a MaterialX or USD file with a texture that could be assigned a color space from a full OCIO config, with understanding that it would be less portable.

    • Doug S.: I see that as out of scope for nanoColor. If you're referencing a texture file in your USD scene and it says it's a color space that's not in the nanoColor color spaces, then that's an error. And something outside of nanoColor needs to convert the texture to one of the standard spaces. But you can maintain an attribute that points to the original image file and identifies the color space. And maybe there is a preprocessor application that could run, but again it's outside the scope of nanoColor's responsibility.

    • Nick: Agree. There is a possibility that there could be some smart caching that would allow arbitrary files to be used, but that would be an extremely heavy lift because our code does not do that today. It assumes it can do the conversions on the fly. If each of the 1000 samples per pixel needed to parse a config file, that would not be acceptable for performance.

    • Doug W.: But what about the case where the renderer is already creating the cache? Doug S.: It's not the renderer that creates the cache, it's a preprocess.

    • Nick: If we blast something off to a render farm and spin up a thousand GPUs, they don't have access to the shared cache state, that's the problem.

    • Doug W.: But even for a simple space like sRGB, wouldn't you want to have that converted to the rendering space and have a .tx file with all the mipmaps be calculated ahead of time?

    • Nick: That's what we do, the strict difference is that it happens before it hits the shader input. We don't do the caching in the shader graph. I think we're saying the same thing. Thomas: Yeah, I think so.

    • Doug W.: I totally agree that we don't want to be invoking full OCIO within the shader graph itself. However, I do think it should be allowed to create a USD file that references a texture that is in a full OCIO color space, not necessarily encouraged or recommended, but at least allowed. So that a studio could make its own trade-off decision about portability vs. flexibility within its own pipeline. Thomas: Yeah. Nick: I don't know.

    • Larry: My impression is that there are two different types of users here. Probably the studios everyone on this call is working for have an entirely offline process that you can think of as part of the publishing process. You know, a painter paints something and they publish it as an asset to be consumed downstream. And by the time it gets to the renderer it's already transformed and in linear space. And then there's a second class of user that is outside my experience but I know they exist and are important as part of the user base of DCCs and they don't understand any of that. And they write their shaders assuming they can pick up any images they have and they expect the DCC to be able to deal with it. And the DCCs and renderers all have different strategies for how to handle that. But I think the essential point is that the shader networks at some places are always designed to assume linear working space inputs and at other places they are not. I understand the desire to accommodate both use-cases. I don't know exactly how to do that, but I think we should acknowledge that they both exist and that we can't simply wave away the one we don't like. Doug W.: Agreed.

    • Doug S.: The people in the second group are also the ones that tend not to understand the distinction between scene- and display-referred. But they've been working without proper color management for years. A basic sRGB transform is likely to meet their needs, so they wouldn't need a full OCIO config.

    • Nick: We're way out of scope for nanoColor. This discussion is more of a feature request to MaterialX and USD. Right now, USD has a very limited set of options for textures and they have to be conformed before they reach the USD file description. I think what you're proposing as a requirement is that if a USD file is authored with a color space that's not in the nanoColor set that USD itself could have a conform step that runs the OCIO conversion to conform it to the nanoColor domain, perhaps invisibly to the user. I think this would be a perfectly fine proposal to take to the USD Propoals process. There may be some pushback, but I think a convincing case could be made that a formal caching mechanism to bridge the rendering domain with an arbitrary domain is reasonable.

    • Doug W.: But if we look at what's in the current release of MaterialX (which is still 1.38), there is still an attribute to specify an OCIO config and the ability to assign any color space from that config to textures. There is no limitation in the spec currently, and Maya already supports that with full OCIO.

    • Doug W.: The reason I bring it up is, going back to our charter, I don't want nanoColor to prevent people, that want to, from using full OCIO. I see nanoColor as the baseline or minimal required functionality, but it should be allowed to go beyond that.

    • Doug S.: I think that's putting the cart before the horse. What we're saying is that if you want the benefit of fast color conversions during render time then you should use nanoColor, which means you need to accept those limitations. And that means the particular list of core color spaces or a custom color space that uses one of the allowed operations. If we try to make nanoColor capable of dealing with the full OCIO list, then we lose the benefits. Doug W.: Yeah, just to be clear, I'm not proposing expanding the scope of nanoColor. If anything, over the last few meetings we've been restricting it. Doug S.: It's not that we're trying to keep other tools out, it's that we're trying to obtain some specific benefits.

    • Nick: One thing we could do to address Doug W.'s point, is to create some best practices. For example, if you have a file with a complex color space associated with it, the best practice is to pre-transform and cache the result in one of the nanoColor spaces for the purposes of consumption in the inner loop of the renderer. It's actually probably very important for us to write down those sorts of guidelines. Doug W.: That would be great, and that's exactly how I would frame it, as a best practice.

    • Nick: I think it would be good for this group to take this to USD Proposals for discussion with the USD team. I think it's equally plausible for them to say "no" and here's why that can't be done, or we didn't realize that was a useful option.

    • Doug W.: Absolutely. And one of the points in our charter was about documentation and best practices, so it fits right in with that. Nick: We have a concrete bullet point here, why don't we put it into the document now, while we're thinking about it.

    • Doug S.: There's a set of deliverables that would include best practices, and another document that has the list of core color space names, what they mean, and the transforms between them. Doug W.: Yes, the second one is a deliverable from the ASWF Color Interop Forum, and that group has been making good progress and I've started to write up that document and produce a config. I'll be sharing an update hopefully next week on the Color Interop Forum Slack channel.

  • Updating the Charter document

    • Nick: There are some unresolved conversations on the charter, it would be nice to resolve those and perhaps be more specific about the deliverables.

    • Doug S.: Is that recommendations for how to incorporate nanoColor into an application, how nanoColor and an application would work together?

    • Nick: We want to capture best practices for interoperation with OCIO.

    • The group added a sixth bullet point to the charter document.

    • Doug W.: The first comment to resolve is from Kimball. Nick: Kimball was asking how to support these names in the context of OpenEXR files. Doug S.: I think he's asking what to do if the chromaticities in the OpenEXR files don't agree with a color space specified externally. This seems like part of the best practices.

    • Larry: The chromaticity attributes are rarely stored, and we usually have low confidence in their accuracy. If there are some standard color space names that emerge from this, maybe we should recommend using those instead for cases where they are appropriate. Thomas: Agree, there should be a new attribute for that.

    • Doug W.: This sort of question is exactly why we've set up the ASWF Color Interop Forum. So rather than taking this on as part of nanoColor, we could recommend OpenEXR bring this topic to the Interop Forum. Nick: Yeah, I would like to encourage the OpenEXR community to do that, speaking as a TSC member, and with Larry present as well!

    • Doug W.: And then Kimball had another comment on point 5 about spectral. Doug S.: As Anders pointed out, spectral colors are above color spaces. Nick: You need an observer. Thomas: Yeah, it's not giving you a color, it's giving you the amount of radiant energy. It's beyond color, it's an energy quantity of some sort. It would be a good topic for the Color Interop Forum. Group: This is an interesting area for future discussion, but is currently out of scope.