Upgrading Houdini From USD 21.08 to 21.11

Owned by Mark Tucker
Last updated: Dec 16, 2021Version comment
4 min read

The upgrade of Houdini's development baseline from USD 21.08 to 21.11 was made unusually substantial by the upgrade from the Ar1.0 to Ar2.0 interfaces.

Ar2.0

  • Houdini's asset resolver was already handling only paths that started with "op:", "opdef:", "oplib:" and "opdatablock:". This means the conversion of our asset resolver to be one that explicitly only supported these schemas was mostly a matter of deleting code.

  • Our asset resolver also used to maintain a pointer to a "fallback resolver", which was the (potentially user-authored) asset resolver that handled all other cases. The Ar2.0 library now handles all of this for us, making our code simpler, and making it easier for users to install multiple asset resolvers that work together, each one handling a different kind of asset path.

  • Because of the new OpenAsset API used consistenly across the USD baseline, we were able to eliminate a lot of ugly code that used to write temp files to disk to hold copies of files that existed elsewhere on disk, or to contain dummy data just so USD could find the file on disk.

  • Our render delegates needed to be updated to use the ArAsset interface for loading texture maps (to ensure they would work with any user-created asset resolver). This was also an opportunity to clean up a bunch of ugly code that we had written in several places for detected and handling texture maps inside USDZ archives. The end result is more robust and easier to maintain (though it was a non-trivial effort to figure out how to replace our disparate hacks with a single unified hook for opening image files without adding a dependency on the USD library to our lower level image loading library).

  • Houdini's layer editing approach often involves loading USD data off disk into anonymous layers in memory. This has always required converting asset paths from layer-relative form to an absolute form (since relative paths from in-memory layers are treated as relative to the application's current working directory). Several Ar APIs we previously used for this operation (IsRelativePath, IsSearchPath) have been removed in Ar2.0. We used these APIs to make relative paths into absolute paths but leave search paths as search paths. We had to abandon this selective approach, and instead convert all paths to fully resolved paths. We don't think this is a big loss though as it is a bit of a niche use case (affecting only studios that use search paths), and Houdini has a mechanism for converting full paths back into search paths when writing USD layers to disk.

  • All in all, this was the vast majority of the work of this upgrade, but the result is an across-the-board improvement in terms of simplicity, addressing old nagging issues, and future extensibility.

UsdLux

  • UsdLux went through another fairly major overhaul, but in general the USD team did a good job of hiding a lot of the details.

  • UsdLuxLight is no longer a primitive type. It has been replaced by the UsdLuxLightAPI schema, which can be applied to any primitive. But the concrete primitive types that inherited from UsdLuxLight (like UsdLuxRectLight) still exist. They just inherit from a different base prim type (UsdLuxBoundableLightBase for lights with some associated geometric representation or UsdLuxNonBoundableLightBase for lights that cannot be accurately represented geometrically). And they all apply the UsdLuxLightAPI schema.

  • Adjusting to this mostly meant replacing IsA<UsdLuxLight>() calls with HasAPI<UsdLuxLightAPI>() calls. In other cases a test to see if a prim inherited from UsdLuxLight became a test if the prim inherited from either UsdLuxBoundableLightBase or UsdLuxNonboundableLightBase.

Hydra

  • The first batch of changes leading toward the hydra revamp seem to have shown up in this build. In particular, the direct use of an HdSceneDelegate object seems to have been replaced with an indirection through an HdSceneIndexAdapterSceneDelegate. Miraculously, this change was almost completely seamless (at least after early testing).

  • The one noticeable effect of the hydra changes was that SamplePrimvar now behaves differently on some primitive types. Previously SamplePrimvar could be called with a Light or Camera primitive to get any attribute on the prim. Eventually the call would fall back to a simple Get() call on the UsdAttribute. Now, at least for Camera prims, there is an early exit because an HdPrimvarsSchema object can't be created from a camera prim. Using SamplePrimvar for this purpose always felt wrong, but it worked and allowed for some additional code reuse. Now we simply have to use a different code path for querying camera attributes. Not a big deal in the end, but fairly difficult to track down.

  • Instancers now return their orientation data in raw quaternion form which requires less data manipulation (in UsdImaging and again in the render delegate). So this is a definite improvement, but resulted in bad transforms on instances until we figured out that this had been changed.

Sdr

  • We provide an Sdr plugin for dealing with VEX shaders used by Karma. SdrShaderProperty objects now support an additional piece of metadata that indicates the Sdf data type of the property separate from the Sdr data type (which is much more limited). It's not entirely clear at this point where or how this additional data gets used from a users perspective. The change was detected by a regression test which outputs shader property data types and default values using python.

Epilogue

  • After running through Houdini's regression tests a few additional challenges were found:

    • Render delegates in their HdLight::Sync virtual functions must clear HdChangeTracker::DirtyVisibility from the dirtybits passed itno the Sync call, or the light prim may remain perpetually dirty. Karma used to simply clear HdLight::AllDirty from the dirtybits. Sumitted an issue on github: https://github.com/PixarAnimationStudios/USD/issues/1719

    • It is no longer possible to call SamplePrimvar to get time sampled values on camera primitives to allow "motion blur" of attributes like focal length. Fixing this will require adding explicit code to make camera attributes accessible through HdSceneIndexAdapterSceneDelegate::_SamplePrimvar.

    • For similar reasons it is no longer possible to use the Get method to evaluate the velocities or accelerations attributes on point instancer primitives. These attributes have never been tracked as primvars, but they used to be accessible because of the fallback mechanisms inside UsdImagingSceneDelegate, which would eventually just get the value from the USD attribute. But because point instancer primitives support primvars, the HdSceneIndexAdapterSceneDelegate::Get method will only allow pulling primvars from that primitive. This hopefully will be fixable by registering these attributes as primvars in the UsdImagingPointInstancerAdapter.

License: CC BY 4.0, Copyright Contributors to the ASWF USD working group.