Introduction
This proposal aims to add native support for Image Planes to USD. An Image Plane is a camera constrained image that can be seen in "screen space" when looking through that camera.
The majority of work in VFX is centered around augmenting existing footage, so it is important to view animation in the context of that footage (it doesn't matter if the animation looks good if it doesn't look integrated). Without this ability, the utility of tools like usdview is quite restricted and requires compositing before work can be evaluated in context.
Applications outside of VFX:
- TODO:
Anatomy of an Image Plane
An Image plane must provide the following information:
- link to camera
- filepath of image (can be animated per frame)
- depth (how far away from camera this should be)
- This will also be important for resolving which pixels to render for cases where you have multiple image planes.
- fit to filmback (how should image fit to cameras filmback)
- fill/stretch, best, horizontal, vertical, to size
- offset to filmback
- x,y (in same units as camera filmback - tenth of scene unit)
- visibility (should be hideable for different workflows)
And should probably also provide:
- alpha gain or "texture like attributes"
Open Questions
Should you have the ability to define multiple Image planes per camera? YES
Should the image plane be a separate prim or part of the camera prim? CAMERA
Should the "image" be implemented as a UsdTexture? Or any other part of Image Plane defined as its component part?
Implementation Strategies
Proposed: As a Multi-Apply Schema
If we implement a Multi-Apply API Schema for Image Planes we can:
- author attributes directly to camera
- author multiple image planes to camera
- This could be used to view CG elements in context with foreground or background plates.
A Minimum example (with sample API):
camera = stage.GetPrimAtPath('/world/cam')
imagePlaneApi = UsdImagePlaneAPI(camera)
imagePlaneApi.SetImagePlane("imagePlane1")
framesToImages = [(f, fileSequence.frameAt(i) for f in fileSequence.frames()]
imagePlaneApi.SetImagePlaneImageAttr(framesToImages, "imagePlane1")
Which would generate this .usda:
def Xform "world" {
def Camera "cam" {
...
string[] imagePlanes = ["imagePlane1"]
asset imagePlane1:image = {
1001: @/path/to/image.1001.exr@,
1002:...
}
float imagePlane1:depth = 1000.0
}
}
Schema Data
The nuts and bolts of the data types proposed are:
TODO:
- image: asset
- depth: float
- fit: token
- offset: (float, float)
- visibility: token
Alternative Strategies
As a concrete prim type
Here is a working implementation that was done before Multi-Apply schemas were added to USD. Its a fully featured solution that is based around Autodesk Maya's "underworld" image plane nodes where many planes can live under a camera. This specific implementation is just a quick attempt to move maya image planes across packages and the hydra implementation is just a workaround that generates an HdMesh rprim (instead of creating a new image plane prim type).
Reference PR from Luma: https://github.com/LumaPictures/USD/pull/1
Pros:
- [visibility, purpose] can be leveraged to provide intuitive functionality
Shortfalls:
- Requires relationships between camera + plane prims
- Would have to traverse below camera to find whether has any image planes
Using existing USD types (Plane + material + camera + expressions)
It is possible to create a UsdPlane with a material and accomplish the same thing as an image plane.
TODO: Get example
Shortfalls:
- Requires users to implement complex authoring code.
- No explicit identity as an "Image Plane". This will make it hard to successfully roundtrip from DCCs to USD.
Example Use Cases
Hydra / Usdview
It would be helpful to be able to view CG elements against a backdrop of the production plate when checking exported usd caches.
DCC Interchange
Having a "UsdLux" style standard for Image Planes would be very useful when implementing importers and exporters for DCCs like Maya, Blender, Nuke, etc.