Page Comparison

Table of Contents

Introduction

...

The majority of work in VFX is centered around augmenting existing footage, so it is important to view animation in the context of that footage (it doesn't matter if the animation looks good if it doesn't look integrated). Without this ability, the utility of tools like usdview is quite restricted and requires compositing before work can be evaluated in context. 

Applications outside of VFX:

• TODO:

High Level Anatomy of an Image Plane

An Conceptually, an Image plane must provide the following information:

• link to associated camera
• filepath of image (can be animated per frame)
• fit to filmback (how image is positioned in relation to cameras filmback)     
• visibility (whether image plane is visible or not, because it should be hide-able for different workflows)

And should probably also provide:

• depth (how far away from pinhole camera this should be)    
  • This will also be important for resolving which pixels to render for cases where you have multiple image planes.
• fit to filmback (how should image fit to cameras filmback)     
  • fill/stretch, best, horizontal, vertical, to size
• offset to filmback     x,y (plane should exist in space) 
• alpha gain or other "image attributes"
• more attributes to fine tune fit

Pinhole Camera Diagram

This proposal will follow the same nomenclature from the Cooke site. Here is a reproduced CG Camera diagram. (from https://cookeoptics.com/i-technology/ > "Cooke Camera Lens Definitions for VFX" > CG Camera diagram 2.1.)

Image Added

UsdGeomImagePlaneAPI Schema

The UsdGeomImagePlaneAPI  schema defines basic properties for rendering an image plane. The UsdGeomImagePlaneAPI  schema derives from UsdSchemaBase  and will be a Multi-Apply Schema that would be restricted to UsdGeomCamera prim types.

Properties

• asset image = @@
  • Path to image file
    • can be time sampled per frame
    • This will have to be per frame, as USD/Hydra do not at this time support a $F substitution for image/texture asset paths.
• double depth = 0
  • distance, in scene units, from the pinhole to the point on the optical axis where the image plane sits

    • l_o in above pinhole camera diagram

    • alternative name: "distance" (Autodesk Maya used "depth")
    • A nice default could be -1 or "infinite" so that it would always be behind CG. That might be hard to coexist with a physically placed in scene camera depth that exists in front of and behind cg elements.
• uniform token fit = "vertical"
  • How the image plane is fit to the filmback. Possible values:
    • "horizontal" - fit image width to filmback and keep image aspect ratio
    • "vertical"- fit image height to filmback and keep image aspect ratio
    • "best" - from Maya, but maybe this behavior is too ambiguous
    • "fill" - stretched to filmback
    • "to size" - constant size, centered on filmback, and requiring more data to define "image size"
• float[2] offset
  • in same units as camera filmback - tenth of scene unit)
• visibility (should be hideable for different workflows)

And should probably also provide:

• alpha gain or "texture like attributes"

Open Questions

Should you have the ability to define multiple Image planes per camera? YES

Should the image plane be a separate prim or part of the camera prim? CAMERA

Should the "image" be implemented as a UsdTexture? Or any other part of Image Plane defined as its component part?

Implementation Strategies

...

• bool enabled
  • Control image plane visibility.
    • alternate name: "visible"
    • regardless of this value, if the (camera) prim itself is invisible, the image plane will not be visible in the viewport/render

Why a Multi-Apply API Schema?

If we implement a Multi-Apply API Schema for Image Planes we can:

• author attributes directly to camera
• author multiple image planes to camera, since multi-apply schemas can be applied several times to the same prim    
  • This multiple image planes could be used to view CG elements in context with foreground or background plates.

...

• • and background plates

API Methods

TODO:

• CreateImagePlane(imagePlaneName)
• SetImagePlane*Attr(imagePlaneName, value)
• GetImagePlane*Attr(imagePlaneName, attribute)
• SetImagePlanes([])
• GetImagePlanes()

A Minimum Usage Example:

camera = stage.GetPrimAtPath('/world/cam')
imagePlaneApi = UsdImagePlaneAPIUsdGeomImagePlaneAPI(camera)
imagePlaneApi.SetImagePlaneCreateImagePlane("imagePlane1")
framesToImages = [(f, fileSequence.frameAt(i) for f in fileSequence.frames()]
imagePlaneApi.SetImagePlaneImageAttr(framesToImages, "imagePlane1")

...

def Xform "world" {
    def Camera "cam" (        
        apiSchemas = ["UsdGeomImagePlaneAPI:imagePlane1"]
    ){
        ...
        string[] imagePlanes = ["imagePlane1"]
        asset imagePlane1:image = {
            1001: @/path/to/image.1001.exr@, 
            1002:...
            }
        float imagePlane1:depth = 1000.0
    }
}

Schema Data

...

Hydra Implementation

TODO:

• image: asset
• depth: float
• fit: token
• offset: (float, float)
• visibility: token

...

Flesh out and discuss with Hydra team.

• Use UsdPreviewSurface textures in a way similar to GeomModel texture cards.
• Do the compositing in an Hdx post task

Open Questions

Should you have the ability to define multiple Image planes per camera? YES

Should the image plane be a separate prim or part of the camera prim? CAMERA

Should the "image" be implemented as a UsdTexture? Or any other part of Image Plane defined as its component part?

Alternative Implementation Strategies

As a concrete prim type

Here is a working implementation that was done before Multi-Apply schemas were added to USD. Its a fully featured solution that is based around Autodesk Maya's "underworld" image plane nodes where many planes can live under a camera. This specific implementation is just a quick attempt to move maya image planes across packages and the hydra implementation is just a workaround that generates an HdMesh rprim (instead of creating a new image plane prim type). 

Reference PR from Luma: https://github.com/LumaPictures/USD/pull/1

Pros:

• Prim attributes like [visibility, purpose] can be leveraged to provide intuitive functionality
• Easy to see Image Planes in a hierarchy

Shortfalls:

• Requires relationships between camera + plane primsWould have to traverse below camera to find whether has any image planes 
• Requires an extra API schema to encode the relationship on the Camera.
• There's the possibility that someone targets something other than an ImagePlane with the relationship, so there are more ways to "get stuff wrong".

Using existing USD types (Plane + material + camera + expressions)

It is possible to create a UsdPlane with a material and accomplish the same thing as an image plane.

TODO: Get example usda file.

Shortfalls:

• Requires users to implement complex authoring code.
• No explicit identity as an "Image Plane". This will make it hard to successfully roundtrip from DCCs to USD.

...