Channel should include display/rendering settings

[rleigh]

Currently in OME-XML the Channel element includes a single display setting attribute: channel colour. It should also include additional display settings, either in addition to colour, or via a separate contained element which will also include colour information.

Rationale: This information is encoded in many different file formats and we currently have no way to represent this in a standardised form, or to export the rendering settings from OMERO. This is of particular importance where the same rendering settings are required over a collection of images for valid scientific interpretation. Case in point: a user uploading images to the JCB dataviewer carefully set the rendering settings identically on groups of related images (controls and treatments) for valid comparison and these were lost upon import. While it's possible to manually set all the rendering settings after import, that's far from ideal and in the case of large datasets and screens is also impractical. We should be able to make use of the data in the original file.

Information this must include at a minimum:

• Channel colour
• Minimum value
• Maximum value

This will allow standard linear contrast adjustments to be made.

Additional desirable values:

• Transfer function type (linear/gamma/log/exponential/polynomial)
• Transfer coefficient(s)
• Lookup tables

Are gamma and polynomial the same thing in this context? I see that the OMERO API defines polynomial but not gamma, but gamma is the typical correction used for visualisation. If the coefficient is not 1.0, then we could default to using gamma correction if no transfer function type is specified (1.0 is linear for gamma).

Several file formats encode per-channel lookup tables. Typically an array of 2^{8 or 2}16 RGB colour values. Might be worth looking at the requirements here so that bioformats can save them correctly.

In order to facilitate the sharing and reuse of display/rendering settings across multiple images it would be nice to refer to them by name/LSID/UUID so that any changes to the settings would be reflected across all images referencing it.

[mlinkert]

See ​https://www.openmicroscopy.org/qa2/qa/feedback/7335 for an example of files that could benefit from having a place to store display/rendering settings.

[rleigh]

Note that the rendering settings could also be considered to be somewhat application/renderer-specific. The above are fairly general. However, when we get to full 3D/4D volume rendering and raytracing and surface rendering the renderers do have a wide variety of parameters which may be used for the same channel in different rendering modes. Examples:

Opacity

• Brightness, Density, Black Level (opacity accumulated per voxel along ray path)

Intensity

• Intensity (accumulated along ray path)

Isosurface

• Brightness
• Black level
• Surface type (e.g. matte/metal/glass)

Raytracing

• Light source orientation(s) and type(s)
• Same rendering settings as for opacity

These are for Volocity Visualisation, and are application-specific. However, all the advanced rendering is appplied after the initial basic linear/gamma transforms, so these are still entirely applicable. So the main point here is really just that having a way to add custom annotations to the rendering settings and/or channel would be useful to allow such extra stuff to be added.

[rleigh]

Test normalised transfer functions

[rleigh]

Visualisation of normalised transfer functions

[rleigh]

See attached PDF. Thoughts for generalised rendering, splitting up existing OMERO renderer into several steps to allow support of arbitrary pixel type/range:

1. Normalise. Clamp pixel intensity between the user-defined min and max and normalise, transforming the value to the range [0,1]. Will allow support of signed int and float types. Default: range for pixel type if integer; [0,1] for float.

2. Transfer. Apply a user-defined transfer function with optional coefficient(s) (default: linear). See attached transfer functions. In implementation: need range of coef(s) and scale type (log/lin) for sliders. Initially define: linear, gamma, exponential, logarithmic. Formulas need checking, log in particular.

3. Lookup colour. Can be linear colour ramp LUT/fixed colour with alpha ramp LUT/arbitrary LUT. LUT should be of output pixel format or float. Default: (alpha?) ramp of channel colour.

Implementations can combine all steps to precompute a single LUT for efficiency and to minimise rounding error.

Questions/thoughts:

• How to cope with complex? Clamping possible? Complex transfer funcs? 2D LUTs?
• Can LUTs include transparency? (Might be defined separately depending on the renderer?)
• 2D/3D transfer functions and/or LUTs for complex rendering?
• When using more complex 2D/3D LUTs, will need ROI equivalent to select area(s) (rather than range as for min/max here); can the min/max here be generalised as a line ROI equivalent?

[rleigh]

Rendering position.

With large/big/tiled images, we are essentially describing a viewport onto the image(s). From the GL perspective, this is the definition of the projection matrix and includes:

• projection type (orthographic, perspective)
• camera position
• camera focus/look position (where in the world the camera is looking from the camera position)
• camera orientation (rotation around the camera vector)
• zoom/size/depth i.e. size for ortho, frustum limits for perspective

Currently OMERO allows zooming in and out with transparent use of subresolutions. This would replace it with an essentially infinite zoom resolution, though the UI might want to implement upper and lower zoom bounds and/or sticking points at standard zoom levels/subres levels e.g 1:2, 2:1, 1:2 etc. The UI might also clamp the camera to certain position/look bounds to avoid moving far outside the immediate sample area in the worldspace. The worldspace should be in physical units where available, else use pixels.

Less sophisticated renderers can ignore parts they can't use. E.g. the existing OMERO renderer could simply use the look position as the view centre and the nearest possible zoom level.

[rleigh]

For visualisation of timeseries we might also want to include a "frame rate" property to control the speed of progression through the series. Note that this is currently being manually specified with the IFormatReader interface.

We might also wish to consider moving the rendering settings mostly or completely out of Channel to a higher-level element to allow multiple rendering settings for a given series and to also allow for channels to be rendered differently or included/excluded from particular renderings. And for properties such as framerate, these are not tied to any individual channel; likewise for most of the properties mentioned above not tied to a specific channel.

[jburel]

• Multiple rendering settings per image is already possible
• excluding channels is also possible: turn off the channel, it will not be rendered.
• I do not think frame rate falls into the category of rendering settings. The settings stored information about what to map.

[rleigh]

We do need somewhere in the model for the frame rate to go, since this is currently hacked directly into the IFormatWriter interface. When it comes to exporting movies, this most certainly is a rendering setting.

[jburel]

Frame rate won't be used when we displayed the image.
If it is read from the file using BF, then it probably needs to be treated as "acquisition metadata."
We have on pixels a field timeincrement, so that could be a place where we can add a new field
When we export movie (via our movie script) frame rate can be specified but it is not stored. I think we are mixing 2 things here.

[ajpatterson]

Closed until ready for future work. Listed on trello card: ​https://trello.com/c/HmCOzJ8c/153-schema-changes-trac-tickets