Voxel API should conform to Cesium3DTileset interface

[ggetz]

Right now Cesium3DTilesVoxelProvider and other voxel classes are distinct from how other 3D tilesets are used in CesiumJS:

const provider = await Cesium.Cesium3DTilesVoxelProvider.fromUrl(
  "..."
);
const voxelPrimitive = viewer.scene.primitives.add(
  new Cesium.VoxelPrimitive({
    provider: provider,
  })
);

The goal is eventually to bring voxels into 3D Tiles as an extension. As such, we want to treat it in the API like like other 3D Tilesets, and be able to operate using familiar paradigms. Furthermore, the properties and functions associated with other 3D Tilesets should be available for voxel tilesets too, particularly:

• the loading events, like allTilesLoaded, initialTilesLoaded, loadProgress, tileFailed, tileLoad, tileLoad, tilesLoaded, tileVisible, tileUnload.
• statistics (for use in other parts of the API)

Where possible, we should also make sure the documentation is in good shape, and include inline examples where we can.

CC #11018
CC #11008

[lukemckinstry]

Question regarding the approach for reproducing the 3DTileset events for Voxels. Looking at the generated sample dataset we only have a single LOD at 0/0/0. Is this always the case now? Or is this due to the specifics on the source data and/or the specific tiler used?

└── Temperaturmodell-ellipsoid
    ├── subtrees
    │   └── 0
    │       └── 0
    │           └── 0
    │               └── 0.subtree
    ├── tilers-stats.json
    ├── tiles
    │   └── 0
    │       └── 0
    │           └── 0
    │               └── 0.voxel
    └── tileset.json

It seems like the 3DTileset event properties either operate on a per-tile or per-tileset basis. If VoxelPrimitive only uses info for a single LOD, does it make sense to only implement the per-tileset events and not the per-tile events?

• Per-tile events: loadProgress, tileLoad, tileFailed, tileVisible, tileUnload - do not seem to have an equivalent in the current voxel setup.
• Per-tileset events: allTilesLoaded makes sense to keep, and fire when the VoxelPrimitive has loaded (it could be a better description to call it primitiveLoaded, but allTilesLoaded keeps the API naming consistent). initialTilesLoaded seems to be redundant with allTilesLoaded for voxels and no necessary to implement separately.

[jjhembd]

@lukemckinstry I'm not quite sure what you're saying about the LODs. The folders are organized as tiles/LOD/x/y/z.voxel. LOD 0 is the level where the entire tileset bounding volume is spanned by one tile, so as you noticed, the x/y/z coordinates only take the value 0/0/0. The directories for higher LODs will show many more tiles.

For the per-tile events, one point for clarity: Individual tiles are handled very differently between Cesium3DTileset and VoxelPrimitive. I jotted down some quick observations below. This is a pretty rough overview, so don't hesitate to let me know which parts aren't clear.

Current state

Cesium3DTileset

A Cesium3DTileset keeps track of its tiles via 3 arrays:

• _selectedTiles: the tiles selected as relevant to the current scene by Cesium3DTilesetTraversal. This list is re-computed every frame.
• _requestedTiles: the tiles for which an HTTP request has been started. Also re-computed every frame.
• _processingQueue: tiles for which the HTTP request completed successfully, but the data is still being processed and uploaded to the GPU. This list is filtered every frame, to avoid spending resources processing tiles which may have moved out of view in the current scene.

Each Cesium3DTile has a Cesium3DTileContentState to keep track of its status. It also has a priority property and an updatePriority method (called from Cesium3DTilesetTraversal) to make sure tile requests are sent off in the right order.

Every change to a tile in _requestedTiles or _processingQueue results in an update to the main Cesium3DTileset.prototype.statistics.

VoxelPrimitive

A VoxelPrimitive doesn't really have any idea of the tiles used to render the primitive. It delegates all of this to VoxelTraversal, which manages a tree of SpatialNodes, each of which can have several KeyframeNodes if the dataset is time-variant. This tree is updated in VoxelTraversal.prototype.update, which has some parallels with Cesium3DTilesetTraveral.selectTiles.

Requesting data is done from VoxelTraversal, via the VoxelProvider.prototype.requestData function, which is a single asynchronous function that does not resolve or report until the request is complete and some processing is done.

For an analogue to Cesium3DTileset statistics, you may want to look at printDebugInformation in VoxelTraversal. This performs a second traversal over the whole tree (in addition to the one that actually selects tiles for loading) and accumulates the number of KeyframeNodes in each KeyframeNode.LoadState. The results are then printed to the console.

Desired state

Ideally, each voxel tile should be a Cesium3DTile. Tile selection and loading should be done in the same way as they are for Cesium3DTileset--or better yet, VoxelPrimitive should be completely replaced by Cesium3DTileset.

One complication is that the SpatialNode / KeyframeNode architecture supports time-variant data. Switching to Cesium3DTileset would either drop support for time-variant voxels, or require modifications to Cesium3DTileset to incorporate time-variant data.

[jjhembd]

@lilleyse:

• Have we already sketched out an architecture somewhere for time-variant 3D Tiles that are not voxels?
• How much are time-variant voxels used? Would it make sense to switch to Cesium3DTileset for voxels now, and add back time dependence later?

[lukemckinstry]

A few follow ups

1. The sample tiled voxel data I am working with only has LOD 0. The file tree I pasted above is from the sample data. One of my original questions was if having one LOD like this is always the case for voxels or if this data being single LOD is specific to the source data or tiler being used?

• It would be helpful to test with voxel data with multiple LODs if possible.

2. Looking at VoxelProvider.prototype.requestData, running on the same single LOD sample dataset. requestData returns successfully for the top level node level: 0 x,y,z: 0/0/0 but fails for all the child nodes. This makes sense since the data only contains LOD 0. My question is if we would want to raise the tileFailed event in this situation. If the setup is comparable in the 3DTiles traversal (recursively requesting child tiles until they are not present in the tileset) it seems we should follow that.

[ggetz]

My question is if we would want to raise the tileFailed event in this situation. If the setup is comparable in the 3DTiles traversal (recursively requesting child tiles until they are not present in the tileset) it seems we should follow that.

@lukemckinstry We should mirror the behavior for implicit tilesets here. Do we have availability data? If so, let's try to use that to determine if the tile is available. Otherwise I believe implicit tilesets raise the tile failed event.

CC @jjhembd

[lukemckinstry]

It seems we do have availability data for voxels in the tileset.json. It would be helpful to know if it is a required field, is there a spec?
I do not see a reason to call requestData for voxel tiles when tileLevel > availableLevels. I think eliminating this extra level of calls to requestData may be the simplest solution to implement the tileFailed event.

[jjhembd]

There are 2 levels of "conforming to Cesium3DTileset interface" that involve very different levels of effort:

1. Use a similar constructor signature for VoxelPrimitive. For example, instead of requiring the user to construct a VoxelProvider, expose a method like VoxelPrimitive.from3DTilesUrl which would behave just like Cesium3DTileset.fromUrl. This could be achieved relatively easily, by constructing the VoxelProvider inside the .fromUrl method.
2. Use the same Cesium3DTileset class to load voxel data. This would be most logical if voxels are actually a part of the 3D Tiles standard. But it would require us to rethink a lot of internal voxel code.

#12432 will reduce some of the internal barriers to voxels as an actual Cesium3DTileset. Tile data is now managed as a VoxelContent attached to the KeyframeNode, which is the closest voxel analogue to Cesium3DTile. However, there are still some significant architectural differences, including:

• Metadata from all voxel tiles are loaded to the same texture on the GPU and rendered in the same draw call. This is in contrast to Cesium3DTile, where each tile has its own GPU resources and draw calls.
• A Cesium3DTile is more analogous to a voxel SpatialNode, with data for each LOD and spatial location. Voxel data is loaded to KeyframeNodes, several of which can be members of the same SpatialNode. Multiple KeyframeNodes at the same SpatialNode contain colocated data at different timestamps.

Aside from the major architectural differences, some smaller differences might be a good opportunity for lower-effort 'upgrades' to Voxel capabilities:

• Use Cesium3DTilesetCache to manage memory and unload unused voxel tiles.
• Move voxelTraversal._highPriorityKeyframeNodes to VoxelPrimitive, and update the nodes from VoxelPrimitive.prototype.update for better alignment with Cesium3DTileset.prototype.update. Or better yet, use the same set of arrays as Cesium3DTileset: _selectedTiles, _requestedTiles, and _processingQueue (described above in Voxel API should conform to Cesium3DTileset interface #12297 (comment)). These would simplify implementation of the loading events.

[ggetz]

Maybe this was so obvious it didn't need saying, but...

• Support for viewer zoomTo and flyTo methods

[lukemckinstry]

I found some additional items to track for performance & profiling purposes. These are statistics from Cesium3DTilesetStatistics which may need to be tracked in VoxelPrimitive as they are in Cesium3DTileset.

• geometryByteLength
• textureByteLength
• batchTableByteLength
• visited
• selected
• numberOfCommands
• numberOfTilesWithContentReady
• numberOfTrianglesReady

I believe I see a clear path to implementation for textureByteLength, visited, selected, numberOfCommands and numberOfTilesWithContentReady.

I am less sure about geometryByteLength, batchTableByteLength, numberOfTrianglesReady. I could use help understanding if these concepts apply to voxels.

[jjhembd]

@lukemckinstry I think you have identified the relevant ones: textureByteLength, visited, selected, numberOfTilesWithContenReady.

numberOfCommands may not be important here, because voxel rendering draws all tiles in the scene in a single draw call. This is different from other kinds of 3D Tiles, which might have several draw commands for a single tile.

geometryByteLength and numberOfTrianglesReady are only relevant for mesh models, and don't really apply to voxels. batchTableByteLength is for per-vertex metadata--again, for mesh models.

Voxels will have all relevant metadata stored in the textures, so textureByteLength is the most important one for estimating memory use.