[Bug] Rendering crashes when rendering multiple viewports at high resolution

[deerzlg]

Describe the Bug

I want to use Cornerstone3D to render multiple viewports. When I increase the resolution of these viewports, I've discovered that the rendering crashes once it reaches a certain size.
Here is a minimal reproduction using cornerstone3D example. When I increased the gridContainer.style.width to 1200px. the rendering crashed. (windows11, gtx1080, edge, 4K monitor scaling175%):

import { RenderingEngine, Types, Enums } from '@cornerstonejs/core';
import {
  initDemo,
  createImageIdsAndCacheMetaData,
  ctVoiRange,
} from '../../../../utils/demo/helpers';

// This is for debugging purposes
console.warn(
  'Click on index.ts to open source code for this example --------->'
);

const { ViewportType } = Enums;

const content = document.getElementById('content');
content.style.height = '100vh';
content.style.display = 'flex';
content.style.justifyContent = 'center';
content.style.alignItems = 'center';

const gridContainer = document.createElement('div');
gridContainer.id = 'grid-container';
gridContainer.style.display = 'grid';
gridContainer.style.gridTemplateColumns = 'repeat(8, 1fr)';
gridContainer.style.gridTemplateRows = 'repeat(8, 1fr)';
gridContainer.style.width = '1200px';
gridContainer.style.height = '100%';

content.appendChild(gridContainer);
// ============================= //

/**
 * Runs the demo
 */
async function run() {
  // Init Cornerstone and related libraries
  await initDemo();

  // Get Cornerstone imageIds and fetch metadata into RAM
  const imageIds = await createImageIdsAndCacheMetaData({
    StudyInstanceUID:
      '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463',
    SeriesInstanceUID:
      '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561',
    wadoRsRoot: 'https://d3t6nz73ql33tx.cloudfront.net/dicomweb',
  });

  // Instantiate a rendering engine
  const renderingEngineId = 'myRenderingEngine';
  const renderingEngine = new RenderingEngine(renderingEngineId);

  // Create 64 stack viewports
  for (let i = 0; i < 64; i++) {
    const element = document.createElement('div');
    element.style.width = '100%';
    element.style.height = '100%';
    gridContainer.appendChild(element);

    const viewportId = `CT_STACK_${i}`;
    const viewportInput: Types.PublicViewportInput = {
      viewportId,
      type: ViewportType.STACK,
      element,
    };

    renderingEngine.enableElement(viewportInput);

    const viewport = renderingEngine.getViewport(
      viewportId
    ) as Types.IStackViewport;

    const stack = [imageIds[0]];

    // Set the stack on the viewport
    await viewport.setStack(stack);

    // Set the VOI of the stack
    viewport.setProperties({ voiRange: ctVoiRange });

    // Render the image
    viewport.render();
  }
}

run();

Steps to Reproduce

1. Render 8x8 = 64 grid stack viewports
2. Increase the overall grid resolution
3. When using 4K monitor, windows scale 175%

The current behavior

There were no errors in the console, but the rendering crashed and the Cornerstone tool also malfunctioned.

The expected behavior

It seems that the rendering crash is also related to the scaling size setting of Windows. Is there any relevant information to solve this problem or methods for identifying the problem. Thank you very much!

OS

Windows11

Node version

18.19.2

Browser

Edge 129.0.2792.52

[sedghi]

Can you check the latest OHIF viewer, which now includes Cornerstone 3D 2.0?

Here are the migration guides:

Try OHIF locally: https://viewer-dev.ohif.org/localbasic
Try Cornerstone3D 2.0 locally: https://www.cornerstonejs.org/live-examples/local.html

https://docs.ohif.org/migration-guide/3p8-to-3p9/

[deerzlg]

I tested using cornerstone3D 2.0 and found that when rendering 64 viewports and reaching a certain overall resolution, it still crashes

Then I used the latest version of OHIF-v3 for testing, and as the number of windows increased, crashes also occurred

[sedghi]

Do you have real need for 64 viewports? what is your usecase?

[deerzlg]

I have developed a web for medical imaging film printing, which is commonly used in 8X8 format. The use of viewport is also for the convenience of adjusting individual images. Do you have any better suggestions? Thank you very much

[sedghi]

Can you provide a simple example of using Cornerstone3D that I can run locally? There shouldn't be any issues since we use one texture, render it offscreen, and copy it over each viewport if the image is the same. If the image differs, that might be a separate issue, possibly exceeding the WebGL maximum texture size limit.

[deerzlg]

The following code is a minimal reproduction that can run in the cornerstone3D repo's example. When using 4K monitor scaling175%, it will crash.

import { RenderingEngine, Types, Enums } from '@cornerstonejs/core';
import {
  initDemo,
  createImageIdsAndCacheMetaData,
  ctVoiRange,
} from '../../../../utils/demo/helpers';

// This is for debugging purposes
console.warn(
  'Click on index.ts to open source code for this example --------->'
);

const { ViewportType } = Enums;

const content = document.getElementById('content');
content.style.height = '100vh';
content.style.display = 'flex';
content.style.justifyContent = 'center';
content.style.alignItems = 'center';

const gridContainer = document.createElement('div');
gridContainer.id = 'grid-container';
gridContainer.style.display = 'grid';
gridContainer.style.gridTemplateColumns = 'repeat(8, 1fr)';
gridContainer.style.gridTemplateRows = 'repeat(8, 1fr)';
gridContainer.style.width = '1200px';
gridContainer.style.height = '100%';

content.appendChild(gridContainer);
// ============================= //

/**
 * Runs the demo
 */
async function run() {
  // Init Cornerstone and related libraries
  await initDemo();

  // Get Cornerstone imageIds and fetch metadata into RAM
  const imageIds = await createImageIdsAndCacheMetaData({
    StudyInstanceUID:
      '1.3.6.1.4.1.14519.5.2.1.7009.2403.334240657131972136850343327463',
    SeriesInstanceUID:
      '1.3.6.1.4.1.14519.5.2.1.7009.2403.226151125820845824875394858561',
    wadoRsRoot: 'https://d3t6nz73ql33tx.cloudfront.net/dicomweb',
  });

  // Instantiate a rendering engine
  const renderingEngineId = 'myRenderingEngine';
  const renderingEngine = new RenderingEngine(renderingEngineId);

  // Create 64 stack viewports
  for (let i = 0; i < 64; i++) {
    const element = document.createElement('div');
    element.style.width = '100%';
    element.style.height = '100%';
    gridContainer.appendChild(element);

    const viewportId = `CT_STACK_${i}`;
    const viewportInput: Types.PublicViewportInput = {
      viewportId,
      type: ViewportType.STACK,
      element,
    };

    renderingEngine.enableElement(viewportInput);

    const viewport = renderingEngine.getViewport(
      viewportId
    ) as Types.IStackViewport;

    const stack = [imageIds[0]];

    // Set the stack on the viewport
    await viewport.setStack(stack);

    // Set the VOI of the stack
    viewport.setProperties({ voiRange: ctVoiRange });

    // Render the image
    viewport.render();
  }
}

run();

[sedghi]

Yeah i can see it, seems like if i make the gridContainer.style.width = '1000px'; it can render 64. Not sure really why

Although i did use the setViewport API

const viewportInputs = [];
  for (let i = 0; i < 64; i++) {
    const element = document.createElement('div');
    element.style.width = '100%';
    element.style.height = '100%';
    gridContainer.appendChild(element);

    const viewportId = `CT_STACK_${i}`;
    const viewportInput: Types.PublicViewportInput = {
      viewportId,
      type: ViewportType.STACK,
      element,
    };
    viewportInputs.push(viewportInput);
  }

  renderingEngine.setViewports(viewportInputs);

[deerzlg]

Yes, this problem only occurs when the resolution reaches a certain size. In addition, my usage scenario mainly involves 64 different images, which is limited by the WebGL maximum texture size. Therefore, I can only rely on limiting the size of elements to avoid crashing, right? Thank you for your response.

[sedghi]

I guess we need to take a look at how much we can increase the resolution of one viewport until it breaks. That should be important.