Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial support for 3D Gaussian Splatting #244

Merged
merged 29 commits into from
Oct 13, 2023
Merged

Add initial support for 3D Gaussian Splatting #244

merged 29 commits into from
Oct 13, 2023

Conversation

slimbuck
Copy link
Member

@slimbuck slimbuck commented Oct 13, 2023

This PR adds initial support for rendering Guassian Splats in the model-viewer.

The bicycle scene in model-viewer:
Screenshot 2023-10-13 at 14 04 03

This PR adds the following

  • a streamed loader for binary PLY files in little endian
  • a container resource for managing and rendering splats
  • a web worker for sorting splats at runtime

Limitations

  • only supports a single PLY
  • artifacting with odd viewport size
  • rendering the 0'th spherical harmonic only
  • sorting update is slow completely done on CPU

Next Steps

  • support compressed file format / runtime data
  • improve rendering and sorting performance
  • support operations on splat data:
    • culling/cropping
    • combining
    • effects
    • lighting

References

@slimbuck slimbuck added the enhancement New feature or request label Oct 13, 2023
@slimbuck slimbuck requested a review from a team October 13, 2023 13:09
@slimbuck slimbuck self-assigned this Oct 13, 2023
const zMinMax = minmax(z);

const aabb = new BoundingBox();
aabb.setMinMax(new Vec3(xMinMax[0], yMinMax[0], zMinMax[0]), new Vec3(xMinMax[1], yMinMax[1], zMinMax[1]));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could probably just use Bounds.computeMinMax

Copy link
Contributor

@willeastcott willeastcott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing!! 🚀

@willeastcott willeastcott changed the title Add initial support for gaussian splats Add initial support for 3D Gaussian Splatting Oct 13, 2023
@slimbuck
Copy link
Member Author

I added a slightly hacky camera focal point calculatoin so zooming the camera in takes you to the center of interest.

@slimbuck slimbuck merged commit b7c47d8 into main Oct 13, 2023
@slimbuck slimbuck deleted the ply-support branch October 13, 2023 14:07
@mvaligursky
Copy link
Contributor

Please list references used to create this as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants