3d-deep-learning

3D Deep Learning works

Tasks

3D Representation

Spherical CNNs

• Taco S. Cohen, Spherical CNNs, ICLR 2018 Best paper [paper]
• Learning SO(3) Equivariant Representations with Spherical CNNs [paper] [code]
• Deep Learning Advances on Different 3D Data Representations: A Survey [paper]

3D Classification

Datasets

• ModelNet10/40

Networks

• 3D CNN
  • 3D-DenseNet
  • Voxnet: A 3d convolutional neural network for real-time object recognition, IROS 2015. [code] [paper]
  • [3D-NIN, network in network]
  • VRN Ensemble, Generative and discriminative voxel modeling with convolutional neural networks, arxiv [paper] [code]
    • Voxception-Resnet Blocks
• 2D CNN - MVCNN, Learned-Miller.Multi- view convolutional neural networks for 3d shape recognition, ICCV2015 [project] [code] [paper][data] [video]
• Point
  • PointNet [project][paper][code][video][slides]
    • global pooling
    • T-net
  • PointNet++ [paper] [code]
    • sampling & grouping to learning local feature for fine-gaint objects
    • two PointNet
• Graph/tree-based
  • Kd-Net, scape from cells: Deep kd- networks for the recognition of 3d point cloud models, arxiv2017 [paper]
    • kd-tree
  • Octnet: Learning deep 3d representations at high resolutions, CVPR2017
    • octree
  • O-cnn: Octree-based convolutional neural networks for 3d shape analysis, TOG2017
    • octree
  • SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018 [paper] [code]
    • point-to-node kNN search Self-Organizing Map (SOM)
  • KCNet, Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling, CVPR2018 [project] [code] [paper][data] [video]
    • Kernel Correlation
    • Graph Pooling

3D Segmentation

Datasets

• HVSMR
• BRATS Data
• ShapeNet

Networks

• HeartSeg, 3D-FC-Densenet, Automatic 3D Cardiovascular MR Segmentation with Densely-Connected Volumetric ConvNets - MICCAI 2017 - [code]
• 3D-Unet [paper]
• ClusterNet: 3D Instance Segmentation in RGB-D Images [paper]
• PointNet [project][paper][code][video][slides]
• PointNet++ [paper] [code]
• KCNet, Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling, CVPR2018 [project] [code] [paper][data] [video]
• SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018 [paper] [code]
• 3D Shape Segmentation with Projective Convolutional Networks. CVPR2017. Project Poster Presentation

3D Object Detection

Datasets

Data types: RGBD, Flow, Laser

• KITTI
• KITTI Object Visualization Tool

Networks

• MV3D, Multi-View 3D Object Detection Network for Autonomous Driving [paper] [code]
• Avod, Joint 3D Proposal Generation and Object Detection from View Aggregation [paper] [code]
• F-PointNet, Frustum PointNets for 3D Object Detection from RGB-D Data [paper] [code]
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection [paper]
• PIXOR: Real-time 3D Object Detection from Point Clouds - CVPR 2018 - [paper] [code]
• 3D Object Proposals using Stereo Imagery for Accurate Object Class Detection [paper]
• 3D Bounding Box Estimation Using Deep Learning and Geometry [paper] [code]
• Learning 3D Object Orientations From Synthetic Images

3D Reconstruction & Generation

Datasets

Data types: RGBD, Flow, Laser

• ShapeNet

Networks

• SO-Net: Self-Organizing Network for Point Cloud Analysis, CVPR2018 [paper] [code]
• 3D-GAN [paper] [code]
• Generating 3D Adversarial Point Clouds [paper]

3D Human Pose Estimation

Datasets

Data types: RGBD, Flow, Laser

• ShapeNet

Networks

• Synthetic Occlusion Data Augmentation -2018 ECCV PoseTrack Challenge - [paper] [code]
• Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach - ICCV 2017 - [paper] [code] [code-pytorch
• 3D human pose estimation from depth maps using a deep combination of poses [paper]

CVPR2016 Tutorial: 3D Deep Learning with Marvin

• CVPR2016 Tutorial: 3D Deep Learning with Marvin
• 3D Shape Retrieval
• C3D, website
• [Video Caffe(C3D)] [code]
• [DeepMedic, Brain Lesion Segmentation] [[code(https://github.com/Kamnitsask/deepmedic)]
• 3D Keypoint Detection and Feature Description

Codes and libs for 3D

• util3d
• spectral-lib
• 3D-Caffe
• 3D-ResNets-PyTorch

DL on Medical Image

• Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples

More 3D Papers

see [ 3D-Machine-Learning]