How to generate mesh_hand and mesh_obj folders

[karta2155802]

Hello, I'm insteresting in your work and trying to run it on my computer.
But I don't know how to generate mesh_hand and mesh_obj folders in dataset preprocessing part. Which part of the code in tool should I use?
Thanks!

[zerchen]

Hi,

Thanks for your interests! For ObMan, I mainly use this toolbox. For the mesh_obj folder, you could have a look at this function about how to load object vertices and faces. Since ObMan annotations provide hand vertices, you could use the closed MANO faces here for the hand mesh. When getting vertices and faces, I use trimesh to export the final mesh. For DexYCB, I mainly use the script provided by the official toolbox to generate the hand mesh and the object mesh.
Hope it helps.

[karta2155802]

Hi,

Thank you very much for answering me.
Because I'm new in this domain, after few days trying, I'm still confusing about the data format, file extention and layout in mesh_hand and mesh_obj folders. Can you please give an example?
And for DexYCB, the script only gives some information of the 1000th example in each split, e.g., label file, mano_side, mano_betas, etc. It cannot directly generate meshes. Does this mean I need to generate meshes with the mano parameters by myself?

[zerchen]

The main script is shown in following lines. You need to put this .py under this repo. For DexYCB, you need to modify the script and generate meshes from mano parameters yourself.

import os
from tqdm import tqdm
import numpy as np
import open3d
from obman.obman import ObMan

MANO_WATERTIGHT_OBJ_PATH = './MANO_WATERTIGHT.obj'

def generate_open3d_mesh(vertices, faces, category = 'hand'):
mesh = open3d.geometry.TriangleMesh()
mesh.vertices = open3d.utility.Vector3dVector(np.copy(vertices))
if category == 'hand':
mesh.triangles = open3d.utility.Vector3iVector(np.copy(faces))
elif category == 'obj':
mesh.triangles = open3d.utility.Vector3iVector(np.copy(faces))
return mesh

parser = argparse.ArgumentParser()
parser.add_argument('--root', default="/home2/zerui/dataset/obman/obman", required=False, help="Path to dataset root")
parser.add_argument(
'--shapenet_root', default="/home2/zerui/dataset/ShapeNetCore.v2", required=False, help="Path to root of ShapeNetCore.v2")
parser.add_argument(
'--split', type=str, default='train', help='Usually [train|test]')
parser.add_argument('--sides', type=str, default='left')
args = parser.parse_args()

pose_dataset = ObMan(args.root, args.shapenet_root, split=args.split, use_cache=False)
os.makedirs(os.path.join(args.root, args.split, 'mesh_hand'), exist_ok=True)
os.makedirs(os.path.join(args.root, args.split, 'mesh_obj'), exist_ok=True)

for i in tqdm(range(len(pose_dataset.image_names))):
filename = pose_dataset.image_names[i].split('/')[-1].split('.')[0]
hand_verts3d = pose_dataset.get_verts3d(i)
hand_faces = np.load('closed_fmano.npy')
obj_verts3d, obj_faces = pose_dataset.get_obj_verts_faces(i)
hand_mesh = generate_open3d_mesh(hand_verts3d, hand_faces, 'hand')
obj_mesh = generate_open3d_mesh(obj_verts3d, obj_faces, 'obj')
open3d.io.write_triangle_mesh(os.path.join(args.root, args.split, 'mesh_hand', filename +'.obj'), hand_mesh)