Privately Owned Vehicle Work Group Meeting - 2025/01/13 - Slot 2

[m-zain-khawaja]

Agenda

• Update on SuperDepth Training
• Update on PathDet dataset curation
• Update on LaneDet dataset curation

Discussion

SuperDepth Network Training Update

@m-zain-khawaja :

A first version of SuperDepth was trained using the UrbanSyn and MUAD synthetic datasets as an initial test.

The network achieved impressive results on validation data from UrbanSyn and MUAD which it had not been shown during training, achieving a validation error on mAE of 0.031 overall.

This network was then tested on the KITTI dataset, which the network had not seen before and as expected, due to simulation-to-real domain gap, the validation results were not as robust, with an overall mAE of 0.121. Certain artefacts were also visible in the KITTI estimates caused by light/shadow effects on the road as these were not present in the UrbanSyn/MUAD training data.

In order to address this, a new scheme had to be developed to be able to account for real-world LIDAR based data whilst factoring in the noise characteristics of LIDAR data. Therefore, a 'validity mask' was calculated (1,0 binary mask) highlighting valid depth estimates projected onto the image plane vs non-valid depth estimates projected onto the image plane. The loss function was modified to only account for 'valid' pixels as identified in the validity mask.

Additionally, I was able to successfully parse the DDAD dataset by successfully building Docker on WSL2 (there was a conflict between IP addresses in WSL2 and Docker which had to be resolved - details here allowing me to utilize the Toyota Research Institue DGP Library to correctly project the LIDAR depth to the image plane. This yields a further 16,600 data samples from a combination of the front-facing and rear-facing vehicle cameras.

Phase 2 SuperDepth Training

I have made significant changes to the load_data_super_depth , super_depth_trainer , and augmentations classes to reflect the above strategy of utilizing both simulated data as well as real-world data using LIDAR projected and interpolated depth combined with a validity mask.

New Loss Function:

The new loss function now also includes a gradient matching loss to better preserve the prediction at boundary pixels, this is done by calculating the x and y gradients between the prediction and ground truth and performing an L1 Loss on the gradient difference. This loss was also utilized in DepthAnythingV2

I expect to begin Phase 2 training once the main training loop train_super_depth has been refactored accordingly as well.

PathDet Dataset Curation Update

@siddas27 has implemented an area-under-the-curve checker to filter out trajectories with excessive curvature (likely caused by turns at junctions) as well as filtering out trajectories which begin on the extreme left/right of the image. Furthermore, those trajectories that have too few points are also removed in filtering

@TranHuuNhatHuy is implementing a new cropping pipeline to account for the three different sizes and aspect ratios of images found in the CurveLane dataset so as to unify the data into a single size with the correct, 2:1 aspect ratio

@docjag has implemented the change requests for his PR and is now saving parsed images with an index number in the JSON, as well as adjusting the image size and aspect ratio. @m-zain-khawaja provided some suggestions on how to implement the cropping and aspect ratio adjustment pipeline so as to avoid squishing the image through re-scaling and ensuring that the shapes of objects in the image are preserved.

@sarun-hub has implemented parsing of the BDD100K drivable path data using the colormap instead of the JSON and deriving the drivable path from the colormap image - this is more reliable than the JSON which contained errors. The visualization of the new drivable path derived on this basis looks good, and Sarun is looking into potential filters to remove unsuitable training data samples

Dataset curation tracking

• TuSimple - 6,394 training data samples
• CULane - 15,800 training data samples
• CurveLane - (parsing completed by @TranHuuNhatHuy, data to be uploaded soon)
• BDD100K - (parsing in progress by @sarun-hub)
• ROADWorks - (parsing in progress by @docjag)
• Comma2k19 - (parsing completed by @siddas27, data to be uploaded soon)

LaneDet Dataset Curation Update

• @devang-marvania has begun parsing the CULane dataset for the LaneDet neural network. Data download is currently in progress.

Dataset curation tracking

• TuSimple - 6,394 training data samples
• CULane - (parsing by @devang-marvania)
• CurveLane - to be started

Attendees

• @m-zain-khawaja (Senior Tech Lead at AWF and Privately Owned Vehicle Work Group Chair)
• Sergey Christ (CEO at Alternative Embedded Innovation)
• David Walmroth (Co-Chair of OpenADKit Work Group at AWF and US Director of Business and Project Development at PIX Moving)
• @docjag (Senior Computer Vision and Machine Learning Engineer)
• Aadarsh Tiwari (Software Engineer at KPIT)
• Atanasko Mitrev (Innovation R&D at EXO Service Solutions)
• Anthony Chen (ITRI Taiwan)

Zoom Meeting Video Recording

Video Meeting Link
Please contact the work group lead (@m-zain-khawaja) to request access to a recording of this meeting.