Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

data preprocess of nuplan #58

Open
lemonci opened this issue Nov 5, 2024 · 1 comment
Open

data preprocess of nuplan #58

lemonci opened this issue Nov 5, 2024 · 1 comment

Comments

@lemonci
Copy link

lemonci commented Nov 5, 2024 •
edited
Loading

According to your documentation,

This is the output of tree --filelimit 10 ./raw

./raw
├── maps
│   ├── nuplan-maps-v1.0.json
│   ├── sg-one-north
│   │   └── 9.17.1964
│   │       └── map.gpkg
│   ├── us-ma-boston
│   │   └── 9.12.1817
│   │       └── map.gpkg
│   ├── us-nv-las-vegas-strip
│   │   └── 9.15.1915
│   │       ├── drivable_area.npy.npz
│   │       ├── Intensity.npy.npz
│   │       └── map.gpkg
│   └── us-pa-pittsburgh-hazelwood
│       └── 9.17.1937
│           └── map.gpkg
└── nuplan-v1.1
    ├── sensor_blobs
    │   ├── 2021.05.12.22.00.38_veh-35_01008_01518
    │   │   ├── CAM_B0  [5100 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_F0  [5100 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L0  [5100 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L1  [5100 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L2  [5100 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R0  [5100 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R1  [5100 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R2  [5100 entries exceeds filelimit, not opening dir]
    │   │   └── MergedPointCloud  [10200 entries exceeds filelimit, not opening dir]
    │   ├── 2021.05.12.22.28.35_veh-35_00620_01164
    │   │   ├── CAM_B0  [5440 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_F0  [5440 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L0  [5440 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L1  [5440 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L2  [5440 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R0  [5440 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R1  [5440 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R2  [5440 entries exceeds filelimit, not opening dir]
    │   │   └── MergedPointCloud  [10880 entries exceeds filelimit, not opening dir]
    │   ├── 2021.05.12.23.36.44_veh-35_00152_00504
    │   │   ├── CAM_B0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_F0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L1  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L2  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R1  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R2  [3520 entries exceeds filelimit, not opening dir]
    │   │   └── MergedPointCloud  [7040 entries exceeds filelimit, not opening dir]
    │   ├── 2021.05.12.23.36.44_veh-35_01133_01535
    │   │   ├── CAM_B0  [4020 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_F0  [4020 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L0  [4020 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L1  [4020 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L2  [4020 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R0  [4020 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R1  [4020 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R2  [4020 entries exceeds filelimit, not opening dir]
    │   │   └── MergedPointCloud  [8040 entries exceeds filelimit, not opening dir]
    │   ├── 2021.05.12.23.36.44_veh-35_02035_02387
    │   │   ├── CAM_B0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_F0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L1  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L2  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R0  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R1  [3520 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R2  [3520 entries exceeds filelimit, not opening dir]
    │   │   └── MergedPointCloud  [7040 entries exceeds filelimit, not opening dir]
    │   ├── 2021.05.25.14.16.10_veh-35_01690_02183
    │   │   ├── CAM_B0  [4930 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_F0  [4930 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L0  [4930 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L1  [4930 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_L2  [4930 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R0  [4930 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R1  [4930 entries exceeds filelimit, not opening dir]
    │   │   ├── CAM_R2  [4930 entries exceeds filelimit, not opening dir]
    │   │   └── MergedPointCloud  [9860 entries exceeds filelimit, not opening dir]
    │   └── 2021.06.03.12.02.06_veh-35_00233_00609
    │       ├── CAM_B0  [3760 entries exceeds filelimit, not opening dir]
    │       ├── CAM_F0  [3760 entries exceeds filelimit, not opening dir]
    │       ├── CAM_L0  [3760 entries exceeds filelimit, not opening dir]
    │       ├── CAM_L1  [3760 entries exceeds filelimit, not opening dir]
    │       ├── CAM_L2  [3760 entries exceeds filelimit, not opening dir]
    │       ├── CAM_R0  [3760 entries exceeds filelimit, not opening dir]
    │       ├── CAM_R1  [3760 entries exceeds filelimit, not opening dir]
    │       ├── CAM_R2  [3760 entries exceeds filelimit, not opening dir]
    │       └── MergedPointCloud  [7492 entries exceeds filelimit, not opening dir]
    └── splits
        ├── mini
        │   ├── 2021.05.12.22.00.38_veh-35_01008_01518.db
        │   ├── 2021.06.09.17.23.18_veh-38_00773_01140.db
        │   ├── 2021.07.09.20.59.12_veh-38_01208_01692.db
        │   └── 2021.10.05.07.10.04_veh-52_01442_01802.db
        └── trainval  [60 entries exceeds filelimit, not opening dir]

And the error I got is:

Traceback (most recent call last):
  File "/drivestudio/datasets/preprocess.py", line 237, in <module>
    dataset_processor = NuPlanProcessor(
  File "drivestudio/datasets/nuplan/nuplan_preprocess.py", line 103, in __init__
    process_log_list.append(self.nuplandb_wrapper.log_names[idx])
TypeError: list indices must be integers or slices, not str

So I tried to printout process_id_list:

process_id_list: ['2021.05.12.22.00.38_veh-35_01008_01518', '2021.05.12.22.28.35_veh-35_00620_01164', '2021.05.12.23.36.44_veh-35_00152_00504', '2021.05.12.23.36.44_veh-35_01133_01535', '2021.05.12.23.36.44_veh-35_02035_02387', '2021.05.25.14.16.10_veh-35_01690_02183']

Please suggest if the way to arrange the directories in raw needs to be changed. Thanks.
@larstonder
Copy link

It seems like appending the idx's directly to the process_log_list might solve the issue, instead of appending the wrapper's log names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@larstonder @lemonci