Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How did you added your own asset to the Carla ? #20

Open
ronyshaji opened this issue Oct 28, 2022 · 18 comments
Open

How did you added your own asset to the Carla ? #20

ronyshaji opened this issue Oct 28, 2022 · 18 comments

Comments

@ronyshaji
Copy link

I have a doubt regarding the addition of your own asset. Did you edited the carla source or added your own asset to it so that it appears after build ?

@LeeYiyuan
Copy link
Contributor

We created a dynamic mesh asset that was burned into the compiled binaries. Then, the CARLA plugin was modified to allow us to spawn PNG images (map tiles) as instances of these dynamic mesh assets, by displaying the images on the surface of the tiles.

@ronyshaji
Copy link
Author

SO you write the c++ code and compiled while build ?

@LeeYiyuan
Copy link
Contributor

Yes -- we modified CARLA's plugin in UE to include the meshes. You may refer to the source code here.

@ronyshaji
Copy link
Author

ronyshaji commented Oct 31, 2022

So if you modified the CARLA plugin from UE, why we still need to do this ? I mean copy from your CustomAssets folder to the carla after build ?

Copy <summit_root>/CustomAssets/M_Tile.uasset to <summit_root>/Unreal/CarlaUE4/Content/Carla/Static/GenericMaterials/Ground/M_Tile.uasset

@LeeYiyuan
Copy link
Contributor

The file defines the dynamic mesh tile asset, which the plugin creates instances of. Thus, both need to be present.

@ronyshaji
Copy link
Author

Well...is nativising the blueprint works ? I mean nativize my blueprint and build the simulator without copying .uasset file ?

@LeeYiyuan
Copy link
Contributor

LeeYiyuan commented Nov 2, 2022

The compiled simulator can't load anything that it does not have access to when it was compiled. Hence, if you exclude a blueprint and compile, you will not be able to use it in the simulator.

@ronyshaji
Copy link
Author

@LeeYiyuan So if i need to include a blueprint from unreal engine to the Summit(CARLA), you have to write it in c++ and compile while building, is that right ?

@LeeYiyuan
Copy link
Contributor

LeeYiyuan commented Nov 18, 2022

You will have to, depending on how you intend to access your blueprint, do the following:

  1. include the blueprint in the resources folder,
  2. load the blueprint from the UE plugin for SUMMIT (DynamicMeshActor.cpp:148),
  3. in the UE plugin, create a hook to spawn an actor from the blueprint (CarlaServer.cpp:291, CarlaServer:301), which will be invoked from the C++ LibCarla library,
  4. modify C++ LibCarla library to include functions to call the UE plugin hook in step 3 (Client.h:99, 101). With this you can spawn the blueprint from any C++ program which uses C++ LibCarla;
  5. modify the Python binding of the C++ LibCarla library to expose the newly added function in step 4 (World.cpp:151-175). With this you can spawn the blueprint from any Python program which imports libcarla.

Steps 1, 2, 3 require rebuilding the plugin and subsequently the compiled binaries. Step 4 requires rebuilding the C++ LibCarla library. Step 5 requires rebuilding the Python bindings.

@ronyshaji
Copy link
Author

ronyshaji commented Nov 18, 2022

@LeeYiyuan For the step 2, you mean i have to replace UMaterial* Material = Cast<UMaterial>(StaticLoadObject( UMaterial::StaticClass(), NULL, TEXT("/Game/Carla/Static/GenericMaterials/Ground/M_Tile"))); with my own blueprint path ?

@LeeYiyuan
Copy link
Contributor

That's the general idea, but the exact requirements depends on what kind of blueprint you are using. We are using a UMaterial and it will not work if your blueprint does not match. Please refer UE's official documentation and the community forum on the specific codes needed to get this to work.

@ronyshaji
Copy link
Author

ronyshaji commented Nov 18, 2022

Thanks for the information. In my case the blueprint contains procedural mesh and i think it is different to summit case. Also the DynamicMeshActor.cpp is created entirely in unreal ? I didnot see any carla headers in that, is that the case ?

@LeeYiyuan
Copy link
Contributor

LeeYiyuan commented Nov 18, 2022

The DynamicMeshActor is in fact a procedural mesh, so perhaps it might help you to get started on what you have in mind. Yes, it is created entirely in unreal, and is part of the UE plugin for SUMMIT/CARLA. The UE plugin is loaded when the compiled simulator binaries run. On the other hand, the carla headers are part of LibCarla, which is an entirely separate modue that resides outside of the UE plugin, but communicates with the UE plugin via RPC calls.

@ronyshaji
Copy link
Author

I already have my blueprints (c++ and .h files) ready. So i guess when i follow your steps mentioned above, i can able to access the blueprint in carla as other build in carla contents, right ?

@LeeYiyuan
Copy link
Contributor

LeeYiyuan commented Nov 18, 2022

Yup, in that case you will need steps 3, 4, 5 depending on where you want to access the blueprint from.

@ronyshaji
Copy link
Author

Thanks for the clarification. I want blueprint to be in the content folder. when i check the summit, i can see that the assets are put in different folders like Genericmaterials, testmaps etc. SO after build we can access the custom assets from this folder, right?

Also i can simply create a folder in root/Unreal/CarlaUE4/Plugins/Carla/Source/Carla and put my c++ blueprint files in that as you did in the summit, right ?

@LeeYiyuan
Copy link
Contributor

That's right

@ronyshaji
Copy link
Author

Hallo, Just for clarification. If we build the c++ prgm for the blueprint while building, do we still need the .uasset file for that blueprint ? or the c++ file will act as a .uasset after building ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants