diff --git a/README.md b/README.md index d37b7ba..53cd280 100644 --- a/README.md +++ b/README.md @@ -1,61 +1,40 @@ -# Cyber-Physical Systems Testing Competition # - -The [SBST Workshop](https://sbst22.github.io/) offers a challenge for software testers who want to work with self-driving cars in the context of the usual [tool competition](https://sbst22.github.io/tools/). - -## Important Dates - -The deadline to submit your tool is: **January 21st 2022** - -The results of the evaluation will be communicated to participants on: **February 25th 2022** - -The camera-ready paper describing your tool is due to: **Sunday March 18th 2020** - -## Goal ## -The competitors should generate virtual roads to test a lane keeping assist system using the provided code_pipeline. - -The generated roads are evaluated in the [**BeamNG.tech**](https://www.beamng.tech/) driving simulator. -This simulator is ideal for researchers due to its state-of-the-art soft-body physics simulation, ease of access to sensory data, and a Python API to control the simulation. - -[![Video by BeamNg GmbH](https://github.com/BeamNG/BeamNGpy/raw/master/media/steering.gif)](https://github.com/BeamNG/BeamNGpy/raw/master/media/steering.gif) +# GenRL at the Cyber-Physical Systems Testing Competition # ->Note: BeamNG GmbH, the company developing the simulator, kindly offers it for free for researcher purposes upon registration (see [Installation](documentation/INSTALL.md)). +GenRL is a tool that **Gen**erates effective test cases for a lane-keeping system in a simulated +environment using **R**einforcement **L**earning (RL). -## Comparing the Test Generators ## +This repository is a fork of [tool-competition-av](https://github.com/se2p/tool-competition-av), in which we implemented our RL-based approach. -Deciding which test generator is the best is far from trivial and, currently, remains an open challenge. In this competition, we rank test generators by considering various metrics of effectiveness and efficiency that characterize the generated tests but also the process of generating them, i.e., test generation. We believe that our approach to compare test generators is objective and fair, and it can provide a compact metric to rank them. - -### Ranking Formula +Install additional dependencies for GenRL with +``` +pip3 install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio===0.10.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html -The formula to rank test generators is the following weighted sum: +pip install -r additional-requirements.txt +``` +To start the test generator using the tool competition pipeline, run `competition.py` with the following command line parameters: ``` -rank = a * OOB_Coverage + b * test_generation_efficiency + c * test_generation_effectiveness +--module-name genrl_sbst2022.genrl_test_generator --class-name GenrlTestGenerator ``` -where: - -- `OOB_Coverage` captures the effectiveness of the generated tests that must expose as many failures as possible (i.e., Out Of Bound episodes) but also as many different failures as possible. We compute this metric by extending the approach adopted in the previous edition of the competition with our recent work on [Illumination Search](https://dl.acm.org/doi/10.1145/3460319.3464811). As an example, our novel approach has been already adopted for the generation of relevant test cases from existing maps (see [SALVO](https://ieeexplore.ieee.org/document/9564107)). Therefore, we identify tests' portion relevant to the OOBs, extract their structural and behavioral features, and populate feature maps of a predefined size (i.e., 25x25 cells). Finally, we define `OOB_Coverage` by counting the cells in the map covered by the exposed OOBs. **Larger values of `OOB_Coverage` identify better test generators.** - -- `test_generation_efficiency` captures the efficiency in generating, but not executing, the tests. We measure it as the inverse of the average time it takes for the generators to create the tests normalized using the following (standard) formula: - - ``` norm(x) = (x - min) / (max - min)``` - - Where `min` and `max` are values empirically found during the benchmarking as the minimum and maximum average times for generating test across all the competitors. +# Cyber-Physical Systems Testing Competition # -- `test_generation_effectiveness` captures the ability of the test generator to create valid tests; therefore, we compute it as the ratio of valid tests over all the generated tests. +The [SBST Workshop](https://sbst22.github.io/) offers a challenge for software testers who want to work with self-driving cars in the context of the usual [tool competition](https://sbst22.github.io/tools/). +## Important Dates -### Setting the Weights +The deadline to submit your tool is: **January 14th 2022** -We set the values of the in the ranking formula's weights (i.e., `a`, `b`, and `c`) to rank higher the test generators that trigger many and different failures; test generation efficiency and effectiveness are given equal but secondary importance. The motivation behind this choice is that test generators' main goal is to trigger failures, while being efficient and effective in generating the tests is of second order importance. +The results of the evaluation will be communicated to participants on: **February 25th 2022** -The following table summarizes the proposed weight assignment: +The camera-ready paper describing your tool is due to: **Sunday March 18th 2020** -| a | b | c | -|---|---|---| -|0.6|0.2|0.2| +## Goal ## +The competitors should generate virtual roads to test a lane keeping assist system using the provided code_pipeline. +The generated roads are evaluated in a driving simulator. We partnered with BeamNG GmbH which offers a version of their simulators for researchers, named [BeamNG.tech](https://www.beamng.tech/). This simulator is ideal for researchers due to its state-of-the-art soft-body physics simulation, ease of access to sensory data, and a Python API to control the simulation. +[![Video by BeamNg GmbH](https://github.com/BeamNG/BeamNGpy/raw/master/media/steering.gif)](https://github.com/BeamNG/BeamNGpy/raw/master/media/steering.gif) ## Implement Your Test Generator ## We make available a [code pipeline](code_pipeline) that will integrate your test generator with the simulator by validating, executing and evaluating your test cases. Moreover, we offer some [sample test generators](sample_test_generators/README.md) to show how to use our code pipeline. diff --git a/genrl_sbst2022/README.md b/genrl_sbst2022/README.md new file mode 100644 index 0000000..2d4a9ff --- /dev/null +++ b/genrl_sbst2022/README.md @@ -0,0 +1,18 @@ +# GenRL at the Cyber-Physical Systems Testing Competition # + +GenRL is a tool that **Gen**erates effective test cases for a lane-keeping system in a simulated +environment using **R**einforcement **L**earning (RL). + +This repository is a fork of [tool-competition-av](https://github.com/se2p/tool-competition-av), in which we implemented our RL-based approach. + +Install additional dependencies for GenRL with +``` +pip3 install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio===0.10.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html + +pip install -r additional-requirements.txt +``` + +To start the test generator using the tool competition pipeline, run `competition.py` with the following command line parameters: +``` +--module-name genrl_sbst2022.genrl_test_generator --class-name GenrlTestGenerator +``` diff --git a/genrl_sbst2022/genrl_test_generator.py b/genrl_sbst2022/genrl_test_generator.py new file mode 100644 index 0000000..1af7bf6 --- /dev/null +++ b/genrl_sbst2022/genrl_test_generator.py @@ -0,0 +1,38 @@ +import logging as log + +from stable_baselines3 import PPO + +from genrl_sbst2022.road_generation_env_transform import RoadGenerationTransformationEnv + +class GenrlTestGenerator: + """ + Generates tests using a RL-based approach + """ + + def __init__(self, executor=None, map_size=None): + self.executor = executor + self.map_size = map_size + + def start(self): + log.info("Starting CaRL test generator") + + # Instantiate the environment + # env = RoadGenerationContinuousEnv(test_executor, max_number_of_points=20) + # env = RoadGenerationDiscreteEnv(test_executor, max_number_of_points=8) + env = RoadGenerationTransformationEnv(self.executor, max_number_of_points=4) + + # Instantiate the agent + model = PPO('MlpPolicy', env, verbose=1) + + # Start training the agent + log.info("Starting training") + model.learn(total_timesteps=int(1e2)) + + # If training is done and we still have time left, we generate new tests using the trained policy until the + # given time budget is up. + log.info("Generating tests with the trained agent") + while not self.executor.time_budget.is_over(): + obs = env.reset() + while not done: + action = model.predict(observation=obs) + obs, reward, done, info = env.step(action) diff --git a/genrl_sbst2022/levels_template/tig/art/road/line_white_d.dds b/genrl_sbst2022/levels_template/tig/art/road/line_white_d.dds new file mode 100644 index 0000000..8e44916 Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/road/line_white_d.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/road/line_white_n.dds b/genrl_sbst2022/levels_template/tig/art/road/line_white_n.dds new file mode 100644 index 0000000..1ef3cab Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/road/line_white_n.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/road/line_yellow_d.dds b/genrl_sbst2022/levels_template/tig/art/road/line_yellow_d.dds new file mode 100644 index 0000000..5d20c49 Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/road/line_yellow_d.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/road/materials.cs b/genrl_sbst2022/levels_template/tig/art/road/materials.cs new file mode 100644 index 0000000..67cc9c2 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/art/road/materials.cs @@ -0,0 +1,73 @@ + +singleton Material(tig_road_rubber_sticky) +{ + mapTo = "tig_road_rubber_sticky"; + diffuseMap[0] = "levels/tig/art/road/road_asphalt_2lane_d.dds"; + doubleSided = "0"; + translucentBlendOp = "LerpAlpha"; + normalMap[0] = "levels/tig/art/road/road_asphalt_2lane_n.dds"; + specularPower[0] = "1"; + useAnisotropic[0] = "1"; + materialTag0 = "RoadAndPath"; + materialTag1 = "beamng"; + specularMap[0] = "levels/tig/art/road/road_asphalt_2lane_s.dds"; + reflectivityMap[0] = "levels/tig/art/road/road_rubber_sticky_d.dds"; + cubemap = "global_cubemap_metalblurred"; + translucent = "1"; + translucentZWrite = "1"; + alphaTest = "0"; + alphaRef = "255"; + castShadows = "0"; + specularStrength[0] = "0"; +}; + + + + +singleton Material(tig_line_white) +{ + mapTo = "tig_line_white"; + doubleSided = "0"; + translucentBlendOp = "LerpAlpha"; + normalMap[0] = "levels/tig/art/road/line_white_n.dds"; + specularPower[0] = "1"; + useAnisotropic[0] = "1"; + materialTag0 = "RoadAndPath"; + materialTag1 = "beamng"; + //cubemap = "cubemap_road_sky_reflection"; + //specularMap[0] = "levels/tig/art/road/line_white_s.dds"; + translucent = "1"; + translucentZWrite = "1"; + alphaTest = "0"; + alphaRef = "255"; + castShadows = "0"; + specularStrength[0] = "0"; + colorMap[0] = "levels/tig/art/road/line_white_d.dds"; + annotation = "SOLID_LINE"; + specularStrength0 = "0"; + specularColor0 = "1 1 1 1"; + materialTag2 = "driver_training"; +}; + +singleton Material(tig_line_yellow) +{ + mapTo = "tig_line_yellow"; + doubleSided = "0"; + translucentBlendOp = "LerpAlpha"; + normalMap[0] = "levels/tig/art/road/line_white_n.dds"; + specularPower[0] = "1"; + useAnisotropic[0] = "1"; + materialTag0 = "RoadAndPath"; + materialTag1 = "beamng"; + //cubemap = "cubemap_road_sky_reflection"; + //specularMap[0] = "levels/tig/art/road/line_yellowblack_s.dds"; + translucent = "1"; + translucentZWrite = "1"; + alphaTest = "0"; + alphaRef = "255"; + castShadows = "0"; + specularStrength[0] = "0"; + annotation = "SOLID_LINE"; + colorMap[0] = "levels/tig/art/road/line_yellow_d.dds"; + specularStrength0 = "0"; +}; diff --git a/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_d.dds b/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_d.dds new file mode 100644 index 0000000..3481b92 Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_d.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_n.dds b/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_n.dds new file mode 100644 index 0000000..b625589 Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_n.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_s.dds b/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_s.dds new file mode 100644 index 0000000..7189856 Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/road/road_asphalt_2lane_s.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/road/road_rubber_sticky_d.dds b/genrl_sbst2022/levels_template/tig/art/road/road_rubber_sticky_d.dds new file mode 100644 index 0000000..ba8f885 Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/road/road_rubber_sticky_d.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/terrains/Grass-01-D.dds b/genrl_sbst2022/levels_template/tig/art/terrains/Grass-01-D.dds new file mode 100644 index 0000000..254cade Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/terrains/Grass-01-D.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/terrains/Grass-01-N.dds b/genrl_sbst2022/levels_template/tig/art/terrains/Grass-01-N.dds new file mode 100644 index 0000000..959585f Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/terrains/Grass-01-N.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/terrains/Macro_grass.dds b/genrl_sbst2022/levels_template/tig/art/terrains/Macro_grass.dds new file mode 100644 index 0000000..8626679 Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/terrains/Macro_grass.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/terrains/Overlay_Grass-01.dds b/genrl_sbst2022/levels_template/tig/art/terrains/Overlay_Grass-01.dds new file mode 100644 index 0000000..49662dd Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/art/terrains/Overlay_Grass-01.dds differ diff --git a/genrl_sbst2022/levels_template/tig/art/terrains/materials.cs b/genrl_sbst2022/levels_template/tig/art/terrains/materials.cs new file mode 100644 index 0000000..3c55ea0 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/art/terrains/materials.cs @@ -0,0 +1,17 @@ + +new TerrainMaterial() +{ + diffuseMap = "levels/tig/art/terrains/Overlay_Grass-01"; + detailMap = "levels/tig/art/terrains/Grass-01-D"; + internalName = "groundmodel_asphalt1"; + diffuseSize = "150"; + detailDistance = "50"; + normalMap = "levels/tig/art/terrains/Grass-01-N"; + macroSize = "100"; + macroStrength = "0.4"; + macroDistance = "450"; + macroMap = "levels/tig/art/terrains/Macro_grass"; + detailSize = "3"; + detailStrength = "0.7"; + annotation = "GRASS_THAT_BEHAVES_LIKE_ASPHALT"; +}; diff --git a/genrl_sbst2022/levels_template/tig/info.json b/genrl_sbst2022/levels_template/tig/info.json new file mode 100644 index 0000000..60ba5c1 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/info.json @@ -0,0 +1,7 @@ +{ + "title": "tig", + "description": "Precrime base level", + "previews": ["template_preview.jpg"], + "size": [2048, 2048], + "authors": "Precrime team" +} \ No newline at end of file diff --git a/genrl_sbst2022/levels_template/tig/main.decals.json b/genrl_sbst2022/levels_template/tig/main.decals.json new file mode 100644 index 0000000..690ffe0 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main.decals.json @@ -0,0 +1,10 @@ +{ + "header":{ + "name":"DecalData File", + "comments":"// Instances format: rectIdx, size, renderPriority, position.x, position.y, position.z, normal.x, normal.y, normal.z, tangent.x, tangent.y, tangent.z", + "version":1} + , + "instances":{ + } + } + \ No newline at end of file diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/CameraBookmarks/items.level.json b/genrl_sbst2022/levels_template/tig/main/MissionGroup/CameraBookmarks/items.level.json new file mode 100644 index 0000000..7bac4ab --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main/MissionGroup/CameraBookmarks/items.level.json @@ -0,0 +1 @@ +{"name":"overviewbookmark","internalName":"NewCamera_0","class":"CameraBookmark","persistentId":"87700263-bfbd-48dd-a941-750c49025e6f","__parent":"CameraBookmarks","position":[17.95940017700195,-19.97909927368164,8.283820152282715],"datablock":"CameraBookmarkMarker","isAIControlled":"0","rotationMatrix":[0.6946083307266235,0.7193881273269653,-1.639127731323242e-07,-0.6582382917404175,0.6355648040771484,-0.4034596681594849,-0.2902439832687378,0.2802465558052063,0.9149974584579468]} diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/PlayerDropPoints/items.level.json b/genrl_sbst2022/levels_template/tig/main/MissionGroup/PlayerDropPoints/items.level.json new file mode 100644 index 0000000..e69de29 diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/Water/items.level.json b/genrl_sbst2022/levels_template/tig/main/MissionGroup/Water/items.level.json new file mode 100644 index 0000000..e69de29 diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/generated/items.level.json-readme.txt b/genrl_sbst2022/levels_template/tig/main/MissionGroup/generated/items.level.json-readme.txt new file mode 100644 index 0000000..307260c --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main/MissionGroup/generated/items.level.json-readme.txt @@ -0,0 +1,3 @@ +This is an example of file that describe the customized objects into the 3d level + +{"name": "street_1", "class": "DecalRoad", "breakAngle": 180, "distanceFade": [1000, 1000], "drivability": 1, "material": "tig_road_rubber_sticky", "overObjects": true, "persistentId": "d62cdb76-3384-4620-9d3d-134b21968f32", "__parent": "generated", "position": [0, 0.0, -28, 8], "textureLength": 2.5, "nodes": [[0, 0.0, -28, 8], [5, 4.207354924039483, -28, 8], [10, 4.546487134128409, -28, 8], [15, 0.7056000402993361, -28, 8], [20, -3.7840124765396412, -28, 8], [25, -4.794621373315692, -28, 8], [30, -1.3970774909946293, -28, 8], [35, 3.2849329935939453, -28, 8], [40, 4.946791233116909, -28, 8], [45, 2.060592426208783, -28, 8], [50, -2.7201055544468487, -28, 8], [55, -4.9999510327535175, -28, 8], [60, -2.6828645900021746, -28, 8], [65, 2.1008351841332047, -28, 8], [70, 4.953036778474352, -28, 8], [75, 3.251439200785584, -28, 8], [80, -1.4395165833253265, -28, 8], [85, -4.806987459397784, -28, 8], [90, -3.7549362338583805, -28, 8], [95, 0.7493860483147617, -28, 8], [100, 4.564726253638138, -28, 8], [105, 4.18327819268028, -28, 8], [110, -0.04425654645201938, -28, 8], [115, -4.231102020875853, -28, 8], [120, -4.527891810033119, -28, 8], [125, -0.6617587504888651, -28, 8], [130, 3.8127922523980136, -28, 8], [135, 4.781879642022515, -28, 8], [140, 1.3545289415393453, -28, 8], [145, -3.318169421064838, -28, 8], [150, -4.940158120464309, -28, 8], [155, -2.0201882266153253, -28, 8], [160, 2.757133406208453, -28, 8], [165, 4.999559300536336, -28, 8], [170, 2.645413430600119, -28, 8], [175, -2.140913347480755, -28, 8], [180, -4.9588942672155785, -28, 8], [185, -3.2176906667849976, -28, 8], [190, 1.4818428935469266, -28, 8], [195, 4.818976931420439, -28, 8], [200, 3.725565802396744, -28, 8], [205, -0.7931133440235449, -28, 8], [210, -4.582607739578169, -28, 8], [215, -4.158873713142992, -28, 8], [220, 0.08850962552706788, -28, 8], [225, 4.254517622670592, -28, 8], [230, 4.508941738244046, -28, 8], [235, 0.61786561372612, -28, 8], [240, -3.841273306618334, -28, 8], [245, -4.768763263797359, -28, 8], [250, -1.3118742685196438, -28, 8], [255, 3.3511458792168733, -28, 8], [260, 4.933137960202426, -28, 8], [265, 1.9796257509091708, -28, 8], [270, -2.793945244258081, -28, 8], [275, -4.9987758667931, -28, 8], [280, -2.607755010434559, -28, 8], [285, 2.1808237762391247, -28, 8], [290, 4.964363240422686, -28, 8], [295, 3.1836900356956894, -28, 8], [300, -1.5240531055110833, -28, 8], [305, -4.8305888500419645, -28, 8], [310, -3.695903483246114, -28, 8], [315, 0.8367785015140345, -28, 8], [320, 4.600130190983953, -28, 8], [325, 4.134143397450517, -28, 8], [330, -0.13275577011983397, -28, 8], [335, -4.277599894876611, -28, 8], [340, -4.489638403446456, -28, 8], [345, -0.5739240689159362, -28, 8], [350, 3.8694534077894454, -28, 8], [355, 4.755273266271873, -28, 8], [360, 1.2691168138101814, -28, 8], [365, -3.383859784436538, -28, 8], [370, -4.925731302341237, -28, 8], [375, -1.9389081770471521, -28, 8], [380, 2.8305381844909014, -28, 8], [385, 4.997600792903657, -28, 8], [390, 2.569892279937676, -28, 8], [395, -2.220563343537542, -28, 8], [400, -4.969443269616876, -28, 8], [405, -3.1494399713722694, -28, 8], [410, 1.5661439121654257, -28, 8], [415, 4.841822305500926, -28, 8], [420, 3.665951600366461, -28, 8], [425, -0.8803780997429355, -28, 8], [430, -4.617292235020299, -28, 8], [435, -4.109089183154112, -28, 8], [440, 0.1769915136683034, -28, 8], [445, 4.300347029062267, -28, 8], [450, 4.4699833180027895, -28, 8], [455, 0.5299375587557843, -28, 8], [460, -3.8973303480790236, -28, 8], [465, -4.741410706349736, -28, 8], [470, -1.2262599273382717, -28, 8], [475, 3.4163085736806047, -28, 8], [480, 4.917938727171725, -28, 8], [485, 1.8980386951376085, -28, 8], [490, -2.8669093599521145, -28, 8], [495, -4.996034170931768, -28, 8]]} \ No newline at end of file diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/items.level.json b/genrl_sbst2022/levels_template/tig/main/MissionGroup/items.level.json new file mode 100644 index 0000000..24e4664 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main/MissionGroup/items.level.json @@ -0,0 +1,7 @@ +{"name":"sky_and_sun","class":"SimGroup","persistentId":"43465d7d-0971-4933-89de-c4833b623141","__parent":"MissionGroup"} +{"name":"CameraBookmarks","class":"SimGroup","persistentId":"10c228e4-8d9c-40fb-bdb6-d20febb48221","__parent":"MissionGroup"} +{"name":"vegetation","class":"SimGroup","persistentId":"4f71ae35-2df2-4faf-9297-383b1a62642a","__parent":"MissionGroup"} +{"name":"Water","class":"SimGroup","persistentId":"f0bb0c96-1127-4a30-bce3-cc64d2f856ec","__parent":"MissionGroup"} +{"name":"PlayerDropPoints","class":"SimGroup","persistentId":"ebe56c16-ada2-45a5-8b62-07095074d620","__parent":"MissionGroup","Enabled":"1"} +{"name":"terrain1","class":"TerrainBlock","persistentId":"4e2586a5-7955-49f0-bf35-73613f8cb368","__parent":"MissionGroup","position":[-1023.514038085938,-1114.167846679688,-539.9841918945313],"terrainFile":"levels/tig/terrain.ter"} +{"name":"generated","class":"SimGroup","persistentId":"c545370a-d2b9-4f42-b6fa-578dda84bd81","__parent":"MissionGroup"} diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/items.level.readme.txt b/genrl_sbst2022/levels_template/tig/main/MissionGroup/items.level.readme.txt new file mode 100644 index 0000000..ce0dd15 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main/MissionGroup/items.level.readme.txt @@ -0,0 +1 @@ +{"class":"GroundPlane","persistentId":"8538a723-a048-47be-8a82-0d46c1159338","__parent":"MissionGroup","position":"0 0 0","material":"WarningMaterial","squareSize":54} \ No newline at end of file diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/sky_and_sun/items.level.json b/genrl_sbst2022/levels_template/tig/main/MissionGroup/sky_and_sun/items.level.json new file mode 100644 index 0000000..56d0428 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main/MissionGroup/sky_and_sun/items.level.json @@ -0,0 +1,5 @@ +{"name":"tod","class":"TimeOfDay","persistentId":"49a3be50-5147-474c-bb92-22b37cb07e18","__parent":"sky_and_sun","position":[36.62279891967773,-62.50979995727539,15.32040023803711],"axisTilt":10,"azimuthOverride":0,"play":false,"startTime":0.1000000014901161,"time":0.1000000014901161} +{"name":"theLevelInfo","class":"LevelInfo","persistentId":"9db07344-2f59-4f04-8117-07d04b08e71e","__parent":"sky_and_sun","Enabled":"1","canvasClearColor":[1,1,1,255],"fogAtmosphereHeight":800,"fogColor":[0.7411760091781616,0.8156859874725342,0.9254900217056274,1],"fogDensity":0.0006000000284984708,"globalEnviromentMap":"BNG_Sky_02_cubemap","gravity":-9.810000419616699,"soundAmbience":"AudioAmbienceDefault","visibleDistance":4500} +{"name":"sunsky","class":"ScatterSky","persistentId":"7d6eb1cf-4a73-4780-a87f-9fcacc227ced","__parent":"sky_and_sun","position":[70.39710235595703,4.539209842681885,73.99230194091797],"ambientScale":[0.5450980067253113,0.5450980067253113,0.549019992351532,1],"azimuth":286.6984558105469,"colorize":[0.4235289990901947,0.6078429818153381,0.8666669726371765,1],"colorizeAmount":2,"elevation":52.818603515625,"exposure":15,"fadeStartDistance":1000,"flareScale":5,"flareType":"BNG_Sunflare_2","fogScale":[0.6941180229187012,0.8549020290374756,0.992156982421875,1],"lastSplitTerrainOnly":true,"logWeight":0.9900000095367432,"mieScattering":0.000607529713306576,"moonLightColor":[0.1960780024528503,0.1960780024528503,0.1960780024528503,1],"moonMat":"Moon_Glow_Mat","moonScale":0.02999999932944775,"nightColor":[0.02352939918637276,0.02352939918637276,0.02352939918637276,1],"nightCubemap":"BNG_Sky_Space","nightFogColor":[0.003921569790691137,0.003921569790691137,0.003921569790691137,1],"overDarkFactor":[40000,8000,1800,650],"rayleighScattering":0.009999999776482582,"shadowDarkenColor":[0,0,0,0],"shadowDistance":1600,"shadowSoftness":0.2000000029802322,"skyBrightness":11,"sunScale":[0.9960780143737793,0.9019610285758972,0.8313729763031006,1],"texSize":1024,"useNightCubemap":true} +{"name":"clouds","class":"CloudLayer","persistentId":"ba1f1451-1006-47f8-a30a-658946b01714","__parent":"sky_and_sun","position":[547.2169799804688,-452.9719848632813,609.7449951171875],"Textures":[{"texScale":1.5,"texSpeed":0.002000000094994903},{"texDirection":[0.800000011920929,0.2000000029802322],"texScale":3,"texSpeed":0.02500000037252903},{"texDirection":[0.2000000029802322,0.5],"texScale":4,"texSpeed":0.03500000014901161}],"baseColor":[0.9960780143737793,0.9960780143737793,0.9960780143737793,0.9960780143737793],"coverage":0.07999999821186066,"exposure":1.299999952316284,"height":3,"texture":"levels/jungle_rock_island/art/skies/SkyNormals_05.dds","windSpeed":0.2000000029802322} +{"name":"clouds1","class":"CloudLayer","persistentId":"541e37ca-096c-40e2-aac3-3cd898df7419","__parent":"sky_and_sun","position":[-1278.56005859375,-1211.329956054688,-186.7310028076172],"Textures":[{"texSpeed":0.002000000094994903},{"texDirection":[0.800000011920929,0.2000000029802322],"texScale":2,"texSpeed":0.02500000037252903},{"texDirection":[0.2000000029802322,0.5],"texScale":0.5,"texSpeed":0.03500000014901161}],"baseColor":[0.9960780143737793,0.9960780143737793,0.9960780143737793,0.9960780143737793],"cloneOrigin":"clouds","coverage":0.2000000029802322,"exposure":1.299999952316284,"height":7,"texture":"levels/jungle_rock_island/art/skies/SkyNormals_05.dds","windSpeed":0.2000000029802322} diff --git a/genrl_sbst2022/levels_template/tig/main/MissionGroup/vegetation/items.level.json b/genrl_sbst2022/levels_template/tig/main/MissionGroup/vegetation/items.level.json new file mode 100644 index 0000000..2822ba6 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main/MissionGroup/vegetation/items.level.json @@ -0,0 +1,2 @@ +{"class":"ForestWindEmitter","persistentId":"6737fa35-d648-4f48-b789-1d13783838dc","__parent":"vegetation","position":[24,-89,124]} +{"name":"theForest","class":"Forest","persistentId":"4b096347-fd00-402d-bbda-f3b269ead79d","__parent":"vegetation"} diff --git a/genrl_sbst2022/levels_template/tig/main/items.level.json b/genrl_sbst2022/levels_template/tig/main/items.level.json new file mode 100644 index 0000000..d22211d --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/main/items.level.json @@ -0,0 +1 @@ +{"name":"MissionGroup","class":"SimGroup","persistentId":"9e79f9e5-f71f-4111-8361-cfb22c3f9112","Enabled":"1"} diff --git a/genrl_sbst2022/levels_template/tig/terrain.ter b/genrl_sbst2022/levels_template/tig/terrain.ter new file mode 100644 index 0000000..79f66ff Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/terrain.ter differ diff --git a/genrl_sbst2022/levels_template/tig/terrain.terrain.json b/genrl_sbst2022/levels_template/tig/terrain.terrain.json new file mode 100644 index 0000000..87a8ad5 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/terrain.terrain.json @@ -0,0 +1,11 @@ +{ + "binaryFormat" : "version(char), size(unsigned int), heightMap(heightMapSize * heightMapItemSize), layerMap (layerMapSize * layerMapItemSize), materialNames", + "datafile" : "levels/terrain2.ter", + "heightMapItemSize" : 2, + "heightMapSize" : 4194304, + "heightmapImage" : "levels/tig/terrain.terrainheightmap.png", + "layerMapItemSize" : 1, + "layerMapSize" : 4194304, + "materials" : [ "groundmodel_asphalt1" ], + "size" : 2048 +} diff --git a/genrl_sbst2022/levels_template/tig/terrain.terrainheightmap.png b/genrl_sbst2022/levels_template/tig/terrain.terrainheightmap.png new file mode 100644 index 0000000..73fc7fb Binary files /dev/null and b/genrl_sbst2022/levels_template/tig/terrain.terrainheightmap.png differ diff --git a/genrl_sbst2022/levels_template/tig/tig-version.json b/genrl_sbst2022/levels_template/tig/tig-version.json new file mode 100644 index 0000000..97c9323 --- /dev/null +++ b/genrl_sbst2022/levels_template/tig/tig-version.json @@ -0,0 +1,3 @@ +{ + "version": 0.7 +} \ No newline at end of file diff --git a/genrl_sbst2022/road_generation_env.py b/genrl_sbst2022/road_generation_env.py new file mode 100644 index 0000000..bf59b8a --- /dev/null +++ b/genrl_sbst2022/road_generation_env.py @@ -0,0 +1,195 @@ +import math +import os +import random +import logging +from typing import Optional + +import numpy as np + +import gym +from gym import spaces +from gym.utils import seeding + +from code_pipeline.executors import MockExecutor +from code_pipeline.tests_generation import RoadTestFactory +from code_pipeline.visualization import RoadTestVisualizer + + +class RoadGenerationEnv(gym.Env): + """ + Description: + The agent aims at generating tests for a lane-keeping system in a simulated environment. + Each test is a sequence of points in a 200x200 map. The agent starts with an empty sequence of points. + For any given state, the agent may choose to add/update or delete a point from the sequence. + Source: + The environment was created to compete in the SBST2022 Cyber-physical systems (CPS) testing competition. + Reward: + Negative rewards for actions that generate invalid tests + Positive rewards for actions that make the vehicle go oob + Max reward for actions that make a test fail + Starting State: + The agent starts with an empty sequence of points. + Episode Termination: + Episode length is greater than self.max_steps + """ + + metadata = {"render.modes": ["human", "rgb_array"], "video.frames_per_second": 30} + + def __init__(self, executor, max_steps=1000, grid_size=200, results_folder="results", max_number_of_points=5, + max_reward=100, invalid_test_reward=-10): + + self.step_counter = 0 + + self.max_steps = max_steps + self.grid_size = grid_size + self.max_number_of_points = max_number_of_points + self.executor = executor + self.max_reward = max_reward + self.invalid_test_reward = invalid_test_reward + self.failing_tests = [] # empty list of failing tests, to check for similarity with previously generated tests + + def step(self, action): + pass + + def reset(self, seed: Optional[int] = None): + pass + + def render(self, mode="human"): + screen_width = 600 + screen_height = 400 + + world_width = self.max_position - self.min_position + scale = screen_width / world_width + carwidth = 40 + carheight = 20 + + if self.viewer is None: + from gym.envs.classic_control import rendering + + self.viewer = rendering.Viewer(screen_width, screen_height) + xs = np.linspace(self.min_position, self.max_position, 100) + ys = self._height(xs) + xys = list(zip((xs - self.min_position) * scale, ys * scale)) + + self.track = rendering.make_polyline(xys) + self.track.set_linewidth(4) + self.viewer.add_geom(self.track) + + clearance = 10 + + l, r, t, b = -carwidth / 2, carwidth / 2, carheight, 0 + car = rendering.FilledPolygon([(l, b), (l, t), (r, t), (r, b)]) + car.add_attr(rendering.Transform(translation=(0, clearance))) + self.cartrans = rendering.Transform() + car.add_attr(self.cartrans) + self.viewer.add_geom(car) + frontwheel = rendering.make_circle(carheight / 2.5) + frontwheel.set_color(0.5, 0.5, 0.5) + frontwheel.add_attr( + rendering.Transform(translation=(carwidth / 4, clearance)) + ) + frontwheel.add_attr(self.cartrans) + self.viewer.add_geom(frontwheel) + backwheel = rendering.make_circle(carheight / 2.5) + backwheel.add_attr( + rendering.Transform(translation=(-carwidth / 4, clearance)) + ) + backwheel.add_attr(self.cartrans) + backwheel.set_color(0.5, 0.5, 0.5) + self.viewer.add_geom(backwheel) + flagx = (self.goal_position - self.min_position) * scale + flagy1 = self._height(self.goal_position) * scale + flagy2 = flagy1 + 50 + flagpole = rendering.Line((flagx, flagy1), (flagx, flagy2)) + self.viewer.add_geom(flagpole) + flag = rendering.FilledPolygon( + [(flagx, flagy2), (flagx, flagy2 - 10), (flagx + 25, flagy2 - 5)] + ) + flag.set_color(0.8, 0.8, 0) + self.viewer.add_geom(flag) + + pos = self.state[0] + self.cartrans.set_translation( + (pos - self.min_position) * scale, self._height(pos) * scale + ) + self.cartrans.set_rotation(math.cos(3 * pos)) + + return self.viewer.render(return_rgb_array=mode == "rgb_array") + + def close(self): + if self.viewer: + self.viewer.close() + self.viewer = None + + def get_road_points(self): + """ + Converts the internal representation of road points (in self.state) into a list of points that can be processed + by the test executor. + """ + pass + + def compute_step(self): + done = False + reward = 0 + execution_data = [] + max_oob_percentage = 0 + road_points = self.get_road_points() + logging.debug("Evaluating step. Current number of road points: %d (%s)", len(road_points), str(road_points)) + + if len(road_points) < 3: # cannot generate a good test (at most, a straight road with 2 points) + logging.debug("Test with less than 3 points. Negative reward.") + reward = self.invalid_test_reward + else: # we should be able to generate a road with at least one turn + the_test = RoadTestFactory.create_road_test(road_points) + # check whether the road is a valid one + is_valid, validation_message = self.executor.validate_test(the_test) + if is_valid: + logging.debug("Test seems valid") + # we run the test in the simulator + test_outcome, description, execution_data = self.executor.execute_test(the_test) + logging.debug(f"Simulation results: {test_outcome}, {description}") + if test_outcome == "ERROR": + # Could not simulate the test case. Probably the test is malformed test and evaded preliminary validation. + logging.debug("Test seemed valid, but test outcome was ERROR. Negative reward.") + reward = self.invalid_test_reward # give same reward as invalid test case + elif test_outcome == "PASS": + # Test is valid, and passed. Compute reward based on execution data + max_oob_percentage = self.get_max_oob_percentage(execution_data) + reward = self.compute_reward(max_oob_percentage) + logging.debug(f"Test is valid and passed. Reward was {reward}, with {max_oob_percentage} OOB.") + elif test_outcome == "FAIL": + max_oob_percentage = self.get_max_oob_percentage(execution_data) + reward = self.max_reward + logging.debug(f"Test is valid and failed. Reward was {reward}, with {max_oob_percentage} OOB.") + # save current test + self.failing_tests.append(the_test) + else: + logging.debug(f"Test is invalid: {validation_message}") + return reward, max_oob_percentage + + def compute_reward(self, max_oob_percentage): + reward = pow(max_oob_percentage * 10, 2) / 10 + return reward + + def get_max_oob_percentage(self, execution_data): + """ + execution_data is a list of SimulationDataRecord (which is a named tuple). + We iterate over each record, and get the max oob percentage. + """ + max_oob_percentage = 0 + for record in execution_data: + # logging.info(f"Processing record with oob: {record.oob_percentage}") + if record.oob_percentage > max_oob_percentage: + # logging.debug(f"New oob max: {record.oob_percentage}") + max_oob_percentage = record.oob_percentage + # logging.debug(f"Returning oob max: {max_oob_percentage}") + return max_oob_percentage + + def check_some_coordinates_exist_at_position(self, position): + return self.state[position][0] != 0 or self.state[position][1] != 0 + + def check_coordinates_already_exist(self, x, y): + for i in range(self.max_number_of_points): + if x == self.state[i][0] and y == self.state[i][1]: + return True + return False diff --git a/genrl_sbst2022/road_generation_env_continuous.py b/genrl_sbst2022/road_generation_env_continuous.py new file mode 100644 index 0000000..ebbd392 --- /dev/null +++ b/genrl_sbst2022/road_generation_env_continuous.py @@ -0,0 +1,152 @@ +import math +import os +import random +import logging +from typing import Optional + +import numpy as np + +import gym +from gym import spaces +from gym.utils import seeding + +from code_pipeline.executors import MockExecutor +from code_pipeline.tests_generation import RoadTestFactory +from code_pipeline.visualization import RoadTestVisualizer + +from road_generation_env import RoadGenerationEnv + + +class RoadGenerationContinuousEnv(RoadGenerationEnv): + """ + Observation: + Type: Box(2n+1) where n is self.number_of_points, the max number of points in the generated roads + + Num Observation Min Max + 0 x coord for 1st point self.min_coord self.max_coord + 1 y coord for 1st point self.min_coord self.max_coord + 2 x coord for 2nd point self.min_coord self.max_coord + 3 y coord for 2nd point self.min_coord self.max_coord + ... + 2n-2 x coord for 2nd point self.min_coord self.max_coord + 2n-1 y coord for 2nd point self.min_coord self.max_coord + n Max %OOB 0.0 1.0 # TODO fix + + Actions: + Type: Box(4) ? + Num Action Min Max + 0 Action type 0 1 + 1 Position 0 self.number_of_points + 2 New x coord self.min_coord self.max_coord + 3 New y coord self.min_coord self.max_coord + """ + + ADD_UPDATE = 0 + REMOVE = 1 + + def __init__(self, executor, max_steps=1000, grid_size=200, results_folder="results", max_number_of_points=5, + max_reward=100, invalid_test_reward=-10): + + super().__init__(executor, max_steps, grid_size, results_folder, max_number_of_points, max_reward, + invalid_test_reward) + + self.min_coordinate = 0.0 + self.max_coordinate = 1.0 + + self.max_speed = float('inf') + self.failure_oob_threshold = 0.95 + + self.min_oob_percentage = 0.0 + self.max_oob_percentage = 1.0 + + # state is an empty sequence of points + self.state = np.empty(self.max_number_of_points, dtype=object) + for i in range(self.max_number_of_points): + self.state[i] = (0, 0) # (0,0) represents absence of information in the i-th cell + + self.low_coordinates = np.array([self.min_coordinate, self.min_coordinate], dtype=np.float16) + self.high_coordinates = np.array([self.max_coordinate, self.max_coordinate], dtype=np.float16) + self.low_observation = np.array([], dtype=np.float16) + self.high_observation = np.array([], dtype=np.float16) + + self.viewer = None + + # action space as a box + self.action_space = spaces.Box( + low=np.array([0.0, 0.0, self.min_coordinate + 0.1, self.min_coordinate + 0.1]), + high=np.array([1.0, float(self.max_number_of_points) - np.finfo(float).eps, self.max_coordinate - 0.1, + self.max_coordinate - 0.1]), + dtype=np.float16 + ) + + # create box observation space + for i in range(self.max_number_of_points): + self.low_observation = np.append(self.low_observation, [0.0, 0.0]) + self.high_observation = np.append(self.high_observation, [self.max_coordinate, self.max_coordinate]) + self.low_observation = np.append(self.low_observation, self.min_oob_percentage) + self.high_observation = np.append(self.high_observation, self.max_oob_percentage) + + self.observation_space = spaces.Box(self.low_observation, self.high_observation, dtype=np.float16) + + def step(self, action): + assert self.action_space.contains( + action + ), f"{action!r} ({type(action)}) invalid" + + self.step_counter = self.step_counter + 1 # increment step counter + + action_type = round(action[0]) # value in [0,1] + position = math.floor(action[1]) # value in [0,self.number_of_points) + x = action[2] # coordinate in [self.min_coordinate,self.max_coordinate] + y = action[3] # coordinate in [self.min_coordinate,self.max_coordinate] + + logging.info(f"Processing action {str(action)}") + + if action_type == self.ADD_UPDATE and not self.check_coordinates_already_exist(x, y): + logging.debug("Setting coordinates for point %d to (%.2f, %.2f)", position, x, y) + self.state[position] = (x, y) + reward, max_oob = self.compute_step() + elif action_type == self.ADD_UPDATE and self.check_coordinates_already_exist(x, y): + logging.debug("Skipping add of (%.2f, %.2f) in position %d. Coordinates already exist", x, y, position) + reward = -10 + max_oob = 0.0 + elif action_type == self.REMOVE and self.check_some_coordinates_exist_at_position(position): + logging.debug("Removing coordinates for point %d", position) + self.state[position] = (0, 0) + reward, max_oob = self.compute_step() + elif action_type == self.REMOVE and not self.check_some_coordinates_exist_at_position(position): + # disincentive deleting points where already there is no point + logging.debug(f"Skipping delete at position {position}. No point there.") + reward = self.invalid_test_reward + max_oob = 0.0 + + done = self.step_counter == self.max_steps + + # return observation, reward, done, info + obs = [coordinate for tuple in self.state for coordinate in tuple] + obs.append(max_oob) + return np.array(obs, dtype=np.float16), reward, done, {} + + def reset(self, seed: Optional[int] = None): + # super().reset(seed=seed) + # state is an empty sequence of points + self.state = np.empty(self.max_number_of_points, dtype=object) + for i in range(self.max_number_of_points): + self.state[i] = (0, 0) # (0,0) represents absence of information in the i-th cell + # return observation + obs = [coordinate for tuple in self.state for coordinate in tuple] + obs.append(0.0) # zero oob initially + return np.array(obs, dtype=np.float16) + + + def get_road_points(self): + road_points = [] # np.array([], dtype=object) + for i in range(self.max_number_of_points): + if self.state[i][0] != 0 and self.state[i][1] != 0: + road_points.append( + ( + self.state[i][0] * self.grid_size, + self.state[i][1] * self.grid_size + ) + ) + return road_points diff --git a/genrl_sbst2022/road_generation_env_discrete.py b/genrl_sbst2022/road_generation_env_discrete.py new file mode 100644 index 0000000..ea86b7f --- /dev/null +++ b/genrl_sbst2022/road_generation_env_discrete.py @@ -0,0 +1,152 @@ +import math +import os +import random +import logging +from typing import Optional + +import numpy as np + +import gym +from gym import spaces +from gym.utils import seeding + +from code_pipeline.executors import MockExecutor +from code_pipeline.tests_generation import RoadTestFactory +from code_pipeline.visualization import RoadTestVisualizer + +from road_generation_env import RoadGenerationEnv + + +class RoadGenerationDiscreteEnv(RoadGenerationEnv): + """ + Observation: + Type: MultiDiscrete(2n+1) where n is self.max_number_of_points + + Num Observation Min Max + 0 x coord for 1st point self.min_coord self.max_coord + 1 y coord for 1st point self.min_coord self.max_coord + 2 x coord for 2nd point self.min_coord self.max_coord + 3 y coord for 2nd point self.min_coord self.max_coord + ... + 2n-2 x coord for 2nd point self.min_coord self.max_coord + 2n-1 y coord for 2nd point self.min_coord self.max_coord + n Max %OOB 0.0 1.0 # TODO fix + + Actions: + Type: MultiDiscrete(4) ? + Num Action Num + 0 Action type 2 + 1 Position max_number_of_points + 2 New x coord grid_size * discretization_precision - 2 * safety_buffer + 3 New y coord grid_size * discretization_precision - 2 * safety_buffer + """ + + ADD_UPDATE = 0 + REMOVE = 1 + + def __init__(self, executor, max_steps=1000, grid_size=200, results_folder="results", max_number_of_points=5, + max_reward=100, invalid_test_reward=-10): + + super().__init__(executor, max_steps, grid_size, results_folder, max_number_of_points, max_reward, + invalid_test_reward) + + self.min_coordinate = 0.0 + self.max_coordinate = 1.0 + + self.max_speed = float('inf') + self.failure_oob_threshold = 0.95 + + self.min_oob_percentage = 0 + self.max_oob_percentage = 100 + + # state is an empty sequence of points + self.state = np.empty(self.max_number_of_points, dtype=object) + for i in range(self.max_number_of_points): + self.state[i] = (0, 0) # (0,0) represents absence of information in the i-th cell + + self.viewer = None + + self.discretization_precision = 10 + self.map_buffer_area_width = self.grid_size / 20 # size of the area around the map in which we do not generate + # points + number_of_discrete_coords_in_map = self.grid_size * self.discretization_precision + width_of_buffer_area = self.map_buffer_area_width * self.discretization_precision + number_of_discrete_coords = number_of_discrete_coords_in_map - 2*width_of_buffer_area + + self.action_space = spaces.MultiDiscrete([2, self.max_number_of_points, number_of_discrete_coords, + number_of_discrete_coords]) + + # create box observation space + discretized_oob_size = self.max_oob_percentage * self.discretization_precision + dimensions_list = [number_of_discrete_coords] * (2*max_number_of_points) # two coords for each point + dimensions_list.append(discretized_oob_size) + self.observation_space = spaces.MultiDiscrete(dimensions_list) + + def step(self, action): + assert self.action_space.contains( + action + ), f"{action!r} ({type(action)}) invalid" + + self.step_counter = self.step_counter + 1 # increment step counter + + action_type = action[0] # value in [0,1] + position = action[1] # value in [0,self.number_of_points-1] + x = action[2] # coordinate + y = action[3] # coordinate + + logging.info(f"Processing action {str(action)}") + + reward = 0 + + if action_type == self.ADD_UPDATE and not self.check_coordinates_already_exist(x, y): + logging.debug("Setting coordinates for point %d to (%.2f, %.2f)", position, x, y) + self.state[position] = (x, y) + reward, max_oob = self.compute_step() + elif action_type == self.ADD_UPDATE and self.check_coordinates_already_exist(x, y): + logging.debug("Skipping add of (%.2f, %.2f) in position %d. Coordinates already exist", x, y, position) + reward = self.invalid_test_reward + max_oob = 0.0 + elif action_type == self.REMOVE and self.check_some_coordinates_exist_at_position(position): + logging.debug("Removing coordinates for point %d", position) + self.state[position] = (0, 0) + reward, max_oob = self.compute_step() + elif action_type == self.REMOVE and not self.check_some_coordinates_exist_at_position(position): + # disincentive deleting points where already there is no point + logging.debug(f"Skipping delete at position {position}. No point there.") + reward = self.invalid_test_reward + max_oob = 0.0 + + done = self.step_counter == self.max_steps + + # return observation, reward, done, info + obs = self.get_state_observation() + obs.append(round(max_oob*100*self.discretization_precision)) # max_oob is in [0,1], we make it in 0..1000 + return np.array(obs, dtype=np.float16), reward, done, {} + + def get_state_observation(self): + obs = [coordinate for tuple in self.state for coordinate in tuple] + return obs + + def reset(self, seed: Optional[int] = None): + # super().reset(seed=seed) + # state is an empty sequence of points + self.state = np.empty(self.max_number_of_points, dtype=object) + for i in range(self.max_number_of_points): + self.state[i] = (0, 0) # (0,0) represents absence of information in the i-th cell + # return observation + obs = self.get_state_observation() + obs.append(0.0) # zero oob initially + return np.array(obs, dtype=np.float16) + + def get_road_points(self): + road_points = [] # np.array([], dtype=object) + for i in range(self.max_number_of_points): + if self.state[i][0] != 0 and self.state[i][1] != 0: + road_points.append( + ( + (self.state[i][0] + self.map_buffer_area_width) / self.discretization_precision, + (self.state[i][1] + self.map_buffer_area_width) / self.discretization_precision + ) + ) + logging.debug(f"Current road points: {str(road_points)}") + return road_points diff --git a/genrl_sbst2022/road_generation_env_transform.py b/genrl_sbst2022/road_generation_env_transform.py new file mode 100644 index 0000000..b924056 --- /dev/null +++ b/genrl_sbst2022/road_generation_env_transform.py @@ -0,0 +1,218 @@ +import math +import os +import random +import logging +from typing import Optional +from collections import deque + +import numpy as np + +import gym +from gym import spaces +from gym.utils import seeding + +from code_pipeline.executors import MockExecutor +from code_pipeline.tests_generation import RoadTestFactory +from code_pipeline.visualization import RoadTestVisualizer + +from genrl_sbst2022.road_generation_env import RoadGenerationEnv + + +class RoadGenerationTransformationEnv(RoadGenerationEnv): + """ + Start with a random sequence of points, and with each action we modify it slightly + Observation: + Type: Box(2n+1) where n is self.max_number_of_points + + Num Observation Min Max + 0 x coord for 1st point self.min_coord self.max_coord + 1 y coord for 1st point self.min_coord self.max_coord + 2 x coord for 2nd point self.min_coord self.max_coord + 3 y coord for 2nd point self.min_coord self.max_coord + ... + 2n-2 x coord for 2nd point self.min_coord self.max_coord + 2n-1 y coord for 2nd point self.min_coord self.max_coord + n Max %OOB 0.0 1.0 + + Actions: + Type: MultiDiscrete(4) ? + Num Action Num + 0 Action type 4 # move up, down, left, right + 1 Position max_number_of_points + 2 Amount 3 # small, medium, high amount of movement? + """ + + MOVE_UP = 0 + MOVE_RIGHT = 1 + MOVE_DOWN = 2 + MOVE_LEFT = 3 + + def __init__(self, executor, max_steps=1000, grid_size=200, results_folder="results", max_number_of_points=5, + max_reward=100, invalid_test_reward=-10): + + super().__init__(executor, max_steps, grid_size, results_folder, max_number_of_points, max_reward, + invalid_test_reward) + + self.min_coordinate = 0.0 + self.mid_coordinate = 0.5 + self.max_coordinate = 1.0 + self.safety_buffer = 0.1 + + self.max_speed = float('inf') + self.failure_oob_threshold = 0.95 + + self.min_oob_percentage = 0.0 + self.max_oob_percentage = 100.0 + + self.low_observation = np.array([], dtype=np.float16) + self.high_observation = np.array([], dtype=np.float16) + + # state is an empty sequence of points + self.state = np.empty(self.max_number_of_points, dtype=object) + for i in range(self.max_number_of_points): + self.state[i] = ( + random.uniform(self.min_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer), + random.uniform(self.min_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer) + ) + + self.viewer = None + + # create box observation space + for i in range(self.max_number_of_points): + self.low_observation = np.append(self.low_observation, [0.0, 0.0]) + self.high_observation = np.append(self.high_observation, [self.max_coordinate, self.max_coordinate]) + self.low_observation = np.append(self.low_observation, self.min_oob_percentage) + self.high_observation = np.append(self.high_observation, self.max_oob_percentage) + + self.observation_space = spaces.Box(self.low_observation, self.high_observation, dtype=np.float16) + + # create action space + self.action_space = spaces.MultiDiscrete([4, self.max_number_of_points, 3]) + self.change_amounts = [0.025, 0.05, 0.25] # corresponding to 5, 10 points on the map + self.change_amounts_names = ["low", "medium", "high"] + self.action_names = ["MOVE UP", "MOVE_RIGHT", "MOVE_DOWN", "MOVE_LEFT"] + + def step(self, action): + assert self.action_space.contains( + action + ), f"{action!r} ({type(action)}) invalid" + + self.step_counter = self.step_counter + 1 # increment step counter + + action_type = action[0] # value in [0,1] + position = action[1] # value in [0,self.number_of_points-1] + amount = action[2] # value in [0,2] for small, medium, high amounts of movement + + logging.info(f"Processing action {str(action)}") + + self.state[position], is_valid = self.process_action(action_type, position, amount) + + done = False + + if is_valid: + logging.info("Action was valid, computing step.") + reward, max_oob = self.compute_step() + else: + reward = self.invalid_test_reward + max_oob = 0.0 + done = True # episode ends if an invalid road is produces + + # episode ends after max number of steps per episode is reached or a failing test is produced + if self.step_counter == self.max_steps or reward == self.max_reward: + done = True + + # return observation, reward, done, info + obs = self.get_state_observation() + obs.append(max_oob) # append oob to state observation to get the complete observation + return np.array(obs, dtype=np.float16), reward, done, {} + + def get_state_observation(self): + obs = [coordinate for tuple in self.state for coordinate in tuple] + return obs + + def reset(self, seed: Optional[int] = None): + # super().reset(seed=seed) + self.reset_state() + # return observation + obs = self.get_state_observation() + obs.append(0.0) # zero oob initially + return np.array(obs, dtype=np.float16) + + def get_road_points(self): + road_points = [] + for i in range(self.max_number_of_points): + if self.state[i][0] != 0 and self.state[i][1] != 0: + road_points.append( + ( + self.state[i][0] * self.grid_size, + self.state[i][1] * self.grid_size + ) + ) + return road_points + + def process_action(self, action_type, position, amount): + old_point = self.state[position] + change_amount = self.change_amounts[amount] + x_change_amount = 0 + y_change_amount = 0 + + logging.debug(f"Processing action {self.action_names[action_type]} ({self.change_amounts_names[amount]}) on " + f"position {position}, with current value ({old_point[0]}, {old_point[1]})") + + if action_type == self.MOVE_UP: + y_change_amount = change_amount + elif action_type == self.MOVE_DOWN: + y_change_amount = -1 * change_amount + elif action_type == self.MOVE_RIGHT: + x_change_amount = change_amount + elif action_type == self.MOVE_LEFT: + x_change_amount = -1 * change_amount + + new_point = ( + old_point[0] + x_change_amount, + old_point[1] + y_change_amount + ) + min_admissible_coord = self.min_coordinate + self.safety_buffer + max_admissible_coord = self.max_coordinate - self.safety_buffer + x = new_point[0] + y = new_point[1] + if min_admissible_coord <= x <= max_admissible_coord and min_admissible_coord <= y <= max_admissible_coord: + logging.debug(f"Position {position} changed from ({old_point[0]}, {old_point[1]}) to ({x}, {y})") + return new_point, True + else: + logging.debug(f"Invalid action, tried changing from ({old_point[0]}, {old_point[1]}) to ({x}, {y})") + return old_point, False + + def reset_state(self): + logging.info("Resetting state") + if self.max_number_of_points != 4: + self.state = np.empty(self.max_number_of_points, dtype=object) + for i in range(self.max_number_of_points): + self.state[i] = ( + random.uniform(self.min_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer), + random.uniform(self.min_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer) + ) + else: + # if we have exactly four points, generate one of them in each quadrant (to reduce initially invalid roads) + # TODO: we should generalize this (both to work with any number of points) + point_q1 = ( + random.uniform(self.mid_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer), + random.uniform(self.mid_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer) + ) + point_q2 = ( + random.uniform(self.min_coordinate + self.safety_buffer, self.mid_coordinate - self.safety_buffer), + random.uniform(self.mid_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer) + ) + point_q3 = ( + random.uniform(self.min_coordinate + self.safety_buffer, self.mid_coordinate - self.safety_buffer), + random.uniform(self.min_coordinate + self.safety_buffer, self.mid_coordinate - self.safety_buffer) + ) + point_q4 = ( + random.uniform(self.mid_coordinate + self.safety_buffer, self.max_coordinate - self.safety_buffer), + random.uniform(self.min_coordinate + self.safety_buffer, self.mid_coordinate - self.safety_buffer) + ) + d = deque([point_q1, point_q2, point_q3, point_q4]) + if random.choice([True, False]): + d.reverse() # make the points go "clockwise" + d.rotate(random.randint(0, 3)) # optionally shift the starting point + self.state = np.array(d, dtype=object) # convert deque to np array diff --git a/genrl_sbst2022/run_genrl.py b/genrl_sbst2022/run_genrl.py new file mode 100644 index 0000000..c7aae45 --- /dev/null +++ b/genrl_sbst2022/run_genrl.py @@ -0,0 +1,44 @@ +import logging + +import gym +from stable_baselines3 import PPO, A2C + +from stable_baselines3.common.env_checker import check_env + +from code_pipeline.beamng_executor import BeamngExecutor +from code_pipeline.executors import MockExecutor +from code_pipeline.visualization import RoadTestVisualizer +from road_generation_env import RoadGenerationEnv +from road_generation_env_continuous import RoadGenerationContinuousEnv +from road_generation_env_discrete import RoadGenerationDiscreteEnv +from road_generation_env_transform import RoadGenerationTransformationEnv + +logging.basicConfig(level=logging.DEBUG) + +# test_executor = MockExecutor(result_folder="results", time_budget=1e10, map_size=200, +# road_visualizer=RoadTestVisualizer(map_size=200)) + +test_executor = BeamngExecutor(generation_budget=10000, execution_budget=10000, time_budget=10000, + result_folder="results", map_size=200, beamng_home="D:\\BeamNG", + beamng_user="D:\\BeamNG_user\\", road_visualizer=RoadTestVisualizer(map_size=200)) + +# env = RoadGenerationContinuousEnv(test_executor, max_number_of_points=20) +# env = RoadGenerationDiscreteEnv(test_executor, max_number_of_points=8) +env = RoadGenerationTransformationEnv(test_executor, max_number_of_points=4) + +# Instantiate the agent +model = PPO('MlpPolicy', env, verbose=1, batch_size=2) +model.learn(total_timesteps=int(2), log_interval=1) + +# check_env(env) + + +# Enjoy trained agent +# obs = env.reset() +# for i in range(100): +# action = env.action_space.sample() +# print(str(action)) +# obs, rewards, dones, info = env.step(action) +# print(f"Lunghezza strada: {len(env.get_road_points())}") +# logging.debug(f"Observation is {str(obs)}") +# #env.render()