1.5.0

Football revamp

We have just revamped VMAS football!

New rewards, observations, options for physical differences and formation spawning and value function plotting.

Try it out for some intense N vs M continuous control gameplay.

We also provide an heuristic opponent with tunable strengths for precision, decision, and speed.

Here are the key new points

• New rewards: sparse reward for scoring, dense reward for moving the ball towards the goal, and dense reward for getting at least one agent close to the ball. These rewards can be tuned and removed in a curriculum setting.
• Opponent heuristic with tunable strengths: It is possible to use an heuristic as the opponent in order to fairly benchmark different algorithms. The heuristic (referret to as AI in the scenario code) has 3 types of tunable strengths, each ranging from 0 to 1: speed strength, decision strength, and precision strength. Speed strength simply dictates the speed of all movement, from dribbling to off-the-ball movement. A value of 0 yields completely stationary agents, and a value of 1 yields a behavior that constantly operates at the maximum allowable velocity. Decision strength affects the agents’ decision-making by adding noise to the value functions that dictate the agent’s decisions. This affects both movement decisions (i.e. the agent might choose to navigate to a position with a lower value, especially if the values have similar magnitude), as well as possession decisions (i.e. the agents will not be able to accurately evaluate whose possession it is, and might consequently assign the ball to the wrong teammate). Lastly, precision strength controls the agent’s ability to execute planned maneuvers.
• Physical differences: you can turn on physical differences to make agents different. Goalkeeper agents will be bigger and slower, while attackers will be smaller and faster
• Formation spawaning: You can spawn your agents at random or in programmatic formations
• Kiccking and non-kicking agents: By default, agents will just use their 2D movement and collisions to move around and move the ball. By turning on shooting actions, agents will have 2 new further actions for rotating and kicking the ball with a continuous force. (All actions are discretizable as in any VMAS scenario)
• Self-play: The setup of observations and rewards is mirrored for the 2 teams. Thus, if instead of using the heuristic opponents you want to use self-play or any other framework that controls both teams you can.

For more technical details on the scenario, see section C.1 of this paper

What's Changed

• [BugFix] Windows paths by @matteobettini in #148
• [Minor] Update links for the CPM road traffic scenario by @Jianye-Xu in #150
• [Scenario] New football by @matteobettini in #97

Full Changelog: 1.4.3...1.5.0