(You may have trouble running the graphical environment in Colab. Run it on your local system and in the Jupyter Notebook for convenience.)
The aim of this project was to get acquainted with the implementation of a complex Fuzzy controller system. We used the MountainCarContinuous-v0 environment here. For more information about this environment, refer to the following link: https://gym.openai.com/envs/MountainCarContinuous-v0
This is a graphical environment in which a one-dimensional car is located between two hills. The goal is to reach the top of the hill on the right, but its engine is not strong enough to do it at once. Therefore, we have to move the car back and forth so that the car reaches the top of the hill on the right.
As mentioned, the goal is to control the car with a fuzzy system to reach the top of the hill on the right. The car has two features in each state. The first feature is the location of the car, which varies from -1.2 to 0.6. The second feature is the car speed, which varies from -0.07 to 0.07. These two state features represent the car at each step. The goal is to place the car more than 0.45. In each step, an Action must be selected according to the state in which the car is located. Action Here is a number between -1 to +1, which is the amount of force that will enter the car. The choice of this action should be based on a fuzzy control system.
Finally, to design this system, we followed bellow steps:
-
First run the gym openai graphical environment.
-
Define and implement the mentioned linguistic variables. We used the scikit-fuzzy library to design this fuzzy control system. The inputs here were the location and speed of the car. We defined fuzzy intervals for these two inputs. Also, the output here was the force on the car, which was the Action, for which we defined and implemented intervals, too.
-
To achieve the goal and according to the linguistic variables the rules should be designed that can take the car to the top of the hill on the right. These rules were implemented using the scikit- fuzzy library.
-
In the implementation loop related to the car control, make a decision at each stage according to the state and using the implemented control system. Finally, the implemented system must be used to select Action to control the car in a graphical environment.
-
Apply this decision on the environment and see the result.
-
If the car does not reach the desired goal and the number of steps is less than a threshold e.g., 500, go to step 4.
-
Show a graph of the rewards received in different stages.