Introduction

Neurolution is a simulation of cars controlled by neural networks, each of which evolves to learn to drive.

Overview

• The simulation is web-based that runs on a browser.
• Much of the code is written in Typescript.
• Phaser 3 is used as a game engine.
• Box2D used as a physics engine.
• Network network engine powers each car.
• Network networks evolve through generations, instead of other training methods such as stochastic gradient descent.

Compiling and Running

Compiling

Introduction

Although a compiled version of the simulation is provided with this repository, it may be necessary in case you change the code (TypeScript). To compile, you need to have TypeScript installed in your system.

TypeScript compiles to JavaScript, which in turn runs on your browser.

Installing TypeScript

To install TypeScript, simply run:

npm install -g typescript

To ensure that the proper installation of TypeScript exists on your system, run:

tsc --version

If TypeScript is installed properly, you should get the version.

Compiling TypeScript Scripts

Since there is already a configuration file for TypeScript in the repository, tsconfig.json, you can simply run the following command to compile every TypeScript script to JavaScript:

tsc

This should compile the entire simulation.

Alternatively, you could have ran:

npm run build

This would do the same thing as the other command.

Running

Introduction

Before you can run the simulation, you must setup the server. The server simply hosts the simulation on the HTTP protocol, which is necessary because of restrictions on JavaScript in the file URI scheme on a standard browser.

If you directly opened the index.html with your browser, none of the sprites would load.

Setting up the Server

Run the following command to setup the server and its necessary requirements:

npm install

Starting the Server

Run the following shell command to start the server:

npm run start

Visit localhost:3000 in your browser to run the simulation.

Controlling the Simulation

Hotkeys:

Key	Description
Spacebar	Pause/Resume.
H	Toggles activation of manual control of the best fit car.
Arrow Keys	Controls any activated car.

If you ever wish to restart the simulation, simply refresh the page on your browser.

Do not assume manual control of any car in the simulation unless you got good reasons to do so, as it would possibly hamper the evolution.

Why TypeScript?

JavaScript doesn't support static typing. Moreover, standardized JavaScript code is generated by the TypeScript compiler, and most errors are caught at compile-time instead of at run-time.

Why Phaser 3?

Phaser 3 is used as a game engine, which is required for setting up and controlling the scene of the simulation. It basically provides a framework for the simulation which would be much more troublesome with vanilla JavaScript.

Why Box2D?

Box2D is the legendary physics engine, which was found in almost every single physics game on the web, back in the good old days when Adobe Flash used to rule the web.

Here, it's used mainly for collision detection, and proximity calculation; especially using techniques like ray casting.

Directory Structure

Directories are in bold, while files are italicized.

• assets - Contains all the sprites and resources associated with the simulation.

• src

  • libs
    • common - Contains scripts required by other libs.
    • neural_network - Represents the entire neural network associated stuff, including evolution.
    • phaser - Everything related to the game engine's (Phaser 3) preload, setup, creation, update, etc.
    • simulation - Holds code related to the simulation, such as the car, road tracks, etc.

• static

  • js - All the JavaScript scripts, such as those generated after compiling all the TypeScript files, and external libraries such as Box2D.

• index.js - A node.js script which simply serves index.html in a server.

• index.html - Webpage where the entire simulation is hosted.

• tsconfig.json - TypeScript config file, which holds the location[s] of the TypeScript scripts, where to save the resultant JavaScript script, and other attributes.

Components

The simulation is divided into several components, each dealing with unique tasks.

Common

This component contains:

• box2dsetup.ts which sets up variables required for linking to Box2D namespace, and necessary attributes.

• helpers.ts contains helping procedures for cloning objects, doing mathematical operations, etc.

• physics.ts sets up contact listeners for collision between cars and tracks, sets up the world, etc.

Neural Network

This component contains:

• activation_functions.ts holds all the activation procedures.
• engine.ts holds procedures which represents the Neural Network engine only (no trainer).
• evolution.ts contains procedures required for neural networks to evolve.

Phaser

This component contains:

• create.ts holds a create procedure which is triggered by Phaser when the scene is created.
• game.ts is the main game handler, which defines all the variables and sets up all the document event handlers necessary for the simulation.
• preload.ts contains a procedure which is triggered before the scene is created.
• update.ts holds a procedure which is called when the scene is updated (every step/frame).

Simulation

This component contains:

• car.ts contains almost everything related to the cars in the simulation.
• road_tracks.ts contains almost everything related to the road tracks in the simulation.

Evolution

Introduction

In this section, we'll look at how the neural networks evolve in each generation.

Before we proceed any further, let's get familiar with few terms:

• generation is a set of neural networks involved in the evolution.
• breeding refers to reproduction of neural networks in the current generation to produce a new generation.
• crossover is a technique of reproduction of neural networks.
• mutation is a technique of altering the weights and biases of a neural network slightly with a certain degree of randomness.
• fitness is a value which determines how well a neural network performs in the test, based on few criterias.

Here are few facts:

• A new generation is always reproduced from the previous one.
• Only the best fit individuals (neural networks) are used for creating the newer generation.
• In this simulation, the distance travelled, and the average speed is considered for calculating the fitness.
• Crossover involves using the gene pool of the best fit individuals to craft a newer one.

The Cars

The simulation session begins with a generation of cars. The cars are each powered by a neural network, and are equipped with 3 proximity sensors at the front. The cars can only accelerate, and steer upto a certain angle (15 degrees).

The neural networks each got 3 input neurons, and 2 output neurons. The 3 input neurons are fed values from the 3 proximity sensors. And the values from the output neurons directly impact acceleration, and torque applied to the front axle.

In other words, the neural network controls the acceleration, and the steering of the car.

You can change the settings related to the population size of each generation, layer sizes of the neural networks, etc by modifying the definition of the initial model.

Breeding New Generation

One generation is tested in the scene at a time. Whenever a car hits the boundary of the track, it is immediately removed from the scene, and its fitness value is marked. When all the cars in a generation are removed, the generation is over, and a new generation needs to be produced to continue the evolution.

To do so, the individuals (cars) are ranked based on their fitnesses, and the best fit individuals are chosen for reproduction. Crossover is used as a technique of reproduction, where a new neural network is created using the weights and biases randomly selected from the neural networks of best fit individuals. In other words, crossover creates new neural networks using the genes of best fit individuals.

Note that in every new generation, all the neural networks are created using crossover from scratch, none of the neural networks from the previous generation are put in the newer one.

After a new generation is created, each of its neural networks are mutated. This ensures that none of the weights and biases are exactly copied from the parents, similar to what happens in nature, that is - genes are never copied 100% accurately.