Drone Routing Problems

Table of Contents

• About the Project
• Installation
• Environment
• File Structure
• Using Epymarl

About The Project

Installation

git clone https://github.com/DING-1994/drp.git
cd drp
pip install -e .

Environment

how to create environments with the gym framework.

import gym
import drp_env
env = gym.make("drp-2agent_map_3x3-v2", state_repre_flag="onehot_fov")

or

import gym
env = gym.make("drp_env:drp-2agent_map_3x3-v2", state_repre_flag="onehot_fov")

Environment name

drp-{agent_num}agent_{map_name}-v2

• agent_num: number of agents,1~6
• map_name: map name in drp_env/map, map_3x3/map_5x4/map_8x5/map_10x6/map_10x8/map_10x10/map_aoba00/map_aoba01
• state_repre_flag: kinds of observation, coordinate/onehot/onehot_fov/heu_onehot/heu_onehot_fov

Action

node number

When taking invalid actions, stop at current position.

Observation

...

Reward

Rewards are set per agentand, determined by reward_list. reward_list contains 4 keys, goal/collision/wait/move

default:

reward_list:
  goal: 100
  collision: -10
  wait: -10
  move: -1

• goal

  When an agent reach its goal, reward = reward_list["goal”].

• collision

  When agents collide with each other, reward = reward_list["collision”] * speed(default:5).

• wait

  When an agent stops at current position, reward = reward_list["wait”] * speed(default:5).

• wait

  When an agent moves, reward = reward_list["move”] * speed(default:5).

Info

• distance_from_start (List of flaot)

  distance from start node of each agents.

  When agents stop or collide, distance +speed(default:5)

• goal (Bool)

  When all agents reach goal, goal=True

• collision (Bool)

  When agents collide with each other, collision=True

• timeup (Bool)

  For epymarl

File Structure

drp
├── README.md
├── drp_env
│   ├── __init__.py
│   ├── drp_env.py
│   ├── EE_map.py
│   ├── map
│   └── state_repre
├── drpload_test.py
└── for_epymarl

Description

name	description
drp_env	the directory for package drp_env
drpload_test.py	a sample file using drp_env
for_epymarl	files required to work with epymarl

Directories/files in drp_env:

name	description
__init__.py	register environments
drp_env.py	environment with gym structure
EE_map.py	process related to network structure
map	csv files about map information
state_repre	manage observation of environments

Using Epymarl

epymarl is a multi agent reinforcement learning framework.

You can use drp_env with epymarl

1. install epymarl

   git clone https://github.com/uoe-agents/epymarl
   

2. Replace epymarl/src/envs/__init__.py with drp/for_epymarl/envs/__init__.py

3. Replace epymarl/src/config/envs/gymma.yaml with drp/for_epymarl/config/gymma.yaml

4. Example of usin drp_env

   python3 src/main.py --config=iql --env-config=gymma with env_args.time_limit=100 env_args.key="drp_env:drp-1agent_map_3x3-v2" env_args.state_repre_flag="onehot"