Merge pull request #3 from jawadmhoque/main

Set up initial scenario structure for sf-lite (Issue #1  and #2)

.gitignore

@@ -0,0 +1 @@
+output/

How to Run TSCORE Scenario.md

@@ -0,0 +1,120 @@
+
+# How to Run T-SCORE Scenarios #
+
+When BEAM is executed, the MATSim engine manages loading of most data (population, households, vehicles, and the network used by PhysSim) as well as executing the MobSim -> Scoring -> Replanning iterative loop. BEAM takes over the MobSim, replacing the legacy MobSim engines (i.e. QSim) with the BeamMobSim. 
+
+## Configuration File
+
+Run the configuration file using either command prompt or the InteliJ IDE. 
+
+https://beam.readthedocs.io/en/latest/users.html#running-beam
+
+Copy over specification of the sf-tscore-25k.conf file to the new configuration file. Build project using InteliJ and then edit the  configuration to include the new configuration file.
+
+The outputs will be written to the scenario/output/sf-tscore or scenario/output/slc-tscore  folder. The required files will then be copied and/or analyzed to the sf-tscore/output folder.
+
+## Input Files ##
+The basic input files necessary to run a simulation are:
+
+- A configuration file (e.g. sf-tscore-25k.conf)
+- The person population/plans and corresponding attributes files (e.g. population.xml and populationAttributes.xml)
+- The household population and corresponding attributes files (e.g. households.xml and householdAttributes.xml)
+- The personal vehicle fleet (e.g. vehicles.csv)
+- The definition of vehicle types including for personal vehicles and the public transit fleet (e.g. vehicleTypes.csv)
+- A directory containing network and transit data used by R5 (e.g. r5/)
+- The open street map network (e.g. r5/sf-muni.osm)
+- GTFS archives, one for each transit agency (e.g. r5/bus.zip)
+- Ridehail fleet information
+
+###1. Population or Plans File###
+MATSim travel demand is described by the agents’ day plans. The full set of agents is also called the population, hence the file name population.xml. Alternatively, plans.xml is also commonly used in MATSim, as the population file essentially contains a list of day plans.
+
+**File name:** population.xml.gz
+
+**Fields:** Person ID, Age, Sex, Activity, Start and End coordinates, Activity end times, Activity leg
+
+**Source for T-Score:** Output from MTC or SFCTA travel demand model.  Current thinking is to use the SF-CHAMP travel demand model outputs because they are more spatially detailed than the current MTC ActivitySim version.  
+### 2. Population Attributes File
+**File Name:** populationAttributes.xml.gz
+
+**Fields:** Person ID, Household Rank, Excluded Modes, Value of Time
+
+**Source for T-Score:** Output from MTC or SFCTA travel demand model.
+
+### 3. Households  File
+**File Name:** households.xml.gz
+
+**Fields:** Household ID, Person ID, Vehicles, Yearly Income.
+
+**Source for T-Score:** Output from MTC or SFCTA travel demand model.
+### 4. Households Attributes File
+**File Name:** householdAttributes.xml.gz
+
+**Fields:** Household ID, Home Coordinates, Housing Type
+
+**Source for T-Score:** Output from MTC or SFCTA travel demand model.
+### 5. Vehicles File
+Replacement to legacy MATSim vehicles.xml file. This must contain an Id and vehicle type for every vehicle id contained in households.xml.
+**File Name:** vehicles.csv
+
+**Fields:** Vehicle ID, Vehicle Type
+
+**Source for T-Score:** Output from MTC or SFCTA travel demand model.Generated based on modeled vehicle ownership.
+
+To generate automatically from population/household data, make changes to the following parameters in the common/matsim.conf file:
+
+matsim.conversion.generateVehicles = true (If true, the conversion will use the population data to generate default vehicles, one per agent)
+
+matsim.conversion.vehiclesFile = “Siouxfalls_vehicles.xml” (optional, if generateVehicles is false, specify the matsim vehicles file name)
+
+### 6. Vehicle Types  File
+**File Name:** vehicleTypes.csv
+
+**Fields:** Vehicle Type ID, Seating Capacity. Standing Room Capacity, Length, Fuel Type and consumption in Joules per Meter, Maximum Velocity, Passenger Car Unit, Vehicle Category, Probability within category,
+
+**Note:** Contains human body as a vehicle type, possibly to enable walk mode
+
+**Source for T-Score:** : Energy modeling is not a priority, so assume a single vehicle type or defaults. 
+
+## Network File
+Contains node coordinates, link IDs, freeflow speed, capacity, number of lanes, modes allowed.
+
+**File Name:** physsim-network.xml 
+
+### From MATsim Network
+
+The following MATSim input data are required to convert the MATsim network files to BEAM compatible physsim network. The config file is located in test/input/common/matsim.conf .
+These files are also used in the BEAM Agent simulation.  
+
+
+- **MATSim network file**: (e.g. network.xml)
+
+- A download of **OpenStreetMap data** for a region that includes your region of interest. Should be **in pbf format**. For North American downloads: http://download.geofabrik.de/north-america.html
+
+- **MATSim plans** (or population) file: (e.g. population.xml)
+
+The following inputs are optional and only recommended if your MATSim scenario has a constrained vehicle stock (i.e. not every person owns a vehicle):
+
+- **MATSim vehicle** definition (e.g. vehicles.xml)
+- **Travel Analysis Zone shapefile** for the region, (e.g. as can be downloaded from https://www.census.gov/geo/maps-data/data/cbf/cbf_taz.html)
+
+Finally, this conversion can only be done with a clone of the full BEAM repository.
+### From Open Street Map Network
+The Open Street Map of the required network is converted to physsim network. This file is in the r5 folder.
+
+- **OSM file in pbf format**, the Open Street Map source data file that should be clipped to the scenario network
+- **OSM Mapdb file**
+- Frequency Adjustment File
+- Link to Grade Percent File
+- TAZ Centroids and Parking files
+- **GTFS Files**
+Source: OpenStreetMap
+
+### Frequency Adjustment File
+Related to transit schedules
+
+**File name:** FrequencyAdjustment.csv
+
+**Fields:** trip_id, start and end time, headway is seconds and exact time
+
+**Source:** GTFS?  No adjustments? 

README.md

@@ -13,6 +13,14 @@ The MMOS scenarios are developed for two metropolitan regions:
 
 ## Project Navigation
 
+There are four key folders in this repository:
+
+1. **convert:** Contains the files created externally, e.g. MATSim network file, ride-hail services etc, that are to be converted to BEAM supported format.
+2. **output:** Contains the total output generated from a BEAM run. Follows the convention specified in BEAM user guide. Will be locally available only, i.e. ignored while committing to github.
+3. **sf-tscore:** Contains the relevant input and output files for MMOS scenario in San Francisco County, California.
+4. **slc-tscore:** Contains the relevant input and output files for MMOS scenario in Wasatch Front region (Salt Lake City and environs), Utah.
+
+The *How to Run TSCORE Scenario* markdown file describes the input files necessary for scenario analyses and how to run these scenario.
 
 ## Handoff Strategy
 This section describes the specifics of how the two different halves of the MMOS 

common/akka-router.conf

@@ -0,0 +1,28 @@
+router-dispatcher {
+  # Dispatcher is the name of the event-based dispatcher
+  type = Dispatcher
+  # What kind of ExecutionService to use
+  executor = "fork-join-executor"
+  # Configuration for the fork join pool
+  fork-join-executor {
+    # Min number of threads to cap factor-based parallelism number to
+    parallelism-min = 2
+    # Parallelism (threads) ... ceil(available processors * factor)
+    parallelism-factor = 1.0
+    # Max number of threads to cap factor-based parallelism number to
+    parallelism-max = 16
+  }
+  # Throughput defines the maximum number of messages to be
+  # processed per actor before the thread jumps to the next actor.
+  # Set to 1 for as fair as possible.
+  throughput = 100
+}
+
+akka {
+  log-dead-letters = 0
+  deployment {
+    /router/router-worker {
+      dispatcher = router-dispatcher
+    }
+  }
+}

common/akka.conf

@@ -0,0 +1,224 @@
+##################################################################
+# Akka
+##################################################################
+my-custom-mailbox {
+  mailbox-type = "akka.dispatch.UnboundedDequeBasedMailbox"
+}
+
+beam-agent-scheduler-pinned-dispatcher {
+  executor = "thread-pool-executor"
+  type = PinnedDispatcher
+  thread-pool-executor.allow-core-timeout=off
+}
+
+zonal-parking-manager-pinned-dispatcher {
+  executor = "thread-pool-executor"
+  type = PinnedDispatcher
+  thread-pool-executor.allow-core-timeout=off
+}
+
+parallel-parking-manager-dispatcher {
+  executor = "thread-pool-executor"
+  thread-pool-executor {
+    keep-alive-time = 120s
+    core-pool-size-max = 64
+  }
+  throughput = 10
+  type = Dispatcher
+}
+
+ride-hail-manager-pinned-dispatcher {
+  executor = "thread-pool-executor"
+  type = PinnedDispatcher
+  thread-pool-executor.allow-core-timeout=off
+}
+
+akka {
+  extensions = ["com.romix.akka.serialization.kryo.KryoSerializationExtension$"]
+
+  loggers = ["akka.event.slf4j.Slf4jLogger"]
+  # , "akka.testkit.TestEventListener"
+  logging-filter = "akka.event.slf4j.Slf4jLoggingFilter"
+  loglevel = "debug"
+  stdout-loglevel = "OFF"
+
+  actor {
+    serializers {
+      kryo = "com.romix.akka.serialization.kryo.KryoSerializer"
+    }
+    serialization-bindings {
+      "java.io.Serializable" = kryo
+    }
+    debug {
+      lifecycle = on
+      # enable DEBUG logging of unhandled messages
+      unhandled = on
+      fsm = off
+    }
+    kryo {
+      # Possibles values for type are: graph or nograph
+      # graph supports serialization of object graphs with shared nodes
+      # and cyclic references, but this comes at the expense of a small
+      # overhead nograph does not support object grpahs with shared nodes,
+      # but is usually faster
+      type = "graph"
+      # Possible values for idstrategy are:
+      # default, explicit, incremental, automatic
+      #
+      # default - slowest and produces bigger serialized representation.
+      # Contains fully-qualified class names (FQCNs) for each class. Note
+      # that selecting this strategy does not work in version 0.3.2, but
+      # is available from 0.3.3 onward.
+      #
+      # explicit - fast and produces compact serialized representation.
+      # Requires that all classes that will be serialized are pre-registered
+      # using the "mappings" and "classes" sections. To guarantee that both
+      # sender and receiver use the same numeric ids for the same classes it
+      # is advised to provide exactly the same entries in the "mappings"
+      # section.
+      #
+      # incremental - fast and produces compact serialized representation.
+      # Support optional pre-registering of classes using the "mappings"
+      # and "classes" sections. If class is not pre-registered, it will be
+      # registered dynamically by picking a next available id To guarantee
+      # that both sender and receiver use the same numeric ids for the same
+      # classes it is advised to pre-register them using at least the "classes" section.
+      #
+      # automatic -  use the pre-registered classes with fallback to FQCNs
+      # Contains fully-qualified class names (FQCNs) for each non pre-registered
+      # class in the "mappings" and "classes" sections. This strategy was
+      # added in version 0.4.1 and will not work with the previous versions
+      idstrategy = "default"
+      # Define a default queue builder, by default ConcurrentLinkedQueue is used.
+      # Create your own queue builder by implementing the trait QueueBuilder,
+      # useful for paranoid GC users that want to use JCtools MpmcArrayQueue for example.
+      #
+      # If you pass a bounded queue make sure its capacity is equal or greater than the
+      # maximum concurrent remote dispatcher threads your application will ever have
+      # running; failing to do this will have a negative performance impact:
+      #
+      # custom-queue-builder = "a.b.c.KryoQueueBuilder"
+      # Define a default size for byte buffers used during serialization
+      buffer-size = 65536
+      # The serialization byte buffers are doubled as needed until they
+      # exceed max-buffer-size and an exception is thrown. Can be -1
+      # for no maximum.
+      max-buffer-size = -1
+      # If set, akka uses manifests to put a class name
+      # of the top-level object into each message
+      use-manifests = false
+      # If set it will use the UnsafeInput and UnsafeOutput
+      # Kyro IO instances. Please note that there is no guarantee
+      # for backward/forward compatibility of unsafe serialization.
+      # It is also not compatible with the safe-serialized values.
+      # The unsafe IO usually creates bugger payloads but is faster
+      # for some types, e.g. native arrays.
+      use-unsafe = false
+      # The transformations that have be done while serialization
+      # Supported transformations: compression and encryption
+      # accepted values(comma separated if multiple): off | lz4 | deflate | aes
+      # Transformations occur in the order they are specified
+      post-serialization-transformations = "off"
+      # Settings for aes encryption, if included in transformations AES
+      # algo mode, key and custom key class can be specified AES algo mode
+      # defaults to 'AES/CBC/PKCS5Padding' and key to 'ThisIsASecretKey'.
+      # If custom key class is provided, Kryo will use the class specified
+      # by a fully qualified class name to get custom AES key. Such a
+      # class should define the method 'kryoAESKey'. This key overrides 'key'.
+      # If class doesn't contain 'kryoAESKey' method, specified key is used.
+      # If this is not present, default key is used
+      # Log implicitly registered classes. Useful, if you want to know all
+      # classes which are serialized. You can then use this information in
+      # the mappings and/or classes sections
+      implicit-registration-logging = false
+      # If enabled, Kryo logs a lot of information about serialization process.
+      # Useful for debugging and lowl-level tweaking
+      kryo-trace = false
+      # If proviced, Kryo uses the class specified by a fully qualified
+      # class name to perform a custom initialization of Kryo instances in
+      # addition to what is done automatically based on the config file.
+      # If enabled, allows Kryo to resolve subclasses of registered Types.
+      #
+      # This is primarily useful when idstrategy is set to "explicit". In this
+      # case, all classes to be serialized must be explicitly registered. The
+      # problem is that a large number of common Scala and Akka types (such as
+      # Map and ActorRef) are actually traits that mask a large number of
+      # specialized classes that deal with various situations and optimizations.
+      # It isn't straightforward to register all of these, so you can instead
+      # register a single supertype, with a serializer that can handle *all* of
+      # the subclasses, and the subclasses get serialized with that.
+      #
+      # Use this with care: you should only rely on this when you are confident
+      # that the superclass serializer covers all of the special cases properly.
+      resolve-subclasses = false
+      # Define mappings from a fully qualified class name to a numeric id.
+      # Smaller ids lead to smaller sizes of serialized representations.
+      #
+      # This section is:
+      # - mandatory for idstrategy="explicit"
+      # - ignored   for idstrategy="default"
+      # - optional  for incremental and automatic
+      #
+      # The smallest possible id should start at 20 (or even higher), because
+      # ids below it are used by Kryo internally e.g. for built-in Java and
+      # Scala types
+      mappings {
+        "package1.name1.className1" = 20,
+        "package2.name2.className2" = 21
+      }
+      # Define a set of fully qualified class names for
+      # classes to be used for serialization.
+      # The ids for those classes will be assigned automatically,
+      # but respecting the order of declaration in this section
+      #
+      # This section is ignored for idstrategy="default" and optional for
+      # all other.
+      classes = [
+        "package3.name3.className3",
+        "package4.name4.className4"
+      ]
+    }
+  }
+  remote {
+    log-remote-lifecycle-events = off
+    log-received-messages = on
+    log-sent-messages = on
+    artery {
+      transport = tcp
+      enabled = on
+
+      # Next configurations will be set at runtime by BeamHelper
+      # canonical.hostname = ${node.host}
+      # canonical.port = ${node.port}
+
+      advanced {
+        maximum-frame-size = 512MiB
+        buffer-pool-size = 128
+        maximum-large-frame-size = 1024MiB
+        large-buffer-pool-size = 32
+        client-liveness-timeout = 400 seconds
+        driver-timeout = 400 seconds
+      }
+    }
+  }
+  cluster {
+    debug.verbose-heartbeat-logging = on
+    debug.verbose-gossip-logging = on
+    # This configuration will be set at runtime by BeamHelper
+    # seed-nodes = [ "akka://ClusterSystem@"${seed.address}]
+
+    # how often should the node send out gossip information?
+    gossip-interval = 1s
+    # discard incoming gossip messages if not handled within this duration
+    gossip-time-to-live = 20s
+    failure-detector {
+      heartbeat-interval = 1s
+      acceptable-heartbeat-pause = 140s
+    }
+  }
+}
+
+##################################################################
+# Non-common Akka
+##################################################################
+akka.log-dead-letters = 1