README.Rmd

---
# title: "Integrating geographic analysis in transport planning: new open source tools for the trade"
# title: "Integrating geographic analysis in transport planning: new tools of the trade"
# title: "Open source software for geographic analysis in transport planning"
# title: "Open source software for geographic analysis in transport planning: a review of emerging ecosystems"
# title: "Integrating geographic analysis in transport planning"
title: "Integrating geographic analysis in transport planning: origins, software and open source solutions"
# title: "Integrating geographic analysis in transport planning workflows: the role of open source software
# author: "Robin Lovelace"
date: '`r Sys.Date()`'
output:
#   bookdown::pdf_document2:
#     latex_engine: xelatex
  # word_document: default
  github_document: default
bibliography:
  - references.bib
  - ortuzar.bib
---

```{r, include=FALSE}
library(tidyverse)
```


```{r, echo=FALSE, include=FALSE, eval=FALSE}
# Priorty: write new tools of the trade section
# get citationscooper_sdna:_2015	
# refs_est = RefManageR::ReadZotero(group = "418217", .params = list(collection = "ABALWWZN", limit = 100))
# RefManageR::WriteBib(refs_est, "trafficEstimatr.bib")
refs_sof = RefManageR::ReadZotero(group = "418217", .params = list(collection = "JFR868KJ", limit = 100))
refs_gis = RefManageR::ReadZotero(group = "418217", .params = list(collection = "KUXG5CJL", limit = 100))
refs_geo = RefManageR::ReadZotero(group = "418217", .params = list(collection = "9K6FRP6N", limit = 100))
refs_mod = RefManageR::ReadZotero(group = "418217", .params = list(collection = "BN2XXKPI", limit = 100))
refs_tds = RefManageR::ReadZotero(group = "418217", .params = list(collection = "R38L2EXB", limit = 100))
o = ls()
r = o[grepl(pattern = "refs_", x = o)]
# refs = c(refs_sof, refs_gis, refs_geo, refs_tds)
refs = do.call(what = c, args = mget(r))
ref_df = as.data.frame(refs)
# View(ref_df)
RefManageR::WriteBib(refs, "references.bib")
# citr::tidy_bib_file(rmd_file = "README.Rmd", messy_bibliography = "references.bib") # fails

# n_words = wordcountaddin::text_stats("README.Rmd")
# word_count = readr::read_csv("word-count.csv")
# word_count_new = rbind(word_count, data.frame(date = Sys.Date(), n_words))
# readr::write_csv(word_count_new, "word-count.csv")
# ggplot(word_count_new) +
#   geom_line(aes(date, n_words)) +
#   xlim(as.Date(c("2019-01-01", "2020-01-01"))) +
#   ylim(0, 5000)
# f = "~/hd/books/David E. Boyce, Huw C.W.L. Williams/Forecasting Urban Travel (599)/Forecasting Urban Travel - David E. Boyce, Huw C.W.L. Williams.pdf"
# img_file <- pdftools::pdf_convert(f, format ='tiff', pages = 1:9, dpi = 400)
# txt = tesseract::ocr(img_file)
```

```{r, include=FALSE}
knitr::opts_chunk$set(echo = FALSE)
```

<!-- should be integrated in transport planning tools. -->
<!-- --- software for transport data analysis, modelling and visualisation --- -->
<!-- workflows in academic, public sector and private consultancy transport planning contexts still tend to separate vital geographic processing and map making stages from the rest of the analysis. -->

# Abstract {-}

<!-- Tools for geographic analysis have long been used in transport planning. -->
<!-- , alongside other (typically primarily economic and engineering) considerations. -->

Transport is inherently geographic.
Movement of people, goods and electrons, between points and polygons along linear ways in spatial networks, is a defining feature of civilisation.
Decarbonising transport systems is vital for the future.
Sustainable transport interventions are most effective when they are located intelligently.  
Geographic analysis therefore has a huge amount to offer transport planning and, more specifically, to transport planning tools.
Yet geographic analysis is often treated as an optional 'add-on', conducted in different programs than those typically used in practice for transport data analysis and modelling.
This dichotomy, between 'geographic' and 'non-geographic' analysis in transport planning is undesirable because it:
(1) reduces researcher effectiveness due to the time-consuming process of 'context switching';
(2) prevents reproducibility, requiring installation and management of geographic and non-geographic tools; and 
(3) hides vital geographic components in transport plans, with adverse consequences for interventions that can benefit from being placed where they are most needed.
These premises are described in the wider context of transport planning research and the landscape of transport planning software, which is dominated by proprietary products.
The paper then shifts gear, to focus on open source solutions.
Three emerging software ecosystems for integrating geographic analysis in transport planning (data analysis and modelling) workflows are reviewed in depth: the statistical programming language R, the general purpose programming language Python, and the geographic information system QGIS.
Each ecosystem is found to provide numerous 'tools of the trade' for integrating geographic analysis in transport research, some of which are already being used in practice.
In addition to integrating geographic considerations, the shift to such open source solutions will have wider benefits for reproducibility, transparency and democratic accountability in transport planning.
<!--  transport planning experts and the public alike, -->
<!-- Ultimately, by highlighting cost effective and geographically targeted interventions, integrating geographic analysis in transport planning could lead to better decision making. -->
<!-- and support the global efforts to transition away from fossil fuels and towards a healthy, low carbon transport system. -->

<!-- ### Notes/questions -->

<!-- This paper is work in progress and its focus has shifted over time. -->
<!-- A previous working title was "Integrating geographic analysis in transport planning: new open source tools for the trade". -->
<!-- The need to integrate geographic analysis in transport planning is still a central theme of the paper but the focus is now on solutions, in the form of open source software. -->
<!-- Does the paper now have too much on the importance on integrating geographic analysis in transport planning, given the new emphasis? -->
<!-- And are there any other open source software projects I should include, within or in addition to the overview of the three ecosystems selected? -->


# Introduction

<!-- : the importance of geographic analysis in transport planning -->
<!-- Should that heading omit the "Introduction:" part? -->

Transport planning is an applied discipline involving the design and placement of physical infrastructure, especially ways, including highways, railways, cycleways and footways [@oflaherty_transport_1997; @parkin_designing_2018].
It also involves visioning transport futures and making the economic and political case for change [@timms_imagineering_2014].
Successful transport plans are therefore a combination of geographically specific recommendations (e.g. "build this way here") and long-term strategies guided by citywide, regional and even national visions (e.g. "imagine the benefits of making the city free from private cars by 2030").
The rewards can be great: transport planners who have designed --- and helped to implement --- plans appropriate to the needs of an area leave a legacy that will benefit people and the environment for generations to come.^[
Articles about successful transport planners illustrate the point.
Ben Hamilton-Baillie (1955 - 2019), for example, was an influential transport planner and street designer whose obituary stated that "hundreds of thousands of people who are safer and happier as a result of his achievements" (Tim Stornor, quoted in [TransportExtra](https://www.transportxtra.com/publications/local-transport-today/news/60655/obituary-ben-hamilton-baillie/)).
]

Transport planning can be considered as "more of an art than a technique", although *good* transport plans also rely on robust analysis and modelling of sometimes large and usually spatial input datasets  [@ortuzars_modelling_2001].
Ways and other pieces of transport infrastructure must *go somewhere* to optimise their ability to take people *where they want to go*, yet the importance of geography in transport systems is often overlooked [@rodrigue_geography_2013].
Transport planning is embedded within broader democratic processes *and* local geographic contexts.

Effective transport planning is inherently embedded within democratic processes (to the extent that democracy exists in the area where transport plans are commissioned) because it determines how public spaces and other shared resources are used, affecting everyone in society.
If transport plans are developed by unaccountable technocrats with little understanding of their impacts, the results can be severe: the transport system now is the "leading cause of death for children and young adults aged 5-29 years" through road traffic casualties alone, with 1.35 million people killed and tens of millions injured and disabled each year [@world_health_organization_global_2018].
The air pollution impacts could be even greater, with a growing body of research linking air pollution to Alzheimer's disease, lung cancer and heart disease among hundreds of millions of sufferers worldwide [@kampa_human_2008; @kilian_emerging_2018].
Transport is responsible for a quarter of global greenhouse gas emissions and growing [@harrison_environmental_2017], and is one of the hardest sectors to decarbonise [@moriarty_prospects_2008], meaning that evidence-based plans to reduce transport energy use is an urgent priority affecting the global population, especially the projected 1.4 billion people who will live in low elevation (less than 10 m above sea level) coastal zones by 2060 [@neumann_future_2015].

Transport planning is inherently embedded within local geographic contexts because transport systems, and the networks of physical infrastructure that underpins them, are highly localised [@barthelemy_spatial_2011; @levinson_network_2012] and to some degree dynamic [@xie_evolving_2011] phenomena.
Transport planning is therefore fundamentally a *spatial activity*.
Perhaps in response to an upsurge in the amount of geographic transport data available, interest in Transport Geography seems to have grown over the last decade relative to other terms, according to data obtained using the **gtrendsR** package [@massicotte_gtrendsr:_2019] from Google Trends (Figure 1).
It is interesting to compare this growth with relative levels of interest in 'geographic information systems' and 'spatial planning', both of which seem to have seen relative declines over time, notwithstanding the limitations associated with search data [@mellon_internet_2014]. 

```{r, eval=FALSE}
library(gtrendsR)
# res_all <- gtrends(c("transport geography", "geographic information systems", "geocomputation"), time = "all")
res <- gtrends(c("transport geography", "geographic information systems", "spatial planning"), time = "2010-01-01 2019-08-01")
head(res$interest_over_time)
tail(res$interest_over_time)
head(res$interest_over_time$time)
class(res$interest_over_time$time)
unique(res$interest_over_time$keyword)
nrow(res$interest_over_time) / 3 / 12
nrow(res$interest_over_time) / 3
res$interest_over_time$date = rep(seq(from = as.Date("2010-01-01"), to = as.Date("2019-07-01"), length.out = 116), 3)
start_rows = 1
end_rows = nrow(res$interest_over_time) / 3

res$interest_over_time$hits_smoothed = c(
  zoo::rollmean(res$interest_over_time$hits[start_rows:end_rows], 6, fill = "extend"),
  zoo::rollmean(res$interest_over_time$hits[(start_rows:end_rows) + end_rows], 6, fill = "extend"),
  zoo::rollmean(res$interest_over_time$hits[(start_rows:end_rows) + 2 * end_rows], 6, fill = "extend")
)
ggplot(res$interest_over_time) +
  geom_line(aes(date, hits_smoothed, colour = keyword), size = 2) +
  ylab("Relative number of hits") +
  theme(legend.position = c(0.7, 0.8))
  
ggplot2::ggsave("google-trends.png", width = 6, height = 4)
```

```{r, fig.cap="Relative level of interest in search terms related to 'geographic information systems', 'geocomputation' and 'transport geography' inferred from google search data, obtained with the gtrendsR R package. Code to reproduce the plot is hosted in this paper's code repository.", out.width="70%"}
knitr::include_graphics("google-trends.png")
```

The growing research interest in the subject is also reflected in teaching.
Modules dedicated to Transport Geography have been advertised at the Universities of Aberdeen and Hofstra and, at the University of Leeds the Masters module Sustainable Spatial Planning and Analysis ([SSPA](https://github.com/ITSLeeds/SSPA)) is focussed on GIS skills for transport planning (declaration of interest: I teach on this module), and there are even dedicated 3 year degrees Transport Geography.

The paper outlines the relationship between geographic analysis and transport planning, in relation to the history of geographic thinking in transport research and practice (in Section 3) and the resulting specialisation and monopolisation of particular transport planning software products (outlined in Section 4).
The focus of the paper is Section 5, which reviews open source software ecosystems that enable an integrated approach, which combines non geographically explicit stages (e.g. modelling) and geographic processing stages *in a single workflow*.
Three software ecosystems (R, Python and QGIS) are reviewed in detail; alternative current and potential future approaches, including 'cloud lock in' are discussed; and the relative merits of different approaches are discussed.
Building on this discussion, Section 6 concludes by returning to the importance of integrating geographic analysis in transport planning workflows.
Before proceeding with the main task of this paper --- to review new tools for integrating geographic analysis in transport planning, and provide guidance to transport researchers and practitioners tackling 21^st^ Century challenges --- it is worth taking a step back, to think about the key policy drivers of 21^st^ century transport planning.

<!-- and outlines concrete steps that can be taken to accelerate the transition to open source software in transport planning in support of policies that accelerate the transition away from fossil fuels. -->

# Policy drivers for geographic analysis in transport planning

The history of transport modelling shows that transport planning software was originally designed to plan for "increased use of cars [for personal travel], and trucks for deliveries and goods movement" [@boyce_forecasting_2015].
Despite the fact that policy drivers have changed dramatically --- with climate change mitigation, air quality improvement and public health prioritised in the 'sustainable mobility paradigm' [@hickman_transitions_2011] --- incumbent transport software still largely based on tools focussed on motor traffic, emphasising travel time savings and (de)congestion effects of interventions at relatively low levels of geographic resolution.
Tools for 21^st^ Century transport planning will need to tackle very different questions, such as:
What are the barriers preventing people from switching to more sustainable modes of transport, and where are these barriers located?
How are transport behaviours likely to shift in the future, in response to technological changes including autonomous vehicles and the continued rise of online working?
Where will different types of intervention be most effective?
And how can citizens be engaged in transport decisions?
Tools that can help answer these questions are becoming an increasingly important part of the transport planner's cabinet [@te_brommelstroet_developing_2008].

To illustrate this point, imagine being the mayor of a major city that has declared a 'climate emergency' and being given the task of leading the transition away from fossil fuels [@hadfield_financing_2019].
Policies such as carbon taxes would undoubtedly have geographic implications, but the intervention itself (charging a fixed price for the extraction and sale of atmosphere polluting substances) could be essentially non-geographic.
Notwithstanding changes to national policies relating to transport, transport planning options, by contrast, are inherently spatial.
Even high level national plans for a walking and cycling revolution must be implemented locally, down to the level of streets, as illustrated by the still ongoing local implementation of Dutch cycling ambitions [@pucher_making_2008]. 
The political-democratic and local-geographic aspects of transport planning can be considered in isolation, but an integrated approach is necessary for effective policies [@hull_policy_2008].
This is well illustrated by prominent Mayoral transport policies in cities such as 
London[^1],
Paris^[
The current Mayor of Paris, Anne Hidalgo, sees transport as a priority and has plans to make public transport free. See 
[paris.fr](https://www.paris.fr/rechercher/transport).
],
and Bogotá^[
Bogotá has an innovative and prominent transport policy, led by the two times mayor Enrique Peñalosa, who has led the roll-out of major bus and cycleway projects in the city. See [sitp.gov.co](https://www.sitp.gov.co/).
].


[^1]: Transport is a major electoral issue in London and the current Mayor, Sadiq Kahn, has made tackling air pollution a policy priority. See [tfl.gov.uk/corporate/about-tfl/the-mayors-transport-strategy](https://tfl.gov.uk/corporate/about-tfl/the-mayors-transport-strategy).

With such issues --- climate change, air pollution, obesity and social inequalities --- high on the political agenda, and the benefits for 'early adopters' of evidence-based interventions to accelerate the shift away from the motor car in cities such as London, Paris and Bogotá, pressure is growing on local, city and national transport planning departments to act.
But what should they do?
This *policy* question raises important *research* questions:
Which methods are most suitable for designing future transport systems?
What is the evidence base, and analysis, that should be used to inform transition towards a healthy, zero carbon transport system?
Which interventions, from the multitude of options available, are most likely to be effective?
And where are different types of intervention most likely to succeed?
The premise of this paper is that new approaches, enabled by software, are needed to provide answers to these questions.
21^st^ Century demands for transport planning cannot be delivered by 20^th^ century technology.
Could open source solutions are poised to bridge the gap between the geographic and the --- historically dominant --- non-geographic aspects of transport planning?
Planning for sustainable modes, walking and cycling in particular, requires analysis at higher geographic resolutions than planning for motor traffic.
Furthermore, the policy context increasingly demands transparency and citizen involvement in the decision-making process.
Only open source ecosystems, of the type outlined in Section \@ref(new-tools-of-the-trade) can deliver true transparency and encourage 'citizen science' for everyone.
These policy drivers make an exploration of open source options for transport planning workflows timely.

Additional drivers of change in transport planning software include new datasets and technologies.
Technological change has historically driven innovation in transport planning tools [@boyce_forecasting_2015] and the rate of change now is faster than ever.
With unprecedented access to increasingly detailed datasets on transport behaviours and infrastructure, transport planners today require tools that enable them to make sense of this 'data revolution' [@transport_systems_catapult_transport_2015].
The sheer volume and complexity of new datasets require new approaches that can scale and integrate multiple data sources [@lovelace_big_2016].
Advances in software and hardware allow not only for current transport systems to be modelled at high temporal and geographic resolution, but for future scenarios and 'model experiments' to be developed, which can support identification and implementation of the most effective interventions [@klosterman_what_1999].

With the explosion in open source software, which has come to dominate data science, policy, data and technological drivers are pushing geographic analysis to be better integrated in transport planning tools, alongside wider shifts for towards more data driven, transparent and democratically accountable transport planning workflows.
At present this dream is far from reality, despite a long history of geographic thinking and geographic methods in transport planning, outlined in the next section.

<!-- Transport planning has been slow to adapt to the data revolution and, while it evolves to enable a wider range of input data sources and analysis 'in the cloud', the open source element is conspicuously lacking. -->
<!-- To understand and effectively challenge the incumbent software landscape (which is described in Section xx) it is worth understanding something of the history that led to this situation. -->

# Geographic analysis and transport planning

The concept of integrating geographic data analysis in transport planning is not new, although tools and datasets for doing so quantitatively are.
Geographic perspectives have contributed to transport thinking for over 100 years, as documented in papers on geographic considerations in railway design and other transport engineering challenges [@farnham_relation_1912; @buxton_balkan_1908].
Since then, the importance of geographic analysis in transport planning has only grown, with the realisation that interventions in the transport system are most effective when they are placed where they are most needed: infrastructure designs and localised policies are most effective when they account for the geographic distribution of intricate spatial networks and interacting places of transport supply and demand
 [@rodrigue_geography_2013; @loidl_gis_2016; @lovelace_propensity_2017].

In the inter-war period (1918-1939) disciplinary homes for transport research had yet to emerge and geographic analysis was limited by lack of datasets and computers on which to process them.
The term 'transport geography' itself only became widespread in the 1950s, as noted in a report commissioned by the US Office of Naval Research: "geographers, both in Europe and America, are coming to recognize that the study of the connections between areas and of spatial interchange can provide a new and deeper insight into the meaning of areal differentiation" [@ullman_transportation_1954]. 
In the pre-computing age the relationship between geographic analysis and transport planning was characterised by growing interest in the topic, an understanding of the importance of geographic considerations in the design and evolution of transport systems, but lack of data and computational resources.^[
@paterson_horse_1926, for example, speculated quite accurately on the continued rise of motor traffic at the expense of horse powered transport during the 20^th^ Century, noting the importance of geographic factors in determining mode choice, down to the street level:
"Many streets, like our Bond Street, Watling Street or Lombard Street, and in Seville, the Calle de las Sierpes or Kalver Straat in Amsterdam, may be unsuited to motor traffic, and frontage values may be so high that widening can hardly be considered."
In a geographic review of Japanese cities @trewartha_japanese_1934 also alluded to the relationship between geography and mode choice:
"widening and paving of [roads] have (sic) been accomplished [allowing] numerous taxis, motor busses, and tram cars contrasting with the slow human and animal-drawn carts and the ubiquitous bicycle".
Rapid industrialisation during the largely unconscious build-up to World War II was associated with major road building schemes in many developed regions, demanding the practical application of new methods from a range of disciplines [e.g. @greenshields_studying_1936].
In the USA, highway engineering even became a recommended case study for geography lessons [@fox_main_1923].
]

```{r, echo=FALSE, eval=FALSE}
txt = tesseract::ocr("/tmp/a515a55e15285bf2fd100fd1d8edd91c.pdf")
writeLines(txt, "/tmp/txt.txt")
file.edit("/tmp/txt.txt")
```

At the turn of the computing age in the 1950s and 1960s transport planners, whose primary task was often to enable rapid growth in car ownership and use, quickly saw the potential for computational tools to assist their work [@boyce_forecasting_2015]: "essential to [new methods in transport planning] were new computational capabilities, the first mainframe computers, unprecedented in memory and speed [yet] tiny from today’s perspective".
As *Forecasting Urban Travel* [@boyce_forecasting_2015] recounts, geographic questions were at the forefront of many planners' minds and a key task for the early transport models was to visualise and model the results of large origin-destination surveys to help decide where new highways should be constructed.

The origins of this 'computational transport planning' activity, in which geographic analysis was an integral part, were publicly funded transport planners and engineers solving real real world problems.
However academics, and quantitative geographers in particular, soon started working with newly available transport datasets.
An important development was the emergence of spatial interaction models, which were formally defined, refined and implemented throughout the 1960s and 1970s [@wilson_statistical_1967; @wilson_family_1971].
It is notable that Alan Wilson, whose research influenced both transport planning and academic practice, worked in both the public sector (for the UK's Ministry of Transport) and academia (the University of Leeds) while writing each paper.
Most academic research at the interface between transport planning and geographic research is far less 'practitioner facing' and, if anything, it seems that tools used in geographic analysis of transport systems in practice since the 1970s have diverged from academic research.

<!-- perhaps explaining why the field of Transport Geography, which emerged in the 1960s and grew rapidly since then [@hay_transport_1979], has (to the author's knowledge) had relatively few other notable impacts on transport planning practice. -->

By the late 1970s, there was enough research for review papers reflecting on the status of Transport Geography as a self-standing branch of Geography [@rimmer_redirections_1978].
A book on the transport geography of India provides an insight into the field at the time, with a focus on infrastructure and statistics, transport geography sat firmly in the quantitative tradition of geographic research [@raza_transport_1986], despite Rimmer (1979) criticism that much of the field ignored the wider impacts of transport systems.
Geographic analysis in transport research was given a substantial boost in the 1990s, with the first publication of the Journal of Transport Geography [@knowles_research_1993].
Transport Geography has subsequently come to be defined as a branch of geography.
Notwithstanding influential methodological and review papers proving transport planners with insight into the state-of-the-art [e.g. @martinez_new_2013], the level of engagement between academic transport geographers and transport planning practitioners is debatable (although the same could also be said of academic planners). 

Around the turn of the century, there were attempts to define a more applied geographic information systems (GIS) approach transport research.
Labelled GIS-T, the field was posited as an academic field at the interface between transport planning and GIS [@miller_potential_1999].
Although the label gained limited traction in academia or practice, Harvey Miller's call for a shift to methods and tools has been answered in the 2000s and 2010s by researchers who have developed ideas and software that transport planners can actually use, including the Australian Research Infrastructure Network (AURIN), which is widely used for transport planning and public health research in Australia [@pettit_australian_2014] and the Propensity to Cycle Tool (PCT, publicly available, including source code, at [www.pct.bike](https://www.pct.bike)) [@goodman_scenarios_2019].

<!-- Search term for interwar period: https://scholar.google.co.uk/scholar?q="transport+geography" -->


<!-- something on the lack of open source? -->

<!-- https://www.abdn.ac.uk/registry/courses/undergraduate/2016-2017/geography/gg4016
https://people.hofstra.edu/jean-paul_rodrigue/course_transport.html
 in Geography with Transport Studies BA advertised by the University of Leeds
-->


<!-- The paper concludes that 'integrated approach' can support efficient, scalable and reproducible transport planning workflows which can provide a strong and transparent evidence base needed for rapid transition away from fossil fuels in the transport sector. -->

# The landscape of transport planning software

Before describing the existing landscape, it is worth outline what transport planning software is.
Software for transport planning can be grouped by the scale at which it operates, with broad categories being 'micro' and 'macro' models [@kotusevski_review_2009].
'Microscopic' transport models represent individual vehicles on the road network and are therefore able to represent localised phenomena such as traffic congestion.
Macro models, by contrast, represent aggregates of vehicular traffic over large spatial scales, with a focus on the implications of future changes in transport behaviour and infrastructure on flow at the route network level.
Of course the distinction is, in reality, an oversimplification: there is a continuum between micro and macro transport modelling software.
With advances in computer hardware and software, an increasing number of developers are attempting to combine both approaches into a single system.
In this paper we focus on macro models, and their geographic representation, rather than micro models. 

The geographic and non-geographic division of labour is a result of the history of transport planning software.
This history is detailed in Chapter 10 of *Forecasting Urban Travel* [@boyce_forecasting_2015].
Titled "Computing environment and travel forecasting software", the chapter provides a unique insight into the software packages that have been popular in transport planning over the years.
Of course, software development has always depended on the physical hardware on which it runs and the early days of transport planning software were characterised by bespoke programs running on mainframe computers and maintained by domain experts.
Transport planning bodies and researchers in the USA led developments in the 1960s and 1970s when computers first started to be used for transport planning, when the main problem that they addressed was how to deal with the explosive growth in car ownership and use that was taking place during those decades.
More overtly political factors also influenced the direction of transport planning software: 
"certain private firms complained to US DoT [Department of Transport] that its agencies were developing software in competition with the private sector", leading to the abandonment of publicly funded transport planning software development projects, notably UTPS [@boyce_forecasting_2015].^[
UTPS stands for the UMT (Urban Mass Transportation Administration, an agency of the DoT responsible for transport planning) Transportation Planning System (UTPS) and PLANPAC
]
This transfer of transport planning software development to the private sector contrasts with the history of GIS, in which open source solutions have come to dominate .
The example of GRASS (Geographic Resources Analysis Support System) illustrates this point and helps explain the dominance of proprietary software in transport planning.
Like UTPS, GRASS was a publicly funded software project.
Unlike UTPS, it was made freely available to the public and was open sourced (in 1999), meaning that it has been under continuous development by state, academic and commercial organisation since 1982 [@neteler_open_2008].
Would the landscape of transport planning software have been different if the DoT had continued to fund software development projects?
That question is outside the scope of this paper.
What is certain, however, is that software used in transport planning over the past three decades has been dominated by companies and that the sector has been slow to adopt open an open source approach.

In response to the 'siloed' development of GIS and transport software, there have been calls for greater integration.
@loidl_gis_2016, building on the observation that "geography and GIS remained a niche topic within traditional transport modeling", made a case for strengthening the 'spatial perspective' in transport modelling.
The paper emphasised the growing importance of well-defined data types, disaggregating detailed (and difficult to interpret) transport model outputs, and geographic data visualisation and concluded that much further research is needed:
"future research and development is needed to combine geospatial functionalities with transport modeling,
while providing an efficient, interactive, visual interface for data exploration, manipulation, analysis
and visualization" [@loidl_gis_2016].
Although the paper focussed on conceptual issues rather than software per-se, it did identify mention four open source programming languages that could provide the foundation for future developments, two of which (R and Python) are covered in the next section.

Data preprocessing and analysis stages are generally done in dedicated transport planning and spreadsheet software.
Geographic analysis and cartographic visualisation stages are often done in a dedicated (GIS).
Some prominent transport planning software products, and levels of support for geographic data analysis, are summarised in Table 1.

```{r, echo=FALSE, message=FALSE, warning=FALSE}
geo_capabilities = "I, B, E, P"
tms = readr::read_csv("https://github.com/ITSLeeds/TDS/raw/master/transport-software.csv")
tms = arrange(tms, desc(Citations))[c(1, 2, 4)]
# knitr::kable(tms, booktabs = TRUE, caption = "Sample of transport modelling software in use by practitioners. Note: citation counts based on searches for company/developer name, the product name and 'transport'. Data source: Google Scholar searches, October 2018.", format = "latex")
# tms$`Geographic capabilities` = c(
#   "Plotting, editing, import, buffer, intersections", # https://www.ptvgroup.com/fileadmin/user_upload/Products/PTV_Visum/Documents/Release-Highlights/PTV_Visum_18_release_highlights_EN.pdf # http://www.traffic-inside.com/2014/11/27/ptv-visum-tip-catchment-areas-accessibility-of-places-in-the-network/
#   "I, E, P, R"
# )
tms$I = c("Y", "Y", "Y", "Y", "Y", "Y", "Y")
tms$G = c("Y", "?", "Y", "?", "Y", "?", "Y")
tms$R = c("Y", "Y", "Y", "?", "Y", "?", "Y")
tms$RNA = c("Y", "Y", "Y", "Y", "Y", "Y", "Y")
tms$SV = c("Y", "Y", "Y", "Y", "Y", "Y", "?")
tms$IV = c("?", "?", "?", "?", "?", "?", "?")
tms$EX = c("?", "?", "?", "?", "Y", "?", "?")
knitr::kable(tms, caption = "Sample of transport modelling software in use by practitioners. Note: citation counts based on searches for company/developer name, the product name and 'transport'. The columns I, G, R, RNA, SV, IV  and EX refer to Import of a wide range of geographic data formats, Geographic capabilities such as buffer calculations and intersections, Route calculation, Route Network Analysis, Static Visual outputs, Interactive Visual outputs for web publication, and Export to a wide range of geographic data formats, with ? meaning partial support (e.g. via add-on software). Data source: Google Scholar searches, October 2018.")
```

Table 1 shows that popular transport planning tools have differing levels of geographic capabilities.
It should be noted that the geographic capabilities were assessed based on reading of publicly available manuals (to be linked to in an appendix accompanying this paper) and that each software product is actively developed, meaning that the results may change with additional information and subsequent releases.
An interesting pattern is that the open source options --- MATSim, SUMO and sDNA --- all have limited 'in house' geographic capabilities.
This can be explained by the 'Unix philosophy', the second tenet of which is modularity, meaning that "each program should do one thing well", reducing duplication of effort and allowing the best tool to be used for each job [@gancarz_linux_2003].
The next section describes the this modularity in more detail, including outstanding support for geographic data in open source software.

A major barrier affecting the current landscape of transport planning tools is accessibility and reproducibility:
All the proprietary products are expensive (costing hundreds of dollars for a single licence), ensuring that only a small fraction of transport planners, let alone the public, has access to them.
Another barrier associated with the proprietary options is platform dependence: as far as the author can tell, they all run only on the proprietary operating system Windows, preventing use in on other operating systems such as Linux, Mac and FreeBSD.
A final issue affecting reproducibility with the proprietary options listed in Table 1 is that they all have a prominent Graphical User Interface (GUI) (although they increasingly offer a command line interface, enabling scripting).
As is the case with GUI based GIS software, this has the "unintended consequence of discouraging reproducibility" by enabling the user to get to a solution without writing a script that others can use [@lovelace_geocomputation_2019:1].

Another barrier, which may affect the open source options listed in Table 1 more than the proprietary options, is that they can be (in the author's experience) difficult to install and use.
This creates an additional barrier to the integration of geographic analysis in transport planning for people, especially the majority of people who have limited computing skills.

A final barrier, which may be more social and organisational than software-related (although discerning cause and effect is difficult), is that organisations' GIS and Transport functions tend to be siloed into their respective departments/teams with little communication between them, meaning that transport planners may not have access to the latest geographic data or software.^[
Thanks to Crispin Cooper, author of sDNA, for raising this barrier.
]
A software-related issue is that, if transport planners and GIS analysts are using different programs for their work, transport planners will be less likely to collaborate with people with geographic analysis skills or identify potential geographic solutions to their domain-specific problems.
To what extent can these barriers be overcome by open source software ecosystems?
That is the topic of the next section.


```{r, echo=FALSE, eval=FALSE}
# alternative way to get citation data (WIP)
library(fulltext)
ft_search("visum", from = "crossref", scopusopts = )
scopus_search(query = "visum", key = Sys.getenv("SCOPUS"))
```

# New tools of the trade

The previous sections support and expand on the two main premises of this paper: that geographic analysis has much to offer transport planning, and that the incumbent proprietary software products are not well suited to tackle 21^st^ Century transport planning needs.
In this section the paper shifts gear, and moves onto solutions.
It outlines the growth of free and open source software (FOSS) and how the movement can provide the foundations for more democratic and transparent transport planning workflows that bridge the 'geographic gap' in transport planning data analysis, modelling and visualisation.
The focus is on three software 'ecosystems' --- R, Python and QGIS --- that are particularly promising for integrated geographic analysis in transport planning.

Before exploring these ecosystems, it is worth first taking a step back and considering the open source software landscape and what 'open source' actually means.
This overview also helps explain why R, Python and QGIS were selected from the range of open source options for closer attention.

<!-- Despite the central role that open source software plays, powering the majority of the world's servers... -->
Open source software differs from proprietary software in that users are free to see, download and modify the underlying source code that defines it.
Freedom is central to open source software, which is sometimes referred to simply as 'free software', defined by the Free Software Foundation ([FSF](https://www.fsf.org/about/what-is-free-software)) as follows:

> software that gives you the user the freedom to share, study and modify it.

This adaptability is conducive to collaboration, the creation of mutually supportive user/developer communities and rapid evolution, making open source software ecosystems fast moving and highly diverse.
It is impossible to discuss all software options that could be used for geographic transport planning: there are literally thousands of software projects, written in hundreds of programming languages, many of which are no longer actively maintained.
Transport planners should use solutions that are future proof and actively maintained --- such as R, Python and QGIS. 

A key advantage of ecosystems based on a *command line interface* (CLI), such as R and Python, rather than ecosystems based on a *graphical user interface* (GUI), such as QGIS, is reproducibility.
A programming approach, in which the user interacts with the computer primarily by typing code, enables others to see and potentially reproduce analysis just by sharing the underlying code.
Programming also provide great flexibility: the user is not constrained by the options provided in the GUI.
The downside of CLIs is that they can take time to learn, especially for people who have been trained on GUI-based software such as Microsoft Word.
Transport data analysis has much in common with the broadly defined field of 'data science', and many of the tools developed for this purpose (including those in the R and Python ecosystems detailed below) have great potential for transport planning. Other open source ecosystems include those surrounding the languages JavaScript, Julia and Rust, and the GUI-based Java application [IRIS](http://iris.dot.state.mn.us/).
Promising as these, and many other, open source projects are, R, Python and QGIS were selected due to their maturity, wide uptake in industry and academia for data science (for R and Python, meaning a large user community) and geographic analysis (for QGIS).
They have also seen impressive growth in popularity over the past decade, suggesting they are future proof, as illustrated in Figure 2.


```{r, eval=FALSE}
library(gtrendsR)
res = gtrends(c("R programming", "Python programming", " VBA programming"), time = "2010-01-01 2019-08-01")
res
p = plot(res)
class(p)

head(res$interest_over_time)
tail(res$interest_over_time)
head(res$interest_over_time$time)
class(res$interest_over_time$time)
unique(res$interest_over_time$keyword)
nrow(res$interest_over_time) / 3 / 12
nrow(res$interest_over_time) / 3
res$interest_over_time$date = rep(seq(from = as.Date("2010-01-01"), to = as.Date("2019-07-01"), length.out = 116), 3)
start_rows = 1
end_rows = nrow(res$interest_over_time) / 3

res$interest_over_time$hits_smoothed = c(
  zoo::rollmean(res$interest_over_time$hits[start_rows:end_rows], 6, fill = "extend"),
  zoo::rollmean(res$interest_over_time$hits[(start_rows:end_rows) + end_rows], 6, fill = "extend"),
  zoo::rollmean(res$interest_over_time$hits[(start_rows:end_rows) + 2 * end_rows], 6, fill = "extend")
)

res2 = gtrends(c("RStudio", "Pycharm", " QGIS"), time = "2010-01-01 2019-08-01")

p2 = plot(res2)

devtools::install_github("thomasp85/patchwork")
library(patchwork)

p + p2

end_rows = nrow(res2$interest_over_time) / 3
class(res2$interest_over_time$hits) = "numeric"

res2$interest_over_time$hits_smoothed = c(
  # slide::slide_dbl(res2$interest_over_time$hits[start_rows:end_rows], ~mean(.x), .before = 6, .after = 6)
  zoo::rollmean(res2$interest_over_time$hits[start_rows:end_rows], 6, fill = "extend"),
  zoo::rollmean(res2$interest_over_time$hits[(start_rows:end_rows) + end_rows], 6, fill = "extend"),
  zoo::rollmean(res2$interest_over_time$hits[(start_rows:end_rows) + 2 * end_rows], 6, fill = "extend")
)

res_all = rbind(res$interest_over_time, res2$interest_over_time)

ggplot(res_all) +
  geom_line(aes(date, hits_smoothed, colour = keyword), size = 2) +
  ylab("Relative number of hits") +
  theme(legend.position = c(0.2, 0.8))
  
ggplot2::ggsave("google-trends-open.png", width = 6, height = 4)
```

```{r, fig.cap="Relative number of searches for terms related to open source (R and Python) and proprietary-based (VBA) programming languages (left) and open source programs (Pychame, QGIS, RStudio) that can be used for transport planning. Code to reproduce the plot is hosted in this paper's code repository.", out.width="70%"}
knitr::include_graphics("google-trends-facet.png")
```

Each open source ecosystem, and it's potential to be used for geographic analysis in transport planning, is outlined below.

## R

R is a "a language and environment for statistical computing and graphics" [@r_core_team_r:_2019].
First announced and released as a binary program in 1993 by University of Aukland statisticians Robert Gentleman and Ross Ihaka, the project was only open sourced and released under the conditions of the GNU General Public License (GPL) in 1995, thanks to input from one of R's first international collaborators, Martin Mächler of ETH Zurich
[@ihaka_r:_1998].
This history highlight's how open source software development is an inherently collaborative process, usually involving people from many different countries and backgrounds.

R has several strengths from the perspective of transport planning, including its proficiency with temporal and geographic data, outstanding visualisation capabilities, and support for a very wide range of statistical techniques, many of which are useful in transport problems [@lovelace_stplanr:_2018].
Out of the box R is a statistical powertool that can solve a wide range of problems, including generalised linear models (GLMs, implemented with the function `glm`) and constrained optimisation problems that appear frequently in transport research.
Additional capabilities are supported by 10,000+ packages that can be installed from a central repository with commands such as `install.packages("stplanr")`.^[
Like Python packages, R packages are analogous to Apps on smartphones and plugins in QGIS (described below), that provide new functionality.
Many implement recently developed statistical and computational techniques (some of which are accompanied by papers describing new methods in academic journals such as the *Journal for Statistical Software*) or provide interfaces to software written in other languages, meaning that R can provide transport researchers with access to many cutting-edge methods via a single system.
]

A good example of a transport problem that R's statistical capabilities are well suited to solving is mode choice.
Unimodal models estimating mode share (or the logit thereof) can use R's inbuilt statistical capabilities, as demonstrated in the Propensity to Cycle Tool project [@lovelace_propensity_2017].
More sophisticated multinomial models are needed when estimating mode share across multiple travel options such as walk, cycle, bus [@martin_individual_2019].
R has mature support for such models via the `multinom` function in the longstanding package `nnet` [@venables_modern_2002], as demonstrated by [Germán Rodríguez](https://data.princeton.edu/wws509/r/c6s2).
Subsequent packages provide additional methods for estimating mode split [@hasan_fast_2016; @croissant_mlogit:_2019].
Appollo and mlr3 are recently developed examples of R packages providing support for sophisticated choice models and cutting edge machine learning functionality, respectively.

<!-- [@hess_apollo_2019;@bischl_mlr:_2016]. -->

R is well known for having outstanding statistical analysis and modelling capabilities, of the type useful in transport planning.
Less known is that R also has a mature ecosystem for working with geographic data, making it well suited to the task of integrating geographic analysis in transport planning: R excels at doing modelling *and* geographic analysis.
This is particular interest here because, as outlined in previous sections, 'context switching' between programs for statistical and geographic analysis is time consuming.^[
The author has first hand experience of the costs of context-switching: during my PhD I used R for the statistical and modelling analysis, and then switched to QGIS for geographic analysis and visualisation.
While this approach worked well, the cognitive burden of having to learn and manage two substantial programs was substantial.
]
Support for geographic data and methods have a long history in R [@rowlingson_rasp:_2003;@bivand_implementing_2006;@pebesma_software_2015;@bivand_applied_2013].
The development of R's spatial capabilities are well documented elsewhere [link to other articles in the special edition].
However, a few advances are worth mentioning due to their relevance to transport transport planning.
The package `sf` [@pebesma_simple_2018] provides a unified and high performance system for working geographic lines (in addition to its support for points and polygons), which can be used to represent roads.
Building on `sf`, the package `stplanr` [@lovelace_stplanr:_2018] provides many functions for working with geographic transport data, including `overline` which enables thousands routes to be aggregrated to create route networks (Morgan and Lovelace, in press) and `dl_stats19`, which has evolved into the `stats19` package [@lovelace_stats19:_2019].
Geographic data visualisation, cartography, is another area where R excels, with packages such as `tmap` [@tennekes_tmap:_2018] providing powerful functions for map making.
These and many other packages for working with geographic data in R are described in detail in *Geocomputation with R* [@lovelace_geocomputation_2019:1].
Chapter 12 this of this open source book is dedicated to transport applications, and provides a good starting point for learning more about using R's impressive geographic capabilities for transport planning.

## Python

Python is a general-purpose programming language originally conceived in the late 1980s and first released in 1991 [@rossum_python_1995].
The language was designed from a computer science perspective, with a focus on code elegance and consistency, rather than R's focus on statistical functionality.
However, Python has become very popular for data analysis and 'data science' thanks to packages such as [Pandas](https://github.com/pandas-dev/pandas), and SciKitLearn [@mckinney_python_2017].
Due to its range of features, large open source community, and flexibility, Python has been used as a 'glue' language to interact with many other software systems.
It is a highly diverse language that is widely used in domains ranging from web development [@grinberg_flask_2018], to computer vision and [@zafar_hands-convolutional_2018] text analysis [@bengfort_applied_2018].

Of particular interest here is Python's support for geographic data.
There are dozens of geographic projects written in Python, ranging from the use of the language to teach low level geographic concepts and algorithms [@xiao_gis_2016] to its use as an interface to libraries such as GDAL.
Dozens of Python packages have been published for solving specific geographic problems, ranging from the processing of scientific gravimetric measurements [@mccubbine_gsolve_2018] to handling remote sensing data for Ireland [@serbin_open-sourced_2018], both of which could have transport applications.
Furthermore, Python is used as the language of choice of choice for command-line interfaces to the popular proprietary GIS ArcMAP [@zandbergen_python_2015].

More general purpose package for handling spatial datasets that could be used for transport research include [GeoPandas](http://geopandas.org/), for handling vector data such as roads and [rasterio](https://rasterio.readthedocs.io/en/stable/) for handling raster datasets.
Building on such foundations, a number of Python developers have written packages with a transport focus.
This include OSMnx [@boeing_osmnx:_2017], for downloading and analysing street network data from OpenStreetMap and the recent and ambitious project 
[@pappalardo_scikit-mobility:_2019] which, despite limited documentation at the time of writing, sets out to create a framework for modelling transport systems.

Because Python is a general purpose language, it has been used as the basis of transport applications that go beyond the transport planning remit of this paper.
A couple of projects are worth mentioning to give an indication of the wider Python transport ecosystem.
[Itinerum](https://github.com/TRIP-Lab) is an open source travel survey development project, which includes a backend written in Python and smartphone apps [@patterson_itinerum:_2019].
A similar project is [E-mission](https://github.com/e-mission) [@shankari_e-mission:_2018] the backend of which is partly written in Python. 


## QGIS

QGIS is a cross-platform desktop GIS application with a huge user base (likely the most popular GIS software in the world) and more than 1000 community supported plugins [@qgis_development_team_qgis_2019].
QGIS itself is written primarily in C++ and Python, meaning that there is strong symbiosis between the Python and QGIS ecosystems.
In fact the majority of QGIS plugins are written in Python, meaning that Python developers can use QGIS as a platform for providing users with a graphical user interface and, conversely, QGIS users can learn to use a CLI, via QGIS's Python console.

A good example of the flexibility of QGIS's plugin model, and illustrating the wider point that open source software tends to be modular and do 'one thing well', is that sDNA, mentioned in Section \@ref(the-landscape-of-transport-planning-software) can be used in QGIS via the sDNA plugin [@cooper_sdna:_2015].
Indeed, the open source transport modelling framework MATSim also benefits from being used alongside QGIS, for road network editing and visualisation [@horni_multi-agent_2016].
QGIS plugins and extensions specifically designed for transport planning applications include the AwaP-IC walkability analysis plugin [@majic_awap-icopen-source_2019], extensions to QGIS's Processing model development framework for assessing road network completeness [@sehra_assessing_2017] and the ORS (OpenRouteService) plugin, for multi-modal routing.

QGIS can also be used as a self-standing application for transport data analysis.
@ilayaraja_road_2013, for example, used QGIS as the platform of choice for analysing street networks using the [Road Graph plugin](https://docs.qgis.org/2.14/en/docs/user_manual/plugins/plugins_road_graph.html), which ships with QGIS by default.
@dong_population-weighted_2016, to provide another example, used QGIS as the basis of geographic analysis of route network efficiency.
Hundreds of other transport planning projects have used QGIS for the mapping and geographic analysis component of the work and, due to the application's user friendly GUI, it is rapidly gaining in popularity among government transport planning departments, including Transport for Wales and Transport for Greater Manchester.

# Conclusion

Geographic analysis is an important yet often under-appreciated aspect of transport planning, and looks set to play a more prominent role in transport policy-making due to technological, data and policy drivers of change.
In the context of the obesity crisis, air pollution concerns and the 'climate emergency' that has been declared by some city authorities, many transport planners have been tasked with discouraging energy intensive modes, particularly car use, and enabling more walking and cycling [@hickman_transitions_2011] .
Furthermore, in the age of evidence-based policy, open data and citizen science, there is an onus on practitioners to provide solutions that are transparent, accessible and, increasingly, participatory [@] .

This poses a challenge to incumbent transport planning software which is expensive and thereby inaccessible to most people, monolithic and (to a greater or lesser extent) limited in terms of geographic capabilities, particularly in relation to publicly accessible interactive visualisation and adaptability.
Partly in response to these pressures and challenges, a number of open source transport planning tools have emerged, notably MATSim, SUMO and sDNA.
Following the Unix philosophy of modularity [@gancarz_linux_2003], each of these tools has a particular niche.
However, despite a long history of geographic thinking in transport planning and calls to strengthen the links between GIS and transport software [@loidl_gis_2016] but none of the open source solutions reviewed provide 'in house' GIS capabilities of the type needed for an integrated transport planning workflow in which a single piece of software can be used to import a range data, undertake statistical analysis and modelling and plot the results in interactive maps, e.g. for the design and modelling of new walking and cycling networks.

With a view to the future of transport planning software, three established 'software ecosystems', each of which has a substantial following and large community of developers building extensions, were reviewed with respect to their capacity to support geographic analysis in transport planning.
The literature shows that R, Python and QGIS communities have already developed several tools for transport planning that, when combined with other open source solutions, can solve a very wide range of spatial transport planning problems.
Although each ecosystem is mature (yet still growing), their use in transport planning is still in a nascent phase, suggesting that much innovation, evolution and consolidation will occur before any strong conclusions about which is most appropriate for different transport planning tasks can be made. 
However, tentative guidance can be made, based on the origins and direction of each project: for statistically-orientated projects in which interactive online visualisation is vital, R provides a strong foundation; for applications in which general purpose languages and interoperability with other frameworks, and integration with other software frameworks, Python may be the most appropriate option; and when a user friendly interface and rapid map making without need for programming skills are required, QGIS may be suitable.
There is a huge amount of overlap between the three ecosystems and, in practice, the prior experience and preferences of transport planners may be more important than functionality.

As the FOSS philosophy described in Section \@ref(new-tools-of-the-trade) emphasises, open source software by its very nature is collaborative, innovative and evolving [@gancarz_linux_2003], allowing it to out-compete and eventually dominate in sectors from machine learning to web development.
The review of capabilities in open source software communities clearly shows that high-performance and innovative solutions are already available in the 'ecological niche' of geographic analysis for transport planning.
Given the nascent nature of many of the transport-oriented packages, plugins and extensions in each ecosystem, fruitful directions of research would explore the relative merits of different options, and combinations of options, in terms of computer and programmer efficiency.
Furthermore, the modular and 'pluginable' nature of open source software suggests there are great opportunities for integration: could there be R and Python interfaces to MATSim, SUMO and sDNA?
And from a research perspective, how can the growth of open source solutions for geographic transport data analysis be monitored to identify 'tipping points' in practitioners' uptake of open source solutions?

These considerations wider questions, about if and when will open source software rise to ascendancy in the wider field of transport planning.
Returning to the most urgent policy driver of climate change mitigation, it is clear than a step change is needed in transport interventions.
If these interventions are made on the basis of analysis undertaken in open source software --- enabling rigorous, transparent and reproducible evidence that can easily be repeated in new settings and when new datasets become available --- they are all the more likely to succeed.
In this sense it may not be an exaggeration to say that open source software can save the world.


# References