-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path04-results.Rmd
111 lines (79 loc) · 21 KB
/
04-results.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
# Results {#results}
We estimated ride-hailing ridership and level of service for each of the nine previously mentioned ActivitySim-to-BEAM mode choice combinations. Specifically, we estimated ride-hailing ridership, wait time, and utilization. Alongside each other, these results shed light on how different combinations of an activity-based model and multi-agent simulation perform at estimating ride-haling ridership and level of service. We can see how differing mode choice structures affect the use and performance of ride-hailing modes.
## Ridership {#res-ridership}
Table \@ref(tab:ridership) shows the number of forecasted trips for the ride-hail, pooled ride-hail, and ride-hail transit type modes for all nine mode choice combinations. Table \@ref(tab:ridership) also includes the number of forecasted trips within the plans with ride-hail created by ActivitySim, before the mode choices were changed by BEAM. Lastly, Table \@ref(tab:ridership) provides a total percentage column, showing the total percentage of ride-hailing modes in relation to the overall modal distribution.
```{r ridership, echo=FALSE}
library(kableExtra)
finalRidership <- tar_read(finalRidership)
kbl(finalRidership, caption = 'Forecasted Ride-hail Trip Ridership by Mode Choice Combination Scenario', caption.short = "Ride-hail Ridership by Number of Total Trips", booktabs = TRUE, linesep = c("\\addlinespace",rep("",9)), format="latex") %>% kableExtra::kable_styling(font_size = 10) %>%
row_spec(1,italic = TRUE, background = "#D9D9D9") %>%
row_spec(2, background = "#FAEBD7") %>%
row_spec(3:6, background = "#FFB6C1") %>%
row_spec(7:10, background = "#ADD8E6")
```
By comparing scenarios against each other, we can understand how slight differences in the mode choice model can significantly affect estimated ridership totals. We first compare the ActivitySim scenario with the AsimRideHail scenario. The ActivitySim scenario represents the ride-hailing input plans created by ActivitySim before being inserted into BEAM and the AsimRideHail scenario represents how these ride-hailing input plans change in BEAM while undergoing no new mode choice. Comparing these two scenarios we see the estimated 4,249 ride-hailing trips of ActivitySim diminished to 300 trips within BEAM because the AsimRideHail scenario was unable to recreate the same ride-hailing paths as ActivitySim. Then, comparing the *RideHail* and *All* type scenarios with the AsimRideHail scenario and ActivitySim scenario we see that when BEAM mode choice innovation is turned on for all or part of the agents, BEAM predicts significantly higher ridership totals. As a result, we suppose that BEAM is prone to estimating higher ridership totals for ride-hailing modes than ActivitySim when modal innovation is turned on, and lower totals when turned off.
Next, we examine the effect the existence of ride-hailing in the input plans has on ridership. As shown in Table \@ref(tab:ridership), minimal differences in ridership is produced between scenarios with the *Asim* prefix vs. without. For example, between the BeamRideHail:Path and AsimBeamRideHail:Path scenarios we see that input plans without ride-hail produce 72,958 estimated trips (6.18% of all trips), whereas input plans with ride-hail produce 67,829 estimated trips (5.75% of all trips). Similarly, the gap between the ride-hailing trips of the BeamRideHail:PPL and AsimBeamRideHail:PPL scenario is relatively close, at 62,463 and 78,597 trips (5.34% and 6.60% of all trips) respectively. Although thousands of trips separate the comparing scenarios, in both cases, less than a 1.5% difference in total modal distribution occurs. In addition, the output ridership trips are almost identical between the BeamAll:PPL and AsimBeamAll:PPL scenarios and close between the BeamAll:Path and AsimBeamAll:Path scenarios (between 0.50% and 0.75% of the total modal distribution). Overall, we see similar ridership results among similar mode choice structures independent of the inclusion of ride-hailing in input plans.
We also notice that different BEAM mode choice models predict different forecasted ride-hailing ridership levels. The *None* type scenario (AsimRideHail) assigns few agents to the ride-hail mode. The *RideHail* type scenarios produce the largest number of ride-hail modes among BEAM mode choice structures. The *All* type scenarios produce more ride-hail modes than the *None* type, but less ride-hail modes than the *RideHail* type. In addition, the *All* type scenarios predicts similar ridership values as ActivitySim. The number of ride-hail only type trips is 2,259 more in the BeamAll:Path scenario than in the ActivitySim scenario. All the *All* type scenarios as well as ActivitySim predict under 1% total ride-hailing modes. Overall, forecasted ridership is affected significantly by which mode choice structure is used by BEAM; this conclusion is clear.
Finally, we analyze the effect the BEAM utility variables have on forecasted ride-hail ridership totals. Comparing the BeamRideHail:Path and BeamRideHail:PPL scenarios we see ride-hail only ridership decreases (45,001 and 18,907) when using path, person, and location variables, but increases with pooled ride-hail (25,014 and 38,935) and ride-hail transit (2,943 and 4,621). This same pattern occurs when analyzing the difference between the BeamAll:Path and BeamAll:PPL, and AsimBeamAll:Path and AsimBeamAll:PPL scenarios. For some oddity though, AsimBeamRideHail:Path and AsimBeamRideHail:PPL follow an opposite pattern. We acknowledge that not all scenarios follow the same pattern, but hypothesize that in general, using only path variables to estimate ride-hail ridership will result in less total ride-hail ridership than if using all path, person, and location type variables.
## Wait Times {#res-waits}
Figure \@ref(fig:waits) shows a detailed distribution of wait times for ride-hailing vehicles for the scenarios. As with ridership, we compare the scenarios with the *Asim* prefix against the scenarios without, and see only a slight difference in maximum wait times. Scenarios AsimBeamRideHail:PPL, AsimBeamAll:Path, and AsimBeamAll:PPL have almost identical mean wait times when compared to their counterparts in BeamRideHail:PPL, BeamAll:Path, BeamAll:PPL. We suppose that the existence of ride-hail in the initial plans will not affect *most* ride-hail wait times.
```{r waits, echo = FALSE, warning = FALSE, message = FALSE, fig.cap = "Distribution of ride-hail wait times by mode choice combination scenario.", fig.scap = "Distribution of ride-hail wait times.", out.extra='', fig.align='center'}
tar_read(waits)
```
Alternately, comparing different BEAM mode options does significantly affect ride-hail wait times. The *None* type scenario (AsimRideHail) has the largest spread of wait times, the lowest mean wait time, and is "bottom heavy" -- referring to the fact that a major cluster of users wait less than 7.5 minutes. The *All* type scenarios have higher mean wait times (~9.3 minutes) than the *None* type and lower mean wait times than the *RideHail* type. Neither top nor bottom heavy, the *All* type scenarios have a more even spread in wait times, ranging from 0.4 to 21 minutes. BEAM seems to paint ride-hail alternatives as more desirable than ActivitySim, as more users are willing to wait longer (12 to 18 minutes in the *All* scenarios). This is especially true with the *RideHail* type scenarios, as a large cluster of users are willing to wait 7.5 to 20 minutes. The *RideHail* type scenarios have the largest mean wait times (~10.65 minutes). By simply viewing the wait time distribution, we see a difference in level of service depending on model structure. Wait time is significantly affected by which mode choice structure is used by BEAM, just like as was concluded with ridership.
We also conducted a statistical comparison between quantile ranges between all three model structures. Using a quantile test based on the method proposed in @wilcox, a significant statistical difference exists between the None and All models, the All and RideHail models, and the None and RideHail models at the 20th and 50th percentile levels. At the 80th percentile level, a significant difference exists between the All and RideHail models and the None and RideHail models, but not between the None and All models. The results of these statistical tests suggest that a different level of service will result based on which mode choice model structure is used. We also ran quantile tests between like model structures of scenarios with and without ride-hailing in the input plans (i.e., BeamRideHail:Path and AsimBeamRideHail:Path) and noticed no statistical difference at the 20th, 50th, and 80th percentiles. These statistical tests indicate that wait times will not change significantly based on the inclusion of ride-haling in the input plans.
The last group to compare collectively is between the Path and PPL models. By comparing BeamRideHail:Path and BeamRideHail:PPL, BeamAll:Path and BeamAll:PPL, and AsimBeamAll:Path and AsimBeamAll:PPL, we see that the Path models estimate a slightly higher maximum wait time. In addition, BeamRideHail:Path and AsimBeamRideHail:Path seem to have a larger cluster above a 10 minute wait time than BeamRideHail:PPL and AsimBeamRideHail:PPL. Besides these two observations though, the differences between utility parameters is minimal. Although ridership was affected by which utility parameters were used, wait time is only slightly affected.
Overall, by analyzing the ridership and wait times among different mode choice structures we learn that ride-hailing ridership and level of service is significantly affected by which mode choice structure is used in BEAM. The mode choice model structure one uses to estimate ride-hailing level of service and wait time matters. We also suggest that initial plans, and whether or not they include ride-hail, do not significantly affect the level at which BEAM estimates ridership or wait times. Lastly, we hypothesize that using all path, person, and location type variables will increase total ridership. We also suggest that the lack of person attributes in the utility equation may cause pooled and transit ride-hail options to look less appealing. Section \@ref(deep-look) takes a deeper look at why some of these patterns in the ridership and wait times results exist.
## Mode Choice Structures {#deep-look}
The results from Table \@ref(tab:tbexperiments) and the results from Figure \@ref(fig:waits) can be explained further by understanding the original setup of the experiments. The clearest distinction in ridership and wait times exist between BEAM mode choice structures. The *None*, *RideHail*, and *All* structure types each produce results at different magnitudes, drawing on the conclusion that mode choice model structure matters. Therefore, to best understand these structures, we explore their inner-workings.
### None Mode Choice Model {#type1}
The *None* mode choice model produces the lowest ridership and shortest wait time values. With modal innovation turned off, this model did not allow agents to choose new modes, and averted them to walk modes if their current trip mode was deemed "impossible". This is verified by looking at Table \@ref(tab:noneloss). Iteration 0 (start) shows the number of ride-hailing modes input to BEAM. By the end of iteration 0, however, more than half the initial ride-hailing selections estimated by ActivitySim were lost. Then, by the end of the final iteration, only 300 ride-hailing trips remained. At the same time, total walk modes increased across each iteration. BEAM was unable to match agents with most of ActivitySim's ride-hailing predictions.
```{r, noneloss}
none <- matrix(c("0 (start)", 2412, 1837, 0, 4249,"0 (end)", 978, 615, 0, 1593, "12 (end)", 269, 21, 0, 300), ncol=5, byrow=TRUE)
colnames(none) <- c("Iteration", "Ride-hail", "Pooled Ride-hail", "Ride-hail Transit", "Total")
nonetab <- as.tibble(none)
kable(nonetab, caption = 'Loss of Ride-hailing Trips in the None Mode Choice Model', booktabs = TRUE) %>%
kable_styling(latex_options="scale_down")
```
### All Mode Choice Model {#type3}
The *All* BEAM mode choice model uses the same mode choice utility function as ActivitySim and has all modal alternatives available. This adjusted model structure helps us understand why we obtained much higher ridership than with the *None* Model. Figure \@ref(fig:piechart) shows from which modes the model assigned agents who switched to ride-hailing came from; in other words, its a comparison of the the trips with ride-hailing after the final iteration and their original modes in the input plans. Interestingly enough, the *All* model assigns the majority of agents who select ride-hail to a car type mode. We define car type modes as either a car, a high occupancy vehicle with two passengers (HOV2), or a high occupancy vehicle with three or more passengers (HOV3).
```{r piechart, echo = FALSE, warning = FALSE, message = FALSE, fig.cap = "Original mode choices of agents who were assigned to a ride-hail type mode.", fig.scap="Original mode choices of agents who were assigned to ride-hail.", out.extra='', fig.align='center'}
tar_read(piechart)
#planshifts <- tar_read(planshifts)
#planshifts
#ggsave("planshifts.png", width = 12, height = 6)
#planshifts_facet <- tar_read(planshifts_facet)
#planshifts_facet
#ggsave("planshifts_facet.png", width = 12, height = 6)
# below is for wRH-All-All scenario
```
We offer some factors as to why the *All* model assigns so many car users to ride-hailing modes. The first is the array of utility parameters boosted the ride-hailing utility, making ride-hailing options attractive alternatives. Figure \@ref(fig:sankey) provides sufficient evidence for this claim. Figure \@ref(fig:sankey) shows a Sankey diagram of all modal decisions at the start of each iteration for those agents in the AsimBeamAll:PPL scenario who the model assigned the ride-hail mode by the end of the 12th iteration [@ggalluvial]. In other words, it shows from what modes the final ride-hail modes came from. For example, of the approximate 6,000 trips with a ride-hail mode assigned after iteration 12, on those same trips in iteration 1 more than 2,000 of them began as car trips, in iteration 5 about 2,000 of them began as car trips, and in iteration 11 less than 1,000 of them began as car trips. Similar logic can be applied to other iterations and to other modes. In the figure's key, the mode "Cleared Mode" describes those modes that were cleared and reset at the beginning of each iteration. Notice how many of the Car, HOV2, HOV2 Passenger, HOV3, and HOV3 Passenger modes shift into the "Cleared Mode" category each iteration. Also notice in the subsequent iteration how many of those "Cleared Mode" choices shift to ride-hailing modes. A shift from the "Cleared Mode" choice to ride-hail represents those agents from whom the model assigned a mode based on the utility value.
\begin{sidewaysfigure}
\centering
\includegraphics[width = 1.05\paperwidth]{planshifts.png}
\caption[Selection process of agents who the model assigned to ride-hail.]{The modal selections at each iteration start of those agents who were assigned a ride-hailing mode at the end of iteration 12 (AsimBeamAll:PPL Scenario).}
\label{fig:sankey}
\end{sidewaysfigure}
Figure \@ref(fig:sankey2) further suggests that the *All* model assigns many agents to use ride-hailing modes because the utility parameters represent them as attractive alternatives [@ggalluvial]. Figure \@ref(fig:sankey2) displays the total number of ride-hailing trips at the end of each iteration as well as which modes were used on those same trips at the beginning of the iteration for the AsimBeamAll:PPL scenario. Notice how for the majority of iterations, a substantial share of agents are assigned from the "Cleared Mode" category to a ride-hailing category instead. Looking closer at this figure also leads us to believe that many ride-hailing trips are not held constant across each iteration. In other words, many ride-hailing trips at the end of one iteration are not the same ride-hailing trips at the end of the next iteration. Although the number of total ride-hailing increases slightly across each iteration, the model assigns a large portion of new individuals to use ride-hail each iteration. This suggests that the model may not be learning which users are "best" for ride-hailing, at least not within only 12 iterations.
\begin{sidewaysfigure}
\centering
\includegraphics[width = 1.05\paperwidth]{planshifts_facet.png}
\caption[Trips that switch to ride-hail by iteration.]{The modal selections at each iteration start of the agents who were assigned a ride-hailing mode at each corresponding iteration end (AsimBeamAll:PPL Scenario).}
\label{fig:sankey2}
\end{sidewaysfigure}
The last deduction from Figure \@ref(fig:sankey2) that we make is from the portion of trips that originate from car and walk trips. The *All* model is changing car and walk type trips directly into ride-hailing trips without first entering them into the "Cleared Mode" stage. Although we do not fully understand why this occurs, we do have a few hypotheses. One suggestion is that a slight error may exist in the internal mode choice program structure that may have allowed some car and walking trips to undergo mode choice even when they were not supposed to. Another hypothesis is that BEAM’s trip based mode choice structure forced some car users to switch modes when a car pathway was not deemed feasible. If this occurred, BEAM's complex car tracking algorithm would have left the car at the previous location, making it unusable on future trips. In other words, the model would have "lost" the agent's car. Overall, the increase in ridership in the *All* model can be explained by ride-hailing being an attractive alternative along with a possible error with the internal mode choice structure or the building of feasible car routes.
### Ridehail Mode Choice Model {#type2}
Finally, the way the *RideHail* BEAM mode choice model was constructed explains why ridership and wait times were high. The *RideHail* model only allowed the assignment of walk and transit users to ride-hailing modes; car-type modes remained locked across each iteration. Whenever a ride-hailing path could be built, the *RideHail* model automatically provided all walk modes the option to choose ride-hail or ride-hail pooled and all transit modes the option to choose ride-hail transit. Although it made logical sense to lock all car-type modes (for reasons described in Section \@ref(type3)), by providing ride-hail options only to walk and transit users, ridership increased even more than in the *All* scenario. The increase in ridership occurred because 1) BEAM's adjusted code forced ride-hail to be an option in almost all cases, and 2) in most cases ride-hail was calculated to be more attractive than walk or transit modes.
Table \@ref(tab:timeutil) provides evidence in ride-hail being an attractive mode choice alternative. Table \@ref(tab:timeutil) displays the ride-hail time utilization for each of the scenarios performed. The same ride-hail fleet was used in each of the nine scenario, composed of 952 ride-hailing driver shifts. Ride-hail time utilization was calculated as the sum of all the driver shift times divided by the sum of all passenger occupied ride-hailing travel time. The AsimRideHail scenario had the lowest ride-hail time utilization, at only 4.0%. Interestingly, the *All* type scenarios ranged from 53.1% to 62.5% ride-hail time utilization. This explains the higher wait times shown in Figure \@ref(fig:waits). Finally, by analyzing the ride-hail time utilization for the *RideHail* scenarios, we fully understand how attractive ride-hail was. With 70.3% to 73.8% of ride-hail time utilization present for the *RideHail* scenarios, we see three fourths of each driver's shift was used to transport passengers. This explains the attractiveness of the choice, the extreme increase in ridership, and also the increased wait time for the *RideHail* type scenarios. Lastly, Table \@ref(tab:timeutil) suggests that ride-hailing ridership and wait times are a function of model structure, as meaningful differences in utilization exist between model structures.
```{r timeutil, echo=FALSE}
time_utilization <- tar_read(ridehail_utilization) %>% select(2,6) %>% rename("Ride-hail Time Utilization" = RideHailTimeUtilization, "Scenario Name" = ScenarioName)
kable(time_utilization, caption = 'Percent Ride-hail Time Utilization by Mode Choice Combination Scenario', caption.short = "Percent Ride-hail Time Utilization", booktabs = TRUE)
#kable_styling(latex_options="scale_down")
```
```{r personutil, echo=FALSE}
#person_utilization <- tar_read(rh_utilization) %>% select(1,6)
#kable(person_utilization, caption = 'Ride-hail (person / hour) utilization by mode choice combination scenario.', booktabs = TRUE) %>%
# kable_styling(latex_options="scale_down")
```
## Summary
As seen by the explanation of the structure of the *None*, *All*, and *RideHail* type scenarios, how BEAM's different mode choice structures were programmed affected ride-hailing ridership and wait time. Fortunately, Section \@ref(discussion) describes the deeper meaning behind the effect different mode choice structure have on ride-hailing ridership and level of service forecasts.