SEMCOG
E8Plus Commercial Vehicle Model
User Guide
Prepared by RSG, Inc.
In cooperation with David
Kriger Consultants, Inc.
For SEMCOG
2023-10-17
1 Introduction
This user guide is a deliverable for the SEMCOG Commercial Vehicle Model project (“the project”). The user guide describes the commercial vehicle model (“the CV model”) that has been developed by RSG with David Kriger Consultants in collaboration with SEMCOG, the Metropolitan Planning Organization for Southeast Michigan.
The user guide documents the software structure of the CV model including its integration into the SEMCOG E8Plus model, SEMCOG’s regional travel demand model (“the E8Plus model”), the CV model’s inputs, scripts, outputs, and other files, and describes how to install and run the CV model and how to make changes to CV model inputs to create alternative scenarios.
The user guide is one of several written deliverables produced during this project. A key accompanying document is the CV model documentation which describes the CV model and its development in more detail than the overview provided in this user guide. The CV model documentation includes sections on data development and how to make more substantial changes to the CV model than are required during typical usage (such as updating the TAZ layer or altering the model time periods).
The following sections of the user guide cover:
- A glossary of terms used in this user guide, the CV model documentation, and reporting from the CV model
- The system architecture which describes the high level model entities, and how they interface with each other
- The CV model’s software structure, including its integration with the E8Plus model
- The CV model’s input files
- The CV model’s output files
- How to run the CV model from the E8Plus model and separately
- How to develop and run alternative scenarios
2 Glossary
This section of the user guide provides a glossary of terms used in this user guide, the CV model documentation, and reporting from the CV model. While the user guide and other documentation assume a working knowledge of travel demand modeling, some of the terminology used in the documentation and reporting is specific to this CV model or to advanced disaggregate travel demand modeling. As such, the glossary is not an exhaustive listing of all technical terms but does attempt to provide definitions for most of the key terms used.
2.1 CV Model Component Terms
The CV model is comprised of three Commercial Vehicle Model Components which are described in the System Architecture section of this user guide:
Firm Synthesis Model which develops a list of business establishment locations and processes zonal land use data used to generate truck trip demand in later steps of the CV model.
Long-Distance Truck Model which estimates long-distance freight truck travel to and from the region, as well as external to external truck travel through the region.
Commercial Vehicle Touring Model (CVTM), which estimates demand for local deliveries and the provision of services by non-freight carrying trucks. The tours and trips simulated to serve this demand, when combined with the travel from the long-distance truck model, means that the CV model simulates all truck movements within, to, from, and through the region.
2.2 CV Modeling Terms
There are many terms used to describe aspects of the commercial vehicle travel that is represented by the model. Several are defined below:
Business Establishment: A specific business location in the CV model region, some of which own or operate commercial vehicles to provide services to, goods deliveries to, or pickups from other businesses and households. In the CV model, the long-distance model is an aggregate model and does not simulate trips from individual business establishments. The CVTM is a disaggregate model representing the operations of individual business establishments in terms of the vehicles that they operate, however the destination stops that the vehicles make are simulated at the TAZ level and not to specific businesses or households.
Commercial Vehicle: A vehicle (auto, SUV, pickup truck, or any single-unit or multi-unit heavy truck) operated for commercial purposes. This means that the vehicle is typically driven during the workday for commercial purposes, as opposed to occasional use of private vehicles for a business related trip.
Vehicle Class: The commercial vehicles simulated in the model are separated into three classes: Light which corresponds to auto, SUVs, and pick-up trucks or other small trucks and vans with single rear wheels, equivalent to FHWA class 2 and 3 vehicles. Medium which corresponds to single unit single axles trucks with double rear wheels, and single-unit trucks with more than two axles, but does not include passenger vehicles such as transit and school buses. It covers FHWA class 5, 6, and 7 (class 4 vehicles are buses). Heavy, which corresponds to all milti-unit tractor trailer combination trucks, equivalent to FHWA class 8 to 13.
Stop: A visit to a destination location made by a commercial vehicle in the CVTM. Stops can be a scheduled stop, for either a service call or a goods delivery or pickup activity, or the stop at the end of a tour, or they can be a non-scheduled stop, for activities such as drivers needs (meals or rest breaks), or a vehicle maintenance or vehicle refueling need. Stops are described with characteristics such as stop purpose (the activity that takes place at the stop), duration (how long the stop takes from arriving at the stop to departing for the next stop), and location (the TAZ in which the stop takes place). In some CVM filenames or model object names intermediate stop is used, which has the same meaning as non-scheduled stop.
Trip: A trip is a movement connecting two stops together in the CVTM. The format of the output from the CVTM describes the trip’s purpose as the same as the activity at the destination stop at the end of the trip. As well as being connected to the characteristics of the departure stop, trips are also described with characteristics such as start and end locations (origin TAZ and destination TAZ), travel time and distance (from skims for the origin to destination TAZ movement), departure time (in minutes after midnight) from the origin TAZ and arrival time (in minutes after midnight) at the destination TAZ, information about the tour that they are part of, and information about the vehicle that makes the trip. In the long-distance model, which is trip-based, the information about trips is more aggregate, with the model outputting number of trips for each origin and destination TAZ pair by time period and vehicle class.
Tour: A tour is a sequence of trips made by a vehicle that (in most cases) starts and ends in the same location, for example from the vehicle’s base (e.g., the business establishment that owns it) to a sequence of service call locations and finally returning to the vehicle’s base. In some cases tours may start and end at different locations, such as starting at the vehicle’s base and ending at an alternative overnight parking location. Trips by each vehicle are grouped into tours in the CVTM, but the long-distance model is a trip-based model and does not create truck tours.
Minutes After Midnight: Time of day in the CVTM are described in terms of “minutes-after-midnight” (MAM), which are the number of minutes from 0 at 12:00am to 1439 at 11:59pm. The CV model aggregates from MAM to the E8Plus model’s five time periods for the purposes of selecting which skimmed travel times and distances to use and when summarizing trips into trip tables. The long-distance model uses the E8Plus model’s five time periods and does not use detailed minute by minute timing.
Long-distance: The long-distance model simulates what are referred to as long-distance trips (as opposed to the short-distance trips simulated by the CVTM). There is no specific distance delineation used in this model for what constitutes as long-distance trip, but there is a spatial definition, based of the types of trips that long-distance model creates: Long-distance trips have at least one trip end outside of the SEMCOG region and also outside of the buffer region. External to internal and internal to external long-distance trips include one trip end inside the SEMCOG region. External to external long-distance trips traverse the SEMCOG region (from an external station to an external station) but do not include any buffer region to buffer region trips.
2.3 Spatial Terms
The CV model simulates truck and light commercial vehicle trips in the SEMCOG region. To do that it includes a spatial system of TAZs and different model components simulate trips to, from, and between different groups of TAZs. This section provides definitions for several terms that describe spatial aspects of the model. A more detailed discussion of the spatial system used in the model is contained in the model design section of the CV Model Documentation.
2.3.1 CV Model TAZ System
The maps and table below display the TAZ system contained in the TAZ_System.csv input file and summarize how each component of the CV model uses the TAZ system. Parts of the CV model use an expanded TAZ system, including the TAZs used by the rest of the E8Plus model and adding additional “buffer” TAZs from the Michigan DOT Statewide Model.
SEMCOG model region: The first map shows the extent of the SEMCOG model region, which is comprised of seven Michigan counties (with Wayne County shown split into the City of Detroit and the rest of Wayne County, labelled “Other Wayne”). This is the internal model region used by the passenger travel portion of the E8Plus model and is also used as the internal model region for the CV model’s output trip tables.
SEMCOG external stations: The first map also shows the external stations to the SEMCOG model region around the outer boundary of the SEMCOG model region. Major externals (e.g., interstate highways, other major routes, and international crossings) are labelled. The external stations are the same as those used by the passenger travel portion of the E8Plus model and are also used as the external stations for the CV model’s output trip tables.
Buffer region: The second map shows the SEMCOG model region surrounded by a set of buffer TAZs that comprise complete counties in Ohio and Michigan that either immediately border at least of the SEMCOG region’s seven counties or form part of a metropolitan area that borders a SEMCOG county (so, for example, Clinton and Easton counties and included along with Ingham as they cover the Lansing area.) Additionally, two bordering Canadian Census Subdivisions are included in the buffer region. The TAZs in the buffer region were extracted from the Michigan DOT Statewide Model TAZ layer. The buffer TAZs are treated as part of the internal model region during the simulation of truck tours by the Commercial Vehicle Touring Model, before trips are clipped at the external stations for aggregation into trip tables.
Figure 1: Maps of SEMCOG Internal TAZs and External Stations
Figure 2: Maps of
SEMCOG Internal TAZs and Buffer TAZs
The table below explains how the CV model components use the TAZ system to represent different elements of commercial vehicle travel.
Table 1: CV Model Components and the TAZ System
| ModelComponent | Description |
|---|---|
| Firm Synthesis | Develops list of firms in all SEMCOG region and buffer region TAZs |
| Long Distance | Long-distance trips with at least one trip end outside of the SEMCOG region and also outside of the buffer region. External to internal and internal to external trips include one trip end inside the SEMCOG region. External to external trips traverse the SEMCOG region but do not include any buffer region to buffer region trips. Trips are generated for each SEMCOG region TAZ and at each external station |
| Commercial Vehicle Touring | Tours and trips are simulated for the full SEMCOG region and buffer region, which is treated as one larger internal model region. Trips to any point outside that larger region are not simulated by this step. Trips that travel between the buffer and the SEMCOG region are tagged with the external station that they would pass through (or the two external stations in the case of trips that travel between two parts of the buffer traversing the SEMCOG region.) The model simulates trips entirely within the buffer that do not traverse the SEMCOG region as well, however these trips are not processed in the following model steps. |
| Trip Tables | Trips tables from the CV model are created for the trips simulated by earlier model steps that are between SEMCOG region TAZs and for the portions of trips between SEMCOG region TAZs and external stations or between external stations. Any trips entirely within the buffer region that do not traverse the SEMCOG region are excluded from the trip tables. |
3 System Architecture
3.1 Commercial Vehicle Model Components
This section of the user guide introduces the CV model’s system architecture and describes its high-level model entities and how they interface with each other:
- The Firm Synthesis Model, which develops a list of business establishment locations and processes zonal land use data used to generate truck trip demand in later steps of the CV model.
- The Long-Distance Truck Model, which estimates long-distance freight truck travel to and from the region, as well as external to external truck travel through the region.
- The Commercial Vehicle Touring Model (CVTM), which estimates demand for local deliveries and the provision of services by non-freight carrying trucks. The tours and trips simulated to serve this demand, when combined with the travel from the long-distance truck model, means that the CV model simulates all truck movements within, to, from, and through the region.
The figure below shows the overall regional travel demand model system used by SEMCOG, called the E8Plus model. The E8Plus model contains a passenger travel demand model used to estimate personal travel by auto and other modes and a highway network loaded with vehicle trips. The components of the CV model are included within the red dashed line. The CV model’s first component is the firm synthesis model (orange box in upper left of the CV model). The firm synthesis model creates a list of business establishments within the model region and a database of zonal land use (blue box labelled “Synthetic Firms…”) that are inputs to both the long-distance truck model and the CVTM. The firm synthesis process that creates a list of business establishments is similar to the household synthesis process in the passenger model.
The long-distance truck model (orange box in upper right of the CV model) estimates long-distance truck trip demand as a function of business employment and the presence and size of high-intensity freight activity locations in the region. The output from this model component is a set of long-distance truck trips between external stations and zones within the region as well external to external trips (blue box labelled “Long-Distance Trips”).
The regional CVTM develops the demand for the remainder of the travel of trucks for local delivery and to provide services to businesses and households by vehicle type (light, medium, and heavy). The CVTM groups truck stops into tours using a series of models that cluster, sequence, and time truck trips and stops based on observed truck travel behavior to create a range of realistic truck touring patterns. The disaggregate simulation segments stops into logical sets when constructing tours, including by vehicle type, industry of the business providing the delivery/offering the service, and trip purpose (local delivery, service, and stops for other purposes such as driver breaks and vehicle fueling).
Figure 3: Model System - SEMCOG Model
3.2 Commercial Vehicle Model Integration
The CV model exports its outputs as trip tables. The trip list from the CVTM with trip start and end locations and trip timing information is aggregated into zone to zone trips by time period and vehicle type and combined with zone to zone trips by time period and vehicle type from the long-distance model (blue box in the figure above). The trip tables can be assigned to the E8Plus model road network (red box) along with the SOV, HOV2, and HOV 3+ trip tables from the passenger model. The figure above also illustrates how the CV model is integrated within the E8Plus model’s feedback loop so the CV model is sensitive to congestion effects.
On the first iteration during a model system run, all components of the CV model are run starting with synthesizing the list of business establishments, then estimating long-distance truck traffic and local demand for deliveries and services. The CV model then uses the initial travel time skims from the E8Plus model as one input during the simulation of tours by CVTM. The trip tables produced during the first iteration are assigned to the network along with the comparable set of first iteration trip tables produced by the passenger model. The CV model steps run during the first iteration are shown in the figure below.
Figure 4: CV Model Components Run During SEMCOG E8Plus Model First Iteration
During subsequent iterations of the model system run (typically the second through fifth iterations, the CV model is run again. In these later iterations, only some components of the CV model are included. The firm synthesis model and long-distance truck model are not sensitive to congested travel times within the region and therefore are not repeated.
Almost all of the components of the CVTM are a function of travel time and therefore are sensitive to congested travel times and must be repeated during each iteration. The updated CVTM trips produced during these later iterations are combined with the long-distance truck trips to produce updated trip tables that are assigned to the network along with the comparable set of updated trip tables produced by the passenger model.
The figure below shows the CV model steps run during the middle iterations (usually second through fourth). In the final iteration (usually the fifth), the run process is the same as in the middle iterations but then the CV model also runs an additional reporting step that includes producing the dashboard (see the second figure below).
Figure 5: CV Model Components Run During SEMCOG E8Plus Model Middle Iterations
Figure 6: CV Model Components Run During SEMCOG E8Plus Model Final Iteration
3.3 Firm Synthesis Model
The CV model’s first component is the firm synthesis model. The firm synthesis model creates two databases used in subsequent steps of the CV model:
- A list of individual business establishments within the CV model region. Business establishments are located spatially within the model’s zone system. Since the CV model region covers the internal SEMCOG model zones and a buffer of additional zones around the boundary of the SECMOG model zones, the list includes business establishments for this larger set of zones. The business establishments are segmented into the 18 SEMCOG employment categories (which are defined as groupings of North American Industrial Classification System (NAICS) codes) and each establishment’s size is described in terms of its number of employees. In the CVTM, the business establishments represent the locations that can own and operate commercial vehicles and provide local deliveries and services to households and employment locations in the model region. In this case, the industry determines the types of services or deliveries offered and the industry and number of employees affect the quantity of activity. The location of those activities is determined based on TAZ-level employment by type and the number of households as noted below.
- The number of households and employment by type at the TAZ level. The TAZ-level land use data are used in the CVTM’s stop generation models to estimate the number of truck stops in each TAZ for local service and goods delivery purposes, and in the long-distance truck model’s internal trip generation model to estimate the number of long-distance truck trips to and from each TAZ.
The firm synthesis model contains two steps, each one producing one of the two databases:
TAZ Land Use. This component combines input employment by category and households for the scenario or forecast year into a database.
Enumerate and Scale Employees. An input describing the base year set of business establishments in the model region by location (TAZ), industry (18 employment categories that are combinations of one or more 2 or 3-digit NAICS code) and firm size (number of employees) is enumerated into a list of individual business establishments. The list is scaled to match TAZ-level employment data by employment category for the scenario and forecast years being run. The TAZ-level employment data for the base year is the 2015 data developed by SEMCOG for the SEMCOG region, combined with data for 2015 from the MDOT Statewide Model for the buffer zone around the SEMCOG model region. For future forecasts/scenarios, the TAZ-level employment is also input from the E8Plus model and the MDOT Statewide Model. Currently, future scenarios have been created using the socioeconomic data for 2020, 2025, 2030, 2035, 2040, and 2045. The scaling includes synthesizing new firms in TAZs where new development is included in the TAZ-level employment data, updating the list of businesses to include some new firms and some growth in existing firms where there is growth in already developed TAZs, and reducing the size of or removing firms where there is a forecasted decline in employment in a TAZ.
The figure below shows the firm synthesis model’s inputs, steps, and outputs.
Figure 7: Firm Synthesis Model Structure
3.4 Long-Distance Truck Model
The long-distance truck model estimates long-distance freight truck travel and its allocation to TAZs inside the SEMCOG region and external stations around the boundary of the SEMCOG model region. It also estimates cross region freight truck movements (external-to-external truck trips) that do not stop at businesses or other freight activity locations in the SEMCOG region.
The external-to-external trips in the long-distance model are based on the expanded ATRI GPS data. During the processing of the ATRI GPS dataset, stops at truck rest stops were removed. Therefore, some of the trips that are included in the long-distance truck model as external-to-external truck trips might in reality have had rest stops within the SEMCOG region.
The long-distance truck model does not cover truck travel between the SEMCOG region and the buffer region around the SEMCOG region, nor does it cover external-to-external travel from one part of the buffer to another part of the buffer that traverses the SEMCOG region. All trips generated by the long-distance truck model have at least one trip end outside of the CV model region, which means that the trip starts or ends beyond the buffer region.
The output from the long-distance truck model is a list of long-distance trips to and from the SEMCOG region. Some of the origins and destinations of long-distance freight truck trips within the region are TAZs that include warehouse and distribution centers, or intermodal facilities, based on the high-intensity freight location database that is an input to the long-distance truck model. The long-distance truck model identifies the external stations where internal-to-external trips leave the region and external-to-internal trips enter the SEMCOG region.
Another output from the long-distance truck is an external-to-external truck trip list, which is combined with the internal-to-external trips and external-to-internal trips to produce a complete list of all truck trips to, from, and through the SEMCOG region that have at least one trip end outside of the SEMCOG region and beyond the buffer region.
The statistical models applied in the long-distance truck model were estimated using two data sources of data:
Explanatory variables are based on land use data within the region (analogous to the outputs from the firm synthesis model) describing generators and attractors of truck trips such as the amount of employment by industry and at high-intensity freight locations such as DTW airport. Additional variables are based on accessibility such as proximity to the interstate network and specific major routes into and out of the region such as interstate highways to the southwest and international crossings.
Dependent variables are based on observed truck movement data (processed and expanded ATRI truck GPS data describing long-distance truck into and out of the region).
The model components of the long-distance truck model take the following form:
Internal to External and External to Internal Trip Ends: Internal trip generation equations estimated at the TAZ level using a function of land use variables, such as employment by industry, number and size of business establishments by industry, the size or capacity of high-intensity freight locations in the TAZ, and accessibility and proximity variables describing variables such as the distance to the interstate highway network.
Internal to External and External to Internal External Trip Ends: External trip generation equations estimated for each external station as a function of commodity flows between external regions of North America and the SEMCOG region.
Trip Distribution: internal trip ends and external trip end for external to internal and internal to external long-distance movements are paired into trips based on a gravity model calibrated using the observed trip length frequencies from the expanded ATRI truck GPS data.
External to External Trips: the observed patterns of external-to-external trips based on the expanded ATRI truck GPS data were used to develop a base year external-to-external truck trip table. This trip table is grown to the horizon year of interest based on commodity flow forecasts of growth the specific flows that lead to these through trips.
External Trips by Time of Day: the daily trips (for internal to external, external to internal, and external to external trips) are allocated to vehicles types and time of day based on the observed time of day distribution of long-distance trips.
The figure below shows the long-distance truck model’s inputs, steps, and outputs.
Figure 8: Long-Distance Model Structure
3.5 Commercial Vehicle Touring Model
The CVTM develops the demand for stops by the remainder of the travel of light, medium, and heavy trucks which is for within-CV model region movements including trucks travelling for non-freight purposes. The within-region movements include providing services and local goods delivery to households and businesses.
Since the CV model region includes the buffer region around the SEMCOG region, from the perspective of just the SEMCOG region some trips developed by the CVTM might be external-to-internal, internal-to-external, external-to-external, or never enter the SEMCOG region at all, as well as internal-to-internal trips.
The CVTM then simulates the truck tours that serves the demand for truck stops. The CVTM groups truck stops into tours using a series of models that cluster, sequence, and time truck tours and stops based on truck travel behavior observed in the commercial vehicle survey and truck GPS data to create a range of realistic truck touring patterns.
Each step in the CVTM is described briefly here:
Establishment Type: The establishment type model tags each synthetic establishment from the firm synthesis model with an aggregated industry type to match with observed distributions of establishment behavior constructed from the commercial vehicle survey data. The model then applies a Monte Carlo simulation method to draw from the observed distributions of establishments by industry type to add a label that indicates whether the establishment makes goods deliveries or pickups, operates commercial vehicles to provide services, or does both.
Stop Generation: The stop generation model predicts one day’s worth of scheduled stops made by trucks operated by each establishment in each destination TAZ. The model uses a count formulation, which predicts positive integer values for the frequency of an event. Scheduled stops are grouped into two market segments: goods delivery or pickup stops and service stops.
Vehicle Assignment: For each stop, the vehicle assignment model assigns a commercial vehicle type. The vehicle types are light (i.e., car, van, and pickups with two axles and single rear wheels), medium (single-unit trucks with two axles and double rear wheels or more than two axles), and heavy (multi-unit trucks). The model is formulated as a multinomial logit model (estimated using the commercial vehicle survey data) and predicts vehicle type as a function of the establishment’s industry type, distance between establishment and stops to be served, and the stop’s purpose—goods delivery or pickup, or providing services.
Expected Stop Duration: The expected stop duration model is applied to scheduled stops generated by the stop generation model. For each stop, the expected stop duration is drawn from a smoothed, empirical distribution of observed stop distributions for each stop type and vehicle type.
Tours and Routing
- Stop Clustering: For each establishment, the stop clustering model groups scheduled goods delivery or pickup stops and service stops into feasible commercial vehicle tours, based on spatial proximity, vehicle type, total travel time, and expected stop duration.
- Tour type: a tour type (whether start and end locations are the same or not and whether the tour starts and ends at the firm’s base) is selected for each tour and appropriate start and end stops are added before and after the scheduled stops.
- Routing Sequence: Given a set of scheduled stops and their locations on a tour, and the establishment location, the model uses a Travelling Salesman Problem algorithm to determine the sequence of stops on the tour.
Arrival Time at First Stop: For each tour, the arrival time at the first stop on the tour is predicted using a multinomial logit model. Morning and afternoon arrival times from just before the AM peak to the end of the PM peak are modeled at half-hour intervals, while evening and night arrival times are modeled at one-hour intervals.
Non-Scheduled Stop Choice: This component predicts any non-scheduled stops between scheduled stops on a tour. The model simulates whether the driver makes one or more non-scheduled stops prior to each scheduled goods delivery or pickup stop or service stop, or prior to returning to the establishment to complete the tour. Purposes for non-scheduled stops include breaks/meals, vehicle service/refueling, and personal business/other.
Non-Scheduled Stop Destination: For each non-scheduled stop on a tour, this model predicts a destination TAZ. Specifically, for each non-scheduled stop, the model selects a set of eligible TAZs based on attraction factor(s), such as retail employment for break/meal stops, and an impedance factor accounting for travel distances or times. This model uses a “rubber banding” method that considers the travel distance or time from the current stop to each alternative destination and from each alternative destination to the next scheduled stop or returning to the establishment. This method minimizes distance or time deviations from direct paths between scheduled stop locations.
The figure below shows the CVTM’s inputs, steps, and outputs.
Figure 9: Commercial Vehicle Touring Model Structure
4 Software Design
4.1 SEMCOG E8Plus - CVM Integrated Model Software Architectrure
The overall model system, the SEMCOG E8Plus model with CV model, is
an integrated model that is controlled by and primarily runs in the
TransCAD transportation planning software platform produced by Caliper
(https://www.caliper.com/tcovu.htm). The E8Plus model
uses TransCAD version 8, build 22365. The E8Plus model has
a graphical user interface (GUI) programmed in TransCAD’s GISDK
scripting language that is used to manage scenarios, start model runs,
produce output reports, and run other modeling utilities.
The CV model has been incorporated into the E8Plus model’s file
structure. The CV model is programmed in R, an open source statistical
programming platform available from The Comprehensive R Archive Network
(https://cran.r-project.org/). The CV model uses
R version 4.1.2 for Windows (64 bit) (https://cran.r-project.org/bin/windows/base/old/4.1.2/R-4.1.2-win.exe).
This is at the time of the release of this version of the CVM model the
current release of R for Windows and was published on 11/01/2021.
This section of the user guide describes the structure of the overall model system focusing on the approach to integrate the CV model. It covers:
Overall model system file structure
Software linkages between the TransCAD portion of the E8Plus model and the CV model
GUI modifications to include the CV model
GISDK modifications to include the CV model
Scenario input and output modifications
4.1.1 Overall Model System File Structure
The E8Plus model has been provided to SEMCOG in two parts:
A release zip file provided via GitHub that contains the model code and common model inputs files
A zip file of scenario inputs that was provided separately due to file size constraints for GitHub releases and data protection requirements.
The installation process is described later in the user guide in the section on Running the Model. Once installed, the final folder structure is somewhat flexible in terms of the location of the root model directory containing the model code and the relative location of the scenarios directory, which can be adjusted using the E8Plus model’s GUI. The structure described here is the file structure that has been used for model development and application testing during the CV model project.
The E8Plus model is included within a release directory, named
semcog_cvtm_vX.X.X where the X.X.X is a
version number. Inside that directory the E8Plus model has the file
structure shown in the figure below. The model is divided into four
primary directories:
The
AddInfolder which contains the TransCAD add-in files including the GISDK scripts and the compiled TransCAD GUI file.The
CVMfolder which contains the CV model’s scripts, parameters, inputs that do not vary across scenarios, and additional R function libraries.The
docsfolder which contains this user guide and other documentationThe
Model Runsfolder which contains folders for each scenario and within those folders the scenario specific inputs and outputs.
The contents of the CVM folder are described in the section on CV Model Software Architecture. The modifications to the remaining portions of the E8Plus model’s GUI, scripts, inputs and outputs are described in more detail below.
Figure 10: SEMCOG E8Plus Model File Structure
semcog_cvtm_vX.X.X
* AddIn TransCAD add-in directory
+ bmp TransCAD GUI images
+ rsc TransCAD GISDK scripts
* CVM CV Model directory
* docs Documentation directory
* Model Runs Scenarios directory
+ 2015_KA15 Base year scenario directory
+ 2020_KC20 Future (2020) scenario directory
+ 2025_KC25 Future (2025) scenario directory
+ 2030_KC30 Future (2030) scenario directory
+ 2035_KC35 Future (2035) scenario directory
+ 2040_KC40 Future (2040) scenario directory
+ 2045_KC45 Future (2045) scenario directory
* README.md Model repository README file
4.1.2 Software Linkages with the CV Model
The CV model is integrated into the E8Plus model file structure as described above. While the E8Plus model runs in TransCAD, the CV model runs in R. In order for the CV model to be run successfully, TransCAD must save model outputs required as inputs by the CV model into a file format that is readable by R, it must start the R application, and it must pass information to R about which CV model component to run, which scenario to run, and where to read inputs from and write outputs to. Once the CV model component has completed, R must close and pass back control to TransCAD so that the next part of the E8Plus model can be run. If there is a problem with the run of the CV model component, the CV model should communicate an error code back to TransCAD to stop the overall model run.
This general process takes place in several steps (additional details are described in the subsequent sections):
Scenario Configuration The E8Plus model’s TransCAD GUI is used to run the model. The scenario configuration in the GUI includes scenario descriptors such as the scenario name, file paths for all inputs and outputs, and, importantly for integration with the CV model, the file path of the
run_semcog_cvtm.batbatch file.Skim Matrices Saved in .omx Format TransCAD matrices are typically saved in .mtx format, which is a proprietary file format that R is unable to read from. However, both TransCAD and R can write and read matrices using the .omx open matrix format (https://github.com/osPlanning/omx). Code has been added to the SEMCOG Highway Skimming step to export a copy of the highway skim matrices to .omx format files.
Trip Tables Saved in .omx Format During a complete run of the E8Plus model, the CV model has been added between the Mode Choice component and the Highway Assignment component. A step in the Mode Choice component, SEMCOG Vehicle Time of Day, has been modified to save the time period and daily vehicle trip tables, which at this point do not include commercial vehicle trips, in .omx format so that the CV trip tables can be added directly by the CV model.
CV Component Macros A series of GISDK macros have been added to the E8Plus model, one for each of the CV model steps (CV Firm Synthesis, CV Long Distance Model, CV Touring Model, CV Trip Tables, and CV Dashboard). All the macros use the same approach:
- Write Configuration File Create a text format file called
run_semcog_cvtm.txtcontaining scenario descriptors and input file paths. This temporary file is saved in theCVMCV Model directory during a run and then deleted once the CV model step has completed successfully. - Call R using RunProgram Function Use TransCAD’s RunProgram
function to execute the
run_semcog_cvtm.batbatch file and pass arguments to the batch file describing which CV model component to run. - Wait for CV Model Component Wait for a status message from the batch file and either continue to the next model step if the CV model step completed or stop if there was an error.
- Write Configuration File Create a text format file called
run_semcog_cvtm.bat Batch File The batch file builds a call to run the
RScript.exeapplication, which is the batch version of R (as opposed to RGui which is the interactive version). The batch file contains information about where R is installed on the machine. The batch file calls RScript and asks it to run therun_semcog_cvtm.Rscript along with the arguments passed from TransCAD which are used to select a specific CV model step to run. The batch file includes error handling to communicate any errors back from RScript to TransCAD.run_semcog_cvtm.R and Master.R Scripts The
run_semcog_cvtm.Rscript is a simple script that captures the arguments passed from the batch file and uses R’s source function to run the CV model’s main R script,__Master.R. This script initiates the CV model run and runs the CV model step requested by TransCAD. During the initiation step, R reads inrun_semcog_cvtm.txtand populates a series of variables with the file paths and scenario descriptors that the text file contains. These are used to read the correct scenario specific files, such as travel time skims, and to identify where to save outputs.CV Trip Tables Written into .omx Trip Table Files The CV Trip Tables step of the CV model, which is run during every iteration of the E8Plus model, saves the CV trip tables simulated by the CV model directly into the time of day and daily trip table .omx files that were created by the E8Plus model earlier in the model run.
End of the CV Model Step Once the CV model step has finished successfully, including writing any files to the scenario outputs locations described in the configuration files, then RScript closes, the batch file closes, and the TransCAD macro that called the CV model step ends. Once the sequence of macros that call the CV model steps for this iteration of the E8Plus model have completed, the E8Plus model moves onto the Highway Assignment component.
Highway Assignment Reads .omx Trip Table The Highway Assignment macro used in the E8Plus model has been modified to read the .omx format time of day trip tables. At this stage of the E8Plus model run the trip table files include the SOV, HOV2 and HOV3+ trip tables created by the passenger travel components of the E8Plus model as well as the Light, Medium, and Heavy truck trips tables added by the CV model.
4.1.3 GUI Modifications for Integration
The E8Plus model’s GUI has been updated to include the CV model. The screenshot below shows the modified main GUI, which includes the CV model step, labelled “Commercial Vehicle”, between the “Mode Choice” and “Trip Assignment” steps.
Figure 11: E8Plus Model GUI
As with the other main model component in the E8Plus model, the CV model is comprised of a series of steps. These can be selected in order to run one or more steps individually using the step settings buttons along the left side of the GUI. The all step settings button and the button for the commercial vehicle step both allow access to the CV model’s steps as shown the in following screenshots.
Figure 12: E8Plus Model GUI, All Step Settings
Figure 13: E8Plus Model GUI, Commercial Vehicle Step Settings
The scenario GUI has also been modified to include CV model inputs.
The screenshot shows several of the new inputs, and the complete set of
additional inputs added to DefaultScenario.ini is shown in
the table below.
Figure 14: E8Plus Model GUI, Scenario Inputs
Figure 15: E8Plus Model GUI, New Inputs in DefaultScenario.ini
[Input.Buffer TAZ Data]
Value = SED\TAZSocioEconomicsBuffer.csv
Desc = TAZ data for the buffer region used in the CVM
[Input.Facilities Data]
Value = SED\Facilities.csv
Desc = Freight activity centers facilties used in the CVM
[Input.CVM IEEI Trucks]
Value = SED\CVM_IEEI_Region.csv
Desc = Daily IE and EI Trucks by External Region, used in the CVM
[Input.CVM EE Trucks]
Value = SED\CVM_EE_Region.csv
Desc = Daily EE Trucks by External Region, used in the CVM
[Input.Base Year Firms]
Value = SED\1.Firms.Rdata
Desc = Base Year Firm Synthesis Output Database (Rdata file) used in any non-base scenario in the CVM
Choice = Optional
[Input.CVM Batch File]
Value = run_semcog_cvtm.bat
Desc = Batch file to run Commercial Vehicle Model
The scenario GUI has also been modified to describe CV model outputs.
The screenshot shows several of the new outputs, and the complete set of
additional output added to DefaultScenario.ini is shown in
the table below.
Figure 16: E8Plus Model GUI, Scenario Outputs
Figure 17: E8Plus Model GUI, New Outputs in DefaultScenario.ini
[Output.CVM]
Firm Synthesis Database = CVM\1.Firms.Rdata | List of Firms from Frim Synthesis
Long Distance Database = CVM\2.LongDistanceTrips.Rdata | Long Distance Truck Trips
Commercial Vehicle Tours = CVM\3.CommercialVehicleTrips.Rdata | Commercial Vehicle Tours and Trips
CVM Trip Database = CVM\4.TripTables.Rdata | Commercial Vehicle Trip Database
CVM Trip Generation = CVM\CV_Trip_Generation_Summary.csv | Commercial Vehicle Trip Generation
CVM Dashboard = CVM\ReportDashboard.html | Commercial Vehicle Dashboard
4.1.4 GISDK Modifications for Integration
The E8Plus model’s main GISDK script, SEMCOG.rsc has
been modified during the project to remove the trip based truck model
that formed part of the E7 model and replace it with the CV model. This
section provides an overview of the changes but does not document all
code changes in a line-by-line manner. The changes can be grouped into
four main types:
- Adding the CV model steps to the macro sequence for the model, including indicating which steps run in which system iteration
- Adding a set of macros to call each CV model step
- Adjusting data formats to save and read skim matrices and trip tables in the .omx file format to allow access directly from R
- Removing the code that implemented the trip base truck model
The macro “SEMCOG Model Steps” has been modified as shown in the figure below to include the CV model step as “Commercial Vehicle”, as the sixth step in the model sequence. The step macros that run the CV model have been inserted into the list of macros (e.g, “SEMCOG CV Firm Synthesis” which runs the firm synthesis model). Some other changes have been made to the grouping of macros and their naming as part of the integration of the CV model. The time of day macros were previously grouped with the Trip Assignment macros but have been moved up to be part of the Mode Choice group so that the trip table files are prepared before the CV model run so that the CV model can directly add the truck trips tables. Also, the airport travel distribution model used to be combined with the truck trip distribution in a macro called “SEMCOG Truck and Air Distribution”. The truck related parts of this macro have been removed and the macro is now just “SEMCOG Air Distribution”.
The final modifications to this macro were to code which CV model
steps to run in which model iteration and to make some additional
adjustments to the feedback approach. As described in System Architecture, firm synthesis and
the long-distance model are only run in the first iteration, indicated
with a “0” in the Scen.StepFeedback object. The CVTM and
trip tables steps are run in every iteration, indicated with a “1”, and
the dashboard is only run in the final iteration, indicated with a “2”.
One additional modification has been made to
Scen.StepFeedback. Previously, the only time period
assignments that were run in every iteration of the model were AM peak
and Midday, since those were the skims used to update the trip
distribution and mode choice results during feedback. However, the CV
model uses a blended skim from all of the time periods in each run of
the CVTM and so PM peak assignment was also set to run during every
iteration (there is relatively little congestion in the evening and
nighttime periods and so no significant changes in travel times between
iterations for those two other time periods).
Figure 18: Modifications to the SEMCOG Model Steps Macro (Showing Part of Code)
// Define default macro and step names
Macro "SEMCOG Model Steps" (Scen)
//names of stages, must match names of buttons in dialog box.
Scen.StageNames = {"Initialization",
"Network Skimming",
"Trip Generation",
"Trip Distribution",
"Mode Choice",
"Commercial Vehicle",
"Trip Assignment"}
Scen.StageIndex = {"INI", "GEN", "SKM", "DST", "MOD", "CVM", "ASN", "RPT"}
//Macros to run for each step
Scen.StepMacro = {{"SEMCOG Update Directory", "SEMCOG Highway Process",
"SEMCOG Zonal Walk Percentages", "SEMCOG Stop Access",
"SEMCOG Transit Process", "SEMCOG Build Highway Network",
"SEMCOG Build Transit Network"},
{"SEMCOG Highway Skimming", "SEMCOG PR Access",
"SEMCOG Transit Skims", "SEMCOG Process Transit Skims"},
{"SEMCOG Auto Availability",
"SEMCOG Process SED", "SEMCOG Trip Generation"},
{"SEMCOG Walk Market Segments", "SEMCOG Create DC MDL Files",
"SEMCOG Trip Distribution MC", "SEMCOG Trip Distribution Logsums",
"SEMCOG Destination Choice", "SEMCOG Air Distribution",
"SEMCOG HBU Allocation"},
{"SEMCOG Mode Choice Probabilities",
"SEMCOG Vehicle Time of Day", "SEMCOG Transit Time of Day"},
{"SEMCOG CV Firm Synthesis", "SEMCOG CV Long Distance Model",
"SEMCOG CV Touring Model", "SEMCOG CV Trip Tables",
"SEMCOG CV Dashboard"},
{"SEMCOG AM Highway Assignment", "SEMCOG MD Highway Assignment",
"SEMCOG PM Highway Assignment", "SEMCOG EV Highway Assignment",
"SEMCOG NT Highway Assignment", "SEMCOG Assignment Combine",
"SEMCOG Transit Assignment", "SEMCOG Transit Combine",
"SEMCOG Feedback"}}
(additional code)
//Set feedback loops to run
// 0 = Run only on the first loop
// 1 = Always run
// 2 = Run only on the final loop -- ONLY VALID AFTER DST due to in-loop RMSE check
Scen.StepFeedback = {{0, 0, 0, 0, 1, 1, 1}, //INI
{1, 1, 1, 1}, //SKM
{1, 1, 1, 1}, //GEN
{1, 1, 1, 1, 1, 1, 1, 1}, //DST
{1, 1, 2}, //MOD
{0, 0, 1, 1, 2}, //CVM
{1, 1, 1, 2, 2, 2, 2, 2, 1}} //ASN
(additional code)
The five macros added to call each step of the CV model follow the same structure. The figure below shows the “SEMCOG CV Firm Synthesis” macro as an example.
The macro creates a text configuration file, with the same name and
location as the CVM batch file, run_semcog_cvtm.bat. The
text file is populated with the scenario name, scenario input and output
directories, the current iteration number, and the CV component name.
The macro then adds the paths to several files specifically required by
this CV model component. In the case of the firm synthesis step, the
paths are to the SEMCOG TAZ data (“Raw TAZ Data”), the TAZ data for the
buffer region (“Buffer TAZ Data”), and to the firm synthesis output from
the base scenario (“Base Year Firms”, which is required only for
alternative/future scenario runs).
After saving and closing the text configuration file, the macro uses
TransCAD’s RunProgram function to execute the
run_semcog_cvtm.bat batch file and pass arguments to the
batch file describing which CV model component to run. The arguments are
a series of TRUE/FALSE Booleans indicating whether each of the five CV
model components should be run. In the firm synthesis macro only the
first is set to TRUE, indicating that only the firm synthesis step
should be run. The macro includes error handling to capture status codes
returned by the batch file and halt the model run if the CV model run is
unsuccessful.
Figure 19: SEMCOG CV Firm Synthesis Macro (Complete Macro)
Macro "SEMCOG CV Firm Synthesis" (Args)
shared UT, Scen
// Run Commercial Vehicle Model Firm Synthesis
bat_file = Args.[CVM Batch File]
// Create a file of input and output file information
paths_file = Substitute(bat_file, ".bat", ".txt", )
cv_paths = OpenFile(paths_file, "w")
// Scenario name, description, input and output directories, iteration, and CV component name
WriteLine(cv_paths, Args.Info.Name)
WriteLine(cv_paths, Args.Info.[Input Directory])
WriteLine(cv_paths, Args.Info.[Output Directory])
WriteLine(cv_paths, string(Scen.Feedback.iteration))
WriteLine(cv_paths, "Firm Synthesis")
// Path to the TAZ SE data for SEMCOG and the buffer and base year firms
WriteLine(cv_paths, Args.[Raw TAZ Data])
WriteLine(cv_paths, Args.[Buffer TAZ Data])
WriteLine(cv_paths, Args.[Base Year Firms])
CloseFile(cv_paths)
// Call the batch file with the component switch arguments set to run firm synthesis only
status = null
status = RunProgram(bat_file + " TRUE FALSE FALSE FALSE FALSE", {{"Maximize", "True"}})
if status<>0 then do
ShowMessage ("CVM Launch Failed!")
goto quit
end
ok = 1
return(ok)
quit:
return(0)
endMacro
The following figures show the code from the “SEMCOG CV Long Distance Model”, “SEMCOG CV Touring Model”, and “SEMCOG CV Trip Tables” macros that provide the CV model with step specific file information. For the long-distance model step, the paths to the freight facilities file and inputs describing assumptions for the internal to external/external to internal and external to external model are provided, along with the matrix core name and location of the distance skim matrix used by the long-distance model.
Figure 20: SEMCOG CV Long Distance Model Macro (Partial Code)
Macro "SEMCOG CV Long Distance Model" (Args)
(additional code)
// Path to the facilities file, IEEI and EE Trucks, name of the distance skim core to use, and path to the midday skim
WriteLine(cv_paths, Args.[Facilities Data])
WriteLine(cv_paths, Args.[CVM IEEI Trucks])
WriteLine(cv_paths, Args.[CVM EE Trucks])
WriteLine(cv_paths, "Miles (orig)")
md_omx_file = Substitute(Args.MDHwySkims, ".mtx", ".omx", )
WriteLine(cv_paths, md_omx_file)
(additional code)
For the CVTM step, the macro writes out the matrix core names and locations for the travel time, distance, and toll matrices by time period. As noted in the comments in the macro, information related to toll skims is not used in the CV model unless the CV model is set up to use toll skims.
Figure 21: SEMCOG CV Touring Model Macro (Partial Code)
Macro "SEMCOG CV Touring Model" (Args)
(additional code)
// Names of the time, distance, and toll skim cores to use
//(if no toll, need to set BASE_TOLL_SKIM_AVAILABLE <- FALSE in CV model, any value here will then by ignored)
WriteLine(cv_paths, "Trav_Time")
WriteLine(cv_paths, "Miles (orig)")
WriteLine(cv_paths, "toll")
// Paths to the time period skims files (.OMX format)
nt_omx_file = Substitute(Args.NTHwySkims, ".mtx", ".omx", )
am_omx_file = Substitute(Args.AMHwySkims, ".mtx", ".omx", )
md_omx_file = Substitute(Args.MDHwySkims, ".mtx", ".omx", )
pm_omx_file = Substitute(Args.PMHwySkims, ".mtx", ".omx", )
ev_omx_file = Substitute(Args.EVHwySkims, ".mtx", ".omx", )
WriteLine(cv_paths, nt_omx_file)
WriteLine(cv_paths, am_omx_file)
WriteLine(cv_paths, md_omx_file)
WriteLine(cv_paths, pm_omx_file)
WriteLine(cv_paths, ev_omx_file)
(additional code)
For the trip tables step, the macro writes out the location of the trip tables that the CV model will add the CV trip tables to.
Figure 22: SEMCOG CV Trip Tables Macro (Partial Code)
Macro "SEMCOG CV Trip Tables" (Args)
(additional code)
// Path to the OD trip tables
WriteLine(cv_paths, Args.[OD Trip Tables])
(additional code)
Two macros were modified to save skim matrices and trip tables in .omx format. The figures below show the code in the SEMCOG Highway Skimming macro to save an additional copy of the highway skim files in .omx format, and in the SEMCOG Vehicle Time of Day macro to use .omx format for the time of day and daily vehicle trip tables.
Figure 23: SEMCOG Highway Skimming Macro (Partial Code)
Macro "SEMCOG Highway Skimming" (Args)
(additional code)
NextStep= "Write OMX Skims - " + period
SetStatus(1, NextStep, )
// Write OMX matrices here so thst they incorporate all adjustments
//skim_curs = CreateMatrixCurrencies(skim_m, GetMatrixBaseIndex(skim_m).[0], GetMatrixBaseIndex(skim_m).[1], )
cores = GetMatrixCoreNames(skim_m)
for c in cores do
skim_curs.(c) := nz(skim_curs.(c))
end
omx_file = Substitute(skim_mat, ".mtx", ".omx", )
copy_matrix_options = {"File Name": omx_file, "OMX":"True", "Compression": 0}
CopyMatrix(skim_curs[1][2], copy_matrix_options)
//EndStep
(additional code)
Figure 24: SEMCOG Vehicle Time of Day Macro (Partial Code)
Macro "SEMCOG Vehicle Time of Day" (Args)
(additional code)
NextStep= "Write OD Tables to Disk"
SetStatus(1, NextStep, )
// Changed to writing OD tables to .OMX for integration with CVM
for period = 1 to pkop_pers.length do
for subper = 1 to sub_pers[period].length do
period_s = pkop_pers[period]
subper_s = sub_pers[period][subper]
//Accumulate daily totals
for c in classes do
dy_curs.(c) := nz(dy_curs.(c)) + nz(veh_curs_sp.(subper_s).(c))
end
//Copy to disk
veh_file = Substitute(veh_files, "%PER_HWY%", subper_s, )
Opts = null
Opts.[File Name] = veh_file
Opts.Label = subper_s + " Vehicle Trips in OD Format"
Opts.Compression = 0
Opts.OMX = "True"
CopyMatrix(veh_curs_sp.(subper_s).SOV, Opts)
veh_curs_sp.(subper_s) = null
end //sub-period
end //period
veh_curs_sp = null
//Copy daily to disk
dy_file = Substitute(veh_files, "%PER_HWY%", "DY", )
Opts = null
Opts.[File Name] = dy_file
Opts.Label = "DY Vehicle Trips in OD Format"
Opts.Compression = 0
Opts.OMX = "True"
CopyMatrix(dy_curs.SOV, Opts)
dy_curs = null
(additional code)
4.1.5 Scenario Input and Output Modifications
The scenario input and output file and folder structure has been modified slightly in the E8Plus model to incorporate the CV model. At the folder level, the input structure is unchanged with a new folder, “CVM”, added for CV model outputs. The figure below shows the folder structure and calls out new inputs required by the CV model and where they are typically located and new or modified outputs produced by the E8Plus model including highway skims and trip tables in .omx format and other CV model outputs.
The actual locations and filenames of scenario inputs to the CV model are flexible as they are not hard coded in the CV model but rather the files are selected using the scenario GUI and their file path and name are passed as variables to the CV model. The full list of CV model inputs and outputs are shown in the sections on CV Model Data Inputs and CV Model Outputs.
Figure 25: SEMCOG E8Plus Model Scenarios File Structure
semcog_cvtm_vX.X.X
* Model Runs Scenarios directory
+ 2015_KA15 Base year scenario directory
- Input Base scenario inputs directory
+ AQ Air quality inputs
+ EJ EJ inputs
+ Network Networks
+ Params Parameters
+ SED Socioeconomic data
- CVM_Externals.csv External stations active in the scenario, used in the CVM
- CVM_EE_Region.csv Daily EE Trucks by External Region, used in the CVM
- CVM_IEEI_Region.csv Daily IE and EI Trucks by External Region, used in the CVM
- Facilities.csv Freight activity centers facilties used in the CVM
- TAZSocioEconomicsBuffer.csv TAZ data for the buffer region used in the CVM
- Output Base scenario outputs directory
+ AQ Air quality outputs
+ CVM CVM outputs
- 1.Firms.Rdata List of Firms from Frim Synthesis
- 2.LongDistanceTrips.Rdata Long Distance Truck Trips
- 3.CommercialVehicleTrips.Rdata Commercial Vehicle Tours and Trips
- 4.TripTables.Rdata Commercial Vehicle Trip Database
- 5.DashboardTables.Rdata Dashboard Input and Tabulation Database
- CV_Trip_Generation_Summary.csv Commercial Vehicle Trip Generation
- ReportDashboard.html Commercial Vehicle Dashboard
- (additional output files) (See CV Model Outputs for comprehensive list)
+ EJ EJ outputs
+ HAssign Highway assignment outputs
- Flow_AM.csv AM time period assignment flow table in .csv format
- Flow_DY.csv Daily assignment flow table in .csv format
- Flow_EV.csv EV time period assignment flow table in .csv format
- Flow_MD.csv MD time period assignment flow table in .csv format
- Flow_NT.csv NT time period assignment flow table in .csv format
- Flow_PM.csv PM time period assignment flow table in .csv format
- OD_AM.omx AM time period trip tables in .omx format
- OD_DY.omx Daily trip tables in .omx format
- OD_EV.omx EV time period trip tables in .omx format
- OD_MD.omx MD time period trip tables in .omx format
- OD_NT.omx NT time period trip tables in .omx format
- OD_PM.omx PM time period trip tables in .omx format
+ HNet Highway network
+ HSkm Highway skims
- AM_HwySkim.omx AM time period highway skims in .omx format
- EV_HwySkim.omx EV time period highway skims in .omx format
- MD_HwySkim.omx MD time period highway skims in .omx format
- NT_HwySkim.omx NT time period highway skims in .omx format
- PM_HwySkim.omx PM time period highway skims in .omx format
+ ModeChoice Mode choice model outputs
+ TNet Transit network
+ Tripdist Trip distribution outputs
+ Tripgen Trip generation outputs
+ TrnAssign Transit assignment outputs
+ TSkm Transit skims
+ 2020_KC20 Future (2020) scenario directory
+ 2025_KC25 Future (2025) scenario directory
+ 2030_KC30 Future (2030) scenario directory
+ 2035_KC35 Future (2035) scenario directory
+ 2040_KC40 Future (2040) scenario directory
+ 2045_KC45 Future (2045) scenario directory
4.2 CV Model Software Architecture
This section of the user guide presents additional details on the software architecture of the CV model itself, within the structure of the overall model system described above. It covers:
CV model file structure
CV model scripts (introduction, with full details in the CV Model Scripts section)
CV model inputs (introduction, with full details in the Data Inputs section)
CV model outputs (introduction, with full details in the Model Outputs section)
CV model reporting
4.2.1 CV Model Software Architecture Overview
The figure below presents the general software architecture of the CV model; each of the CV model components conforms to this structure. The R code components are in two sections:
- First, the CV model includes a function library in the form of an R
package called
rFreight(orange box) that contains functions used in the model - Secondly, a set of scripts call functions from the
rFreightlibrary to implement each of the model components (green boxes). A master controller file,__Master.Racts as the main interface for the models, initiating the model run and sequentially running the script files that apply each of the individual model components. The scripts are consistently structured to apply a single model component (such as firm synthesis): they load inputs, carry out any required data processing, apply the model, save outputs, and produce summaries from that model component before passing control back to the controller file. In this way, the model code is modularized, and individual model components can be updated in future phases of model development. - The CV model’s R code includes functions to load model inputs (blue box), such as tabular inputs describing the number of business establishments or travel time skims, from the file system.
- The CV model’s R code includes functions to produce summary tabulations of results and to build an interactive dashboard to display results in tables, charts, and maps.
- The CV model relies on the functionality of the TransCAD portions of the E8Plus model with which it is integrated to skim transportation networks and to assign truck trip tables onto the highway network.
Figure 26: Software Architecture Diagram
4.2.2 CV Model File Structure
The CV model (except for scenario inputs and outputs) is included
entirely within the CVM directory, which is located in the
root directory of the E8Plus model. Inside the CVM
directory, the model has the file structure shown in the figure below.
The model application contain two directories, only one of which is
required to actually run the model, and several files that are important
for integration with the E8Plus model:
The
libfolder which contains the model’s scripts, parameters, inputs that do not vary across scenarios, and the R function packages used to run the model’s scripts. It also includes the rFreight package .zip file and the installer for Pandoc which is used convert the model dashboard to HTML (see Install Pandoc for more infomation about Pandoc).The
devfolder which contains model development scripts and other files. This directory is useful for making updates to model inputs but is not required for model application runs. The sections describing Creating and Running Scenarios and [Updating the Model] provide more information about the contents of thedevfolder.
The model scripts, data inputs (both common and scenario specific) and the model outputs are described in more detail below.
Figure 27: CV Model File Structure
semcog_cvtm_vX.X.X
* CVM CV Model directory
* dev Model development directory
* lib Model library directory
+ data Common input data directory
+ pkgs R application and R packages directory
- library R packages directory
- R-4.1.2 R application directory (if R installed inside the CV model)
+ scripts R scripts directory
- db_markdown Dashboard script directory
+ pandoc-1.19-windows.msi Pandoc installer
+ rFreight_0.1-33.zip rFreight package
* run_semcog_cvtm.bat Batch file called by TransCAD to run the CV model
* run_semcog_cvtm.r R script called from the batch file to start the model run
4.2.3 CV Model Scripts
The R scripts that implement the CV model are included in the
CVM/lib/scripts folder of the application (with the one
exception being run_semcog_cvtm.r which is in the
CVM directory). The scripts are listed and described in the
table below. There are several types of script which are named using the
following naming convention:
Master A master controller file,
__Master.Racts as the main interface for the models, initiating the model run and sequentially running the script files that apply each of the individual model components.Initialization Scripts names prefixed with “init”, e.g.,
init_start_rfreight_model.R, contain code to start the model application and support functions such as loading the required R package function libraries.Variables There are several scripts that contain global variables used through the CV model. The script naming convention is upper case and starting with an underscore, e.g.
_BASE_VARIABLES.R. The variables in these scripts are described in Variables.Control Each CV model component is run by a control script that either runs a model simulation or builds an output. In the case of a model simulation, the script is named after the model component suffixed with “_sim”, e.g.,
ld_sim.Rfor the long-distance model control script. For the model steps that build an output, the script is named after the model component suffixed with “_build”, e.g.,tt_build.Rfor the trip tables control script.Input Processing Each CV model component contains a script that loads the input files used by the model component into an environment in memory and does some initial processing. This script is run as the first step of each model component. These scripts have the same name as the control script for the model component with the additional suffix of “_process_inputs”, e.g.,
ld_sim_process_inputs.Rfor the long-distance model’s input processing function.Model Step The control script for each CV model component can call additional scripts that contain functions implementing individual model steps. These scripts have the same name as the control script for the model component with an additional suffix describing the model step, e.g.,
ld_sim_generation.Rwhich contains the function to implement the long-distance model’s trip generation model.
The CV Model Scripts section of the user guide describes the content of the CV model’s scripts in more detail.
Table 2: List of model scripts
| StepNumber | StepName | Filename | Description | ScriptType |
|---|---|---|---|---|
| 0 | Initialization | _BASE_VARIABLES.R | List of base variables | Variables |
| 0 | Initialization | _SCENARIO_VARIABLES.R | List of scenario variables | Variables |
| 0 | Initialization | _SYSTEM_VARIABLES.R | List of system variables | Variables |
| 0 | Initialization | _USER_VARIABLES.R | List of user varibales | Variables |
| 0 | Initialization | __Master.R | Master script to control commercial vehicle model flow | Master |
| 0 | Initialization | init_install_special_packages.R | Initialization install R packages | Intialization |
| 0 | Initialization | init_start_rfreight_model.R | Initialization start model application | Intialization |
| 1 | Firm Synthesis | firm_sim.R | Firm synthesis control function | Control |
| 1 | Firm Synthesis | firm_sim_process_inputs.R | Firm synthesis input processing function | Input Processing |
| 2 | Long Distance | ld_sim.R | Long distance control function | Control |
| 2 | Long Distance | ld_sim_distribution.R | Long distance trip distribution model | Model Step |
| 2 | Long Distance | ld_sim_generation.R | Long distance trip generation model | Model Step |
| 2 | Long Distance | ld_sim_process_inputs.R | Long distance input processing function | Input Processing |
| 2 | Long Distance | ld_sim_throughtrips.R | Long distance through trip scaling function | Model Step |
| 2 | Long Distance | ld_sim_veh_type_time_split.R | Long distance vehicle type and time-of-day splitting function | Model Step |
| 3 | CV Touring | cv_sim.R | CVTM control function | Control |
| 3 | CV Touring | cv_sim_activities.R | CVTM activities model | Model Step |
| 3 | CV Touring | cv_sim_externalstations.R | CVTM external stations for I-E and E-I trips | Model Step |
| 3 | CV Touring | cv_sim_intermediatestops.R | CVTM non-scheduled stops model | Model Step |
| 3 | CV Touring | cv_sim_process_inputs.R | CVTM input processing function | Input Processing |
| 3 | CV Touring | cv_sim_scheduledstops.R | CVTM stop generation model | Model Step |
| 3 | CV Touring | cv_sim_scheduledtrips.R | CVTM trip scheduling model | Model Step |
| 3 | CV Touring | cv_sim_stopduration.R | CVTM stop duration model | Model Step |
| 3 | CV Touring | cv_sim_tours.R | CVTM tour model (clustering and sequencing) | Model Step |
| 3 | CV Touring | cv_sim_vehicle.R | CVTM vehicle choice model | Model Step |
| 4 | Trip Tables | tt_build.R | Trip table building function | Control and Model Step |
| 4 | Trip Tables | tt_build_process_inputs.R | Trip table input processing function | Input Processing |
| 5 | Dashboard | db_build.R | Dashboard control function | Control |
| 5 | Dashboard | db_build_graphics_functions.R | Dashboard helper functions | Support Functions |
| 5 | Dashboard | db_build_process_inputs.R | Dashboard input processing function | Input Processing |
| 5 | Dashboard | db_build_render.R | Dashboard rendering and post-processing | Model Step |
| 5 | Dashboard | db_build_spreadsheet.R | Spreadsheet summary of model outputs | Model Step |
| 5 | Dashboard | db_build_spreadsheet_functions.R | Spreadsheet summary helper functions | Support Functions |
| 5 | Dashboard | db_markdown | Contains dashboard Rmarkdown template and supporting files | Directory |
4.2.4 CV Model Data Inputs
The tabular and parameter inputs to the CV model are in two locations:
Inputs that are common across all model scenarios are located in the
CVM/lib/datadirectoryInputs for a particular scenario are included in the
Model Runs/[scenario name]directory
The first group, the common inputs, are listed in the first table below, while the scenario specific inputs are listed in the second table below. The files are mainly comma separated variables (.csv) format files or encoded into one of R’s binary formats for either single objects (.rds) or a work space capable of holding multiple objects (.Rdata), but there are also ESRI shapefiles (.shp) files of TAZ layers and .omx matrices.
The scenario specific inputs include static input file prepared before a scenario is run and those produced during a run such as skims matrices.
The Data Inputs section of the user guide describes the development and format of each file in turn.
Table 3: List of inputs that are common across scenarios
| StepNumber | StepName | Filename | Description | InputType |
|---|---|---|---|---|
| 1 | Firm Synthesis | EstSizeCategories.csv | Establishment size categories for Firm Synthesis | Category definitions and correspondences |
| 1 | Firm Synthesis | Establishments.csv | List of establishments for Firm Synthesis | Tabular data |
| 1 | Firm Synthesis | NAICS3_to_EmpCats.csv | Correspondence between NAICS codes and employment Categories | Category definitions and correspondences |
| 1 | Firm Synthesis | TAZ_System.csv | TAZ descriptions, tabular form of shapefile attribute data | Tabular data |
| 2 | Long Distance | ld_distribution_model.csv | Long distance distribution model parameters | Model coefficients and parameters |
| 2 | Long Distance | ld_ee_trip_table.csv | Long distance static external-to-external trip table | Tabular data |
| 2 | Long Distance | ld_external_model.csv | Long distance external station allocation model | Model coefficients and parameters |
| 2 | Long Distance | ld_internal_gen_model.RDS | Long distance generations model coefficients for internal zones | Model coefficients and parameters |
| 2 | Long Distance | ld_trip_time_vehicle_proportions.csv | Long distance trip proportions by vehicle type and time-of-day | Model coefficients and parameters |
| 3 | CV Touring | TravelTimeSkimBuffer.omx | Travel time skims for the buffer TAZs | Matrix data |
| 3 | CV Touring | cv_activities_model.RDS | Commercial Vehicle activities model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_arrival_model.RDS | Commercial Vehicle arrival model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_goods_model.RDS | Commercial Vehicle goods stops model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_intermediate_deviations.rds | Commercial Vehicle non-scheduled stop deviation thresholds | Model coefficients and parameters |
| 3 | CV Touring | cv_intermediate_model.rds | Commercial Vehicle non-scheduled stop generation model | Model coefficients and parameters |
| 3 | CV Touring | cv_intermediate_model_attraction.rds | Commercial Vehicle non-scheduled stop location model | Model coefficients and parameters |
| 3 | CV Touring | cv_service_model.RDS | Commercial Vehicle service stops model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_settings.RData | Commercial Vehicle model parameters | Model coefficients and parameters |
| 3 | CV Touring | cv_stopduration_model.RDS | Commercial Vehicle stop duration model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_tours_model.rds | Commerical Vehicle tour type model | Model coefficients and parameters |
| 3 | CV Touring | cv_vehicle_model.RDS | Commercial Vehicle vehicle choice model coefficients | Model coefficients and parameters |
| 5 | Dashboard | TAZ_System_Shape.dbf | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | TAZ_System_Shape.prj | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | TAZ_System_Shape.shp | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | TAZ_System_Shape.shx | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | counts_daily_class.csv | Count data containing daily counts for validation spreadsheet | Tabular data |
| 5 | Dashboard | counts_time_period.csv | Count data containing time period counts for validation spreadsheet | Tabular data |
Table 4: List of inputs that are scenario specific
| Filename | Location | Description |
|---|---|---|
| CVM_EE_Region.csv | Input/SED | External region to external region growth factors |
| CVM_Externals.csv | Input/SED | External stations active in the scenario, and external allocation factors for trucks |
| CVM_IEEI_Region.csv | Input/SED | Internal to external and external to internal external region control totals |
| Facilities.csv | Input/SED | Descriptions of high-intensity freight locations |
| TAZSocioEconomicsBuffer.csv | Input/SED | Employment, households, and population data by TAZ for the buffer TAZs |
| base_transcad_taz.csv | Input/SED | Employment, households, and population data by TAZ for the SEMCOG internal TAZs |
4.2.5 CV Model Outputs
The main tabular outputs from the CV model for a particular scenario
are included in the Output/CVM folder for the scenario. The
output files are either .csv format files or R binary format files
(.Rdata) that contain compressed tables. In addition, the CV model adds
trip tables to the .omx trip tables in the Output/HAssign
folder for the scenario. These outputs are described in the table
below.
In addition to the main tabular and matrix outputs, the model produces many summary outputs which are also listed in the table below. These tabular, matrix, and summary outputs are in addition to the downstream outputs such as vehicle link flow tables and HTML reports that the E8Plus model’s highway assignment and reporting steps produce.
The Model Outputs section of the user guide describes the format of each output file in turn.
Table 5: List of model outputs
| Filename | Location | Description |
|---|---|---|
| 1.Firms.RData | Output/CVM | Database of inputs and outputs from Firm Synthesis model component |
| 1.Firms.RData.BaseYearFirms.csv | Output/CVM | CSV export of firm synthesis base year firms table |
| 1.Firms.RData.ScenarioFirms.csv | Output/CVM | CSV export of firm synthesis scenario firms table |
| 1.Firms.RData.TAZLandUseCVTM.csv | Output/CVM | CSV export of processed land use database |
| 2.LongDistanceTrips.RData | Output/CVM | Dababase of inputs and outputs from Long Distance model component |
| 2.LongDistanceTrips.RData.ld_trips.csv | Output/CVM | CSV export of long distance model trip database |
| 2.LongDistanceTrips.RData.zoneProdsAttrs.csv | Output/CVM | CSV export of long distance model zonal productions and attractions |
| 3.CommercialVehicleTrips.RData | Output/CVM | Database of inputs and outputs from CVTM model component |
| 3.CommercialVehicleTrips.RData.cv_trips.csv | Output/CVM | CSV export of CVTM trips and tours table |
| 4.TripTables.RData | Output/CVM | Database of inputs and outputs from Trip Table model component |
| 4.TripTables.RData.TripTable.csv | Output/CVM | CSV export of trip table in list format |
| 5.DashboardTables.RData | Output/CVM | Database of inputs to the dashboard component |
| CVTM_Stops_Duration_Activity_Non-Scheduled.csv | Output/CVM | CVTM stop durations by activity, non scheduled (dashboard) |
| CVTM_Stops_Duration_Activity_Scheduled.csv | Output/CVM | CVTM stop durations by activity, scheduled (dashboard) |
| CVTM_Stops_Duration_Vehicle.csv | Output/CVM | CVTM stop durations by vehicle (dashboard) |
| CVTM_Stops_Duration_Vehicle_Activity.csv | Output/CVM | CVTM stop durations by activity, vehicle (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Activity.csv | Output/CVM | CVTM first stop arrival by activity (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Vehicle.csv | Output/CVM | CVTM first stop arrival by vehicle (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Vehicle_Activity.csv | Output/CVM | CVTM first stop arrival by activity, vehicle (dashboard) |
| CV_Trip_Generation_Summary.csv | Output/CVM | CV model trip generation summary (for HTML report) |
| CV_Trip_VMT_VHT_Summary.csv | Output/CVM | CV model trip, VMT, and VHT summary (dashboard) |
| Commercial_County_Vehicle_Segment.csv | Output/CVM | CVTM summary by count, vehicle, OD segment (dashboard) |
| Commercial_County_Vehicle_TOD.csv | Output/CVM | CVTM summary by count, vehicle, time of day (dashboard) |
| Commercial_Heavy_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, heavy vehicles (dashboard) |
| Commercial_Light_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, light vehicles (dashboard) |
| Commercial_Medium_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, medium vehicles (dashboard) |
| Commercial_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by activity (dashboard) |
| Commercial_Stop_Activity_by_County.csv | Output/CVM | CVTM stop activity by county (dashboard) |
| Commercial_Stop_Activity_by_County_Heavy.csv | Output/CVM | CVTM stop activity by county for heavy vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Light.csv | Output/CVM | CVTM stop activity by county for light vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Medium.csv | Output/CVM | CVTM stop activity by county for medium vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Vehicle.csv | Output/CVM | CVTM trip Activity by vehicle type (dashboard) |
| Commercial_Tour_NonScheduled_Stops_Tours.csv | Output/CVM | CVTM tours by number of non scheduled stops (dashboard) |
| Commercial_Tour_Scheduled_Stops_Tours.csv | Output/CVM | CVTM tours by number of scheduled stops (dashboard) |
| Commercial_Tour_Start_Locations_by_Region.csv | Output/CVM | CVTM tour start locations by region (dashboard) |
| Commercial_Tour_Stops_Tours.csv | Output/CVM | CVTM tours stops by tour (dashboard) |
| Commercial_Trip_Destinations_by_County_Buffer.csv | Output/CVM | CVTM trip destinations by SEMCOG county and buffer (dashboard) |
| Commercial_Trip_OD_Vehicle.csv | Output/CVM | CVTM trips by OD segment by vehicles (dashboard) |
| Commercial_Trip_Origins_by_County_Buffer.csv | Output/CVM | CVTM trip origins by SECMOG county and buffer (dashboard) |
| Commercial_Trip_TOD_Vehicle.csv | Output/CVM | CVTM trips by time of day and vehicle (dashboard) |
| Commercial_Trips_II_County_County.csv | Output/CVM | CVTM II trips by county OD (dashboard) |
| Commercial_Trips_IX_County_External.csv | Output/CVM | CVTM IX trips by county-external OD (dashboard) |
| Commercial_Trips_XI_External_County.csv | Output/CVM | CVTM XI trips by external-county OD (dashboard) |
| Commercial_Trips_XX_External_External.csv | Output/CVM | CVTM XX trips by external-external OD (dashboard) |
| Employees_by_County.csv | Output/CVM | Firm Synthesis: Employees by County (dashboard) |
| Employees_by_Industry.csv | Output/CVM | Firm Synthesis: Employees by Industry (dashboard) |
| Employees_by_Industry_and_County.csv | Output/CVM | Firm Synthesis: Employees by Industry and County (dashboard) |
| Employees_by_Industry_and_Size_Buffer.csv | Output/CVM | Firm Synthesis: Employees by Industry and Firm Size in the buffer (dashboard) |
| Employees_by_Industry_and_Size_SEMCOG.csv | Output/CVM | Firm Synthesis: Employees by Industry and Firm Size, SEMCOG (dashboard) |
| Employees_by_Size.csv | Output/CVM | Firm Synthesis: Employees by Firm Size (dashboard) |
| Facilities_by_Type.csv | Output/CVM | Firm Synthesis: Facilites by facility type (dashboard) |
| Firms_by_County.csv | Output/CVM | Firm Synthesis: Firms by County (dashboard) |
| Firms_by_Industry.csv | Output/CVM | Firm Synthesis: Firms by Industry (dashboard) |
| Firms_by_Industry_and_County.csv | Output/CVM | Firm Synthesis: Firms by Industry and County (dashboard) |
| Firms_by_Industry_and_Size_Buffer.csv | Output/CVM | Firm Synthesis: Firms by Industry and Firm Size in the buffer (dashboard) |
| Firms_by_Industry_and_Size_SEMCOG.csv | Output/CVM | Firm Synthesis: Firms by Industry and Firm Size, SEMCOG (dashboard) |
| Firms_by_Size.csv | Output/CVM | Firm Synthesis: Firms by Firm Size (dashboard) |
| Long-Distance_Trips_IXXIODSummary.csv | Output/CVM | Long Distance Trips IX and XI OD Summary (dashboard) |
| Long-Distance_Trips_IX_County_External.csv | Output/CVM | Long Distance IX Trips by County and External (dashboard) |
| Long-Distance_Trips_XI_External_County.csv | Output/CVM | Long Distance XI Trips by External and County (dashboard) |
| Long-Distance_Trips_XX_External_External.csv | Output/CVM | Long Distance XX Trips by External OD (dashboard) |
| Long-Distance_Trips_XX_ODSummary.csv | Output/CVM | Long Distance XX Trips OD Summary (dashboard) |
| Long-Distance_Trips_by_County.csv | Output/CVM | Long Distance Trips by County (dashboard) |
| Long-Distance_Trips_by_DistBin.csv | Output/CVM | Long Distance Trips by Distance range (dashboard) |
| Long-Distance_Trips_by_External.csv | Output/CVM | Long Distance Trips by External Station (dashboard) |
| Long-Distance_Trips_by_External_TAZ.csv | Output/CVM | Long Distance Trips by External and TAZ (dashboard) |
| Long-Distance_Trips_by_External_TOD.csv | Output/CVM | Long Distance Trips by External and Time Period (dashboard) |
| Long-Distance_Trips_by_External_TOD_ODSeg.csv | Output/CVM | Long Distance Trips by External, Time Period and OD segment (dashboard) |
| Long-Distance_Trips_by_TOD.csv | Output/CVM | Long Distance Trips by Time Period (dashboard) |
| ReportDashboard.html | Output/CVM | CVM dashboard for the scenario |
| TripTable_Heavy_PctTrips_by_County_OD.csv | Output/CVM | Trip table heavy percent trips by county OD (dashboard) |
| TripTable_Heavy_Trips_by_County_OD.csv | Output/CVM | Trip table heavy trips by county OD (dashboard) |
| TripTable_Light_PctTrips_by_County_OD.csv | Output/CVM | Trip table light percent trips by county OD (dashboard) |
| TripTable_Light_Trips_by_County_OD.csv | Output/CVM | Trip table light trips by county OD (dashboard) |
| TripTable_Medium_PctTrips_by_County_OD.csv | Output/CVM | Trip table medium percent trips by county OD (dashboard) |
| TripTable_Medium_Trips_by_County_OD.csv | Output/CVM | Trip table medium trips by county OD (dashboard) |
| TripTable_PctTrips_by_County_OD.csv | Output/CVM | Trip table percent trips by county OD (dashboard) |
| TripTable_Trips_IX_County_External.csv | Output/CVM | Trip table IX trips by county external OD (dashboard) |
| TripTable_Trips_XI_External_County.csv | Output/CVM | Trip table XI trips by external county OD (dashboard) |
| TripTable_Trips_XX_External_External.csv | Output/CVM | Trip table XX trips by external external OD (dashboard) |
| TripTable_Trips_by_County_OD.csv | Output/CVM | Trip table trips by county OD (dashboard) |
| TripTable_Trips_by_Vehicle_Segment_TOD.csv | Output/CVM | Trip table trips by vehicle, OD segment, and time period (dashboard) |
| Validation_2015_KA15_2022_07_04_08_15_17.xlsx | Output/CVM | Validation spreadsheet with time and date naming (dashboard) |
| log | Output/CVM | Folder of log files for model runs |
| skims_buffer.rds | Output/CVM | Combined skims table including buffer region |
| skims_tod.rds | Output/CVM | Combined skims table for all time periods for SEMCOG region |
| OD_AM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for AM time period |
| OD_DY.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for full day |
| OD_EV.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for EV time period |
| OD_MD.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for MD time period |
| OD_NT.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for NT time period |
| OD_PM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for PM time period |
4.2.6 CV Model Reporting
The CV model’s outputs are summarized using two reporting mechanisms which are documented in the Model Outputs section of the user guide.
CV Model Dashboard The dashboard component of the CV model produces a visualization dashboard that summarizes results from the CV model only. This is a scenario specific file such as
Dashboard_base_2015.htmlfor the base scenario that is saved in theOutput/CVMfolder for the scenario. The dashboard is a single HTML file that is produced at the end of each model run. Since it is an HTML file, it can be linked directly to a website or distributed and opened in a web browser. It contains interactive tabulations, charts, and maps embedded directly into the HTML file so that users can explore the scenario results.E8Plus Model Report The E8Plus model’s GUI includes options to produce an HTML report from some or all steps of the E8Plus model run, including the CV model. This also a scenario specific file such as
TransCAD4 E8Plus V1.0 KA15 Test_Summary.htmlfor the base scenario that is saved in theOutputfolder for the scenario.
5 CV Model Scripts
5.1 Introduction
The R scripts that implement the CV model are included in the
CVM/lib/scripts folder of the application (with the one
exception being run_semcog_cvtm.r which is in the
CVM directory along with the
run_semcog_cvtm.bat batch file). The scripts are listed and
described in the table below. There are several types of script which
are named using the following naming convention:
Run Scripts File names prefixed with “run”, including a batch file
run_semcog_cvtm.batand an R scriptrun_semcog_cvtm.r, facilitate integration with TransCAD, allowing the CV model to be called from TransCAD, to communicate run parameters to the CV model, and to return error codes to TransCAD. These are the only scripts that are in the root of theCVMdirectory.Master A master controller file,
__Master.Racts as the main interface for the models, initiating the model run and sequentially running the script files that apply each of the individual model components.[Initialization] Script names prefixed with “init”, e.g.,
init_start_rfreight_model.R, contain code to start the model application and support functions such as loading the required R package function libraries.Variables There are several scripts that contain global variables used through the CV model. The script naming convention is upper case and starting with an underscore, e.g.
_BASE_VARIABLES.R.Control Each CV model component is run by a control script that either runs a model simulation or builds an output. In the case of a model simulation, the script is named after the model component suffixed with “_sim”, e.g.,
ld_sim.Rfor the long-distance model control script. for the model steps that build an output, the script is named after the model component suffixed with “_build”, e.g.,tt_build.Rfor the trip tables control script.Input Processing Each CV model component contains a script that loads the input files used by the model component into an environment in memory and does some initial processing. This script is run as the first step of each model component. These scripts have the same name as the control script for the model component with the additional suffix of “_process_inputs”, e.g.,
ld_sim_process_inputs.Rfor the long-distance model’s input processing function.Model Step The control script for each CV model component can call additional scripts that contain functions implementing individual model steps. These scripts have the same name as the control script for the model component with an additional suffix describing the model step, e.g.,
ld_sim_generation.Rwhich contains the function to implement the long-distance model’s trip generation model.
The remainder of this chapter documents the content of the CV model’s with a section discussing each type of script.
Table 6: List of model scripts in CVM/lib/scripts
| StepNumber | StepName | Filename | Description | ScriptType |
|---|---|---|---|---|
| 0 | Initialization | _BASE_VARIABLES.R | List of base variables | Variables |
| 0 | Initialization | _SCENARIO_VARIABLES.R | List of scenario variables | Variables |
| 0 | Initialization | _SYSTEM_VARIABLES.R | List of system variables | Variables |
| 0 | Initialization | _USER_VARIABLES.R | List of user varibales | Variables |
| 0 | Initialization | __Master.R | Master script to control commercial vehicle model flow | Master |
| 0 | Initialization | init_install_special_packages.R | Initialization install R packages | Intialization |
| 0 | Initialization | init_start_rfreight_model.R | Initialization start model application | Intialization |
| 1 | Firm Synthesis | firm_sim.R | Firm synthesis control function | Control |
| 1 | Firm Synthesis | firm_sim_process_inputs.R | Firm synthesis input processing function | Input Processing |
| 2 | Long Distance | ld_sim.R | Long distance control function | Control |
| 2 | Long Distance | ld_sim_distribution.R | Long distance trip distribution model | Model Step |
| 2 | Long Distance | ld_sim_generation.R | Long distance trip generation model | Model Step |
| 2 | Long Distance | ld_sim_process_inputs.R | Long distance input processing function | Input Processing |
| 2 | Long Distance | ld_sim_throughtrips.R | Long distance through trip scaling function | Model Step |
| 2 | Long Distance | ld_sim_veh_type_time_split.R | Long distance vehicle type and time-of-day splitting function | Model Step |
| 3 | CV Touring | cv_sim.R | CVTM control function | Control |
| 3 | CV Touring | cv_sim_activities.R | CVTM activities model | Model Step |
| 3 | CV Touring | cv_sim_externalstations.R | CVTM external stations for I-E and E-I trips | Model Step |
| 3 | CV Touring | cv_sim_intermediatestops.R | CVTM non-scheduled stops model | Model Step |
| 3 | CV Touring | cv_sim_process_inputs.R | CVTM input processing function | Input Processing |
| 3 | CV Touring | cv_sim_scheduledstops.R | CVTM stop generation model | Model Step |
| 3 | CV Touring | cv_sim_scheduledtrips.R | CVTM trip scheduling model | Model Step |
| 3 | CV Touring | cv_sim_stopduration.R | CVTM stop duration model | Model Step |
| 3 | CV Touring | cv_sim_tours.R | CVTM tour model (clustering and sequencing) | Model Step |
| 3 | CV Touring | cv_sim_vehicle.R | CVTM vehicle choice model | Model Step |
| 4 | Trip Tables | tt_build.R | Trip table building function | Control and Model Step |
| 4 | Trip Tables | tt_build_process_inputs.R | Trip table input processing function | Input Processing |
| 5 | Dashboard | db_build.R | Dashboard control function | Control |
| 5 | Dashboard | db_build_graphics_functions.R | Dashboard helper functions | Support Functions |
| 5 | Dashboard | db_build_process_inputs.R | Dashboard input processing function | Input Processing |
| 5 | Dashboard | db_build_render.R | Dashboard rendering and post-processing | Model Step |
| 5 | Dashboard | db_build_spreadsheet.R | Spreadsheet summary of model outputs | Model Step |
| 5 | Dashboard | db_build_spreadsheet_functions.R | Spreadsheet summary helper functions | Support Functions |
| 5 | Dashboard | db_markdown | Contains dashboard Rmarkdown template and supporting files | Directory |
5.2 Run Scripts
There are two files in the root of the CVM directory
that facilitate integration with TransCAD, allowing the CV model to be
called from TransCAD, to communicate run parameters to the CV model, and
to return error codes to TransCAD. These are the only scripts that are
in the root of the CVM directory. The two file are
described in this section:
5.2.1 run_semcog_cvtm.bat Batch File
The run_semcog_cvtm.bat batch file builds a call to run
the RScript.exe application, which is the batch version of
R (as opposed to RGui which is the interactive version). The batch file
contains information about where R is installed on the machine. The
batch file calls RScript and asks it to run the
run_semcog_cvtm.R script along with the arguments passed
from the E8Plus model’s TransCAD macro. The arguments are used to select
a specific CV model step to run. The batch file includes error handling
to communicate any errors back from RScript to TransCAD. The full code
of the batch file is shown in the figure below and each section is
described here.
User Defined Paths to R installation
This section of the batch file includes variables that connect the CV model to the installation of R on the machine. Setting these paths during installation of the model is described in Configure the CV Model.
pathtorinsidemodel: Location of R (if a version of R is installed inside the CV model, which can be helpful if other versions of R are installed on the machine so for portability of the model so that it can be coped from one computer to another without need to re install R and making configuration changes)pathtorcdrive: Location of the R install on the computer in a typical install location such as on the C drive.
Variable definitions
This section of the batch file reads the arguments passed to the batch file describing which of the CV model components to run. The arguments comprise five TRUE or FALSE variables, one for each of the five CV model component. In the absence of any arguments (for example if the batch file is just double clicked in windows explorer), values of TRUE will be used by default which runs all of the CV model steps. The full set of arguments that be can used during a run of the CV is described in run_semcog_cvtm.R R Script.
Run CV Model For Selected Components
This section of the batch file builds a call to run the
run_semcog_cvtm.R script in Rscript.exe based
on the location variable for the installation of R and the arguments
passed to the batch file. The message output from running the
Rscript.exe is saved to log file called
run_semcog_cvtm_log.txt that is saved in the root of the
CVM folder. This section of the batch file includes code to
support returning error handling messages to TransCAD and also removes
the temporary configuration file run_semcog_cvtm.txt
written by TransCAD and read by the CV model if the run of the CV model
completes successfully.
Figure 28: run_semcog_cvtm_integrated.bat
:: SEMCOG Commercial Vehicle Model Batch File
:: Batch file intended to be called from SEMCOG E8Plus TransCAD model
:: Set up to run component as called from the calling TransCAD macro which will send command line arguments:
:: %1 - %5 TRUE/FALSE switches for each of the CVM components in this order:
:: firm synthesis, long distance, commercial vehicle touring model, trip table export, and dashboard.
:: If manually run (e.g., executed by double clicking the batch) defaults to running all steps of the base scenario
:: User Defined Paths to R installation
:: for R installed inside the model, e.g. ".\lib\pkgs\R-4.1.2\bin\x64"
:: (if there is not an R installation at this location inside the model,
:: the C drive location specified in "pathtorcdrive" below will be checked)
set pathtorinsidemodel=".\lib\pkgs\R-4.1.2\bin\x64"
:: for R installed elsewhere such as the typical program files location, for example "C:\Program Files\R\R-4.1.2\bin\x64"
:: (if there is not an R installation at this location either, the batch file will pause with a message and then exit)
set pathtorcdrive="C:\Program Files\R\R-4.1.2\bin\x64"
:: Path to the model installation
:: find the location of this batch file
set pathtocvtmmodel=%~dp0
cd /D %pathtocvtmmodel%
:: Check the paths to R installation
if EXIST %pathtorinsidemodel%\Rscript.exe (
set pathtor=%pathtorinsidemodel%
goto getcommandargs
)
set pathtor=%pathtorcdrive%
if NOT EXIST %pathtor%\Rscript.exe (
echo Check settings for pathtorinsidemodel or pathtorcdrive variables in the %pathtocvtmmodel%\run_semong_cvtm.bat batch file, Rscript.exe not found
echo %pathtor%
pause
)
:: Variable definitions
:: Check for command line arguments
:getcommandargs
if [%1]==[] goto setsteps
set runfirmsyn=%1
set runlongdist=%2
set runcvtm=%3
set runttexp=%4
set rundashboard=%5
goto runcvm
:: No command line arguments passed (e.g., batch file run manually)
:setsteps
set runfirmsyn="TRUE"
set runlongdist="TRUE"
set runcvtm="TRUE"
set runttexp="TRUE"
set rundashboard="TRUE"
echo No command line arguments so all steps will be run
goto runcvm
:: Run CV Model For Selected Components
:runcvm
%pathtor%\Rscript.exe run_semcog_cvtm.R %runfirmsyn% %runlongdist% %runcvtm% %runttexp% %rundashboard% >run_semcog_cvtm_log.txt 2>&1
:: Check for errors, exit and return error code if error
if %errorlevel% neq 0 exit /B %errorlevel%
:: Remove the run_semcog_cvtm.txt file used to pass path information to the CVM
if EXIST "run_semcog_cvtm.txt" del "run_semcog_cvtm.txt" /f /q
:: Add pause to pause the run at the end of the component
::pause
5.2.2 run_semcog_cvtm.R R Script
The run_semcog_cvtm.R script has two relatively simple
steps; it captures the arguments passed from the
run_semcog_cvtm.bat batch file and then uses R’s source
function to run the CV model’s main R script, __Master.R.
The arguments direct the CV model on which model steps to run along with
three optional arguments to run a specific scenario and run year. The
comments in the run_semcog_cvtm.R script, shown in the
figure below, document the eight arguments.
Later steps in the CV model depend on the results from earlier steps. Until all previous steps have been run at least once, a step will not run without error. For example, for the third step of the model, the CVTM, to be run, the firm synthesis model and the long distance model must have been run. Once they have been run, a run of the CV model can be done starting with the CVTM. This functionality allows, for example, the CVTM and trip table steps to be run iteratively in the E8Plus model, using inputs produced by the firm synthesis and long distance model run in the first iteration of the E8Plus model; they do not need to be repeatedly run with each system iteration.
Figure 29: run_semcog_cvtm script
##############################################################################################
#Title: SEMCOG Commercial Vehicle Model
#Project: SEMCOG Commercial Vehicle Model project
#Description: A commercial vehicle simulation model of the SEMCOG region, including Detroit
#Date: 2-28-2022
#Author: Resource Systems Group, Inc.
#Copyright: Copyright 2022 RSG, Inc. - All rights reserved.
##############################################################################################
# 1. Specific scenario arguments from the command line:
# [1] Run Firm Synthesis component (Boolean, TRUE or FALSE, defaults to TRUE)
# [2] Run Long Distance component (Boolean, TRUE or FALSE, defaults to TRUE)
# [3] Run CVTM component (Boolean, TRUE or FALSE, defaults to TRUE)
# [4] Run TT component (Boolean, TRUE or FALSE, defaults to TRUE)
# [5] Run Dashboard component (Boolean, TRUE or FALSE, defaults to TRUE)
# [6] Scenario Name (Character string, must be a valid scenario folder name,
# detaults to base scenario name)
# (not required if writing out "run_semcog_cvtm.txt")
# [7] Scenario Description (Character string, any valid string acceptable,
# detaults to base scenario name)
# (not required if writing out "run_semcog_cvtm.txt")
# [8] Iteration (Integer, defaults to 1)
# (not required if writing out "run_semcog_cvtm.txt")
# [9] Scenario Reference Name (Character string, must be either:
# "Validation" to compare with observed data, or
# a valid scenario folder name to compare with scenario
# Detaults to "Validation")
# (not required if writing out "run_semcog_cvtm.txt")
SYSTEM_COMMAND_ARGS <- commandArgs(TRUE)
# 2. Run the application
source(file.path("lib", "scripts", "__Master.R"))A simple batch file to run the run_semcog_cvtm.R script
can be written and used separately from
run_semcog_cvtm_integrated.bat. This can be used to run the
CV model in a standalone mode independently of the rest of the E8Plus
model. While this approach still requires certain input files produced
by the E8Plus model to be present in the scenario outputs (e.g.,
passenger trip tables, highway skim files) it can be a useful way to run
the model repeatedly when, for example, doing development testing to
evaluate changes to only inputs that affect the CV model.
The following code for a batch file saved in the root of
CVM would run the firm synthesis, long-distance model, and
CVTM for a scenario called “test_2045”. The key difference between this
approach and the approach used in
run_semcog_cvtm_integrated.bat is that the final optional
parameters are used to specify and describe the scenario to run. In
run_semcog_cvtm_integrated.bat, these parameters are not
used as the scenario name, description, and other file path information
is passed to the CV model using the run_semcog_cvtm.txt
configuration file, which is not used in this standalone approach.
Figure 30: Example simple batch file to call run_semcog_cvtm.R
:: set the path to R on this machine
set pathtor="C:\Program Files\R\R-4.1.2\bin\x64"
:: find the location of this batch file
set pathtocvtmmodel=%~dp0
cd /D %pathtocvtmmodel%
:: run the CV model
%pathtor%\Rscript.exe run_semcog_cvtm.R TRUE TRUE TRUE FALSE FALSE test_2045 "Test Scenario for 2045" >run_semcog_cvtm_log.txt 2>&1
pause
5.3 Master
5.3.1 __Master.R R Script
A master controller file, __Master.R acts as the main
interface for the models, initiating the model run and sequentially
running the script files that apply each of the individual model
components. The script is run using R’s source function
from the run_semcog_cvtm.R R
Script. The script contains six sections. The first one, shown in
the figure below, calls the initialization script to start the model
application. The subsequent five sections each run through the scripts
that implement each of the five components of the CV model using a
generally consistent sequence of code. The second figure below shows the
code that runs the firm synthesis model.
The initialize application step begins by using the
source function to run the main initialization script,
init_start_rFreight_model.R. The steps that this script
runs are described in [Initialization]. Once the model is initialized,
the description and name of the scenario being run is printed to the
log.
Figure 31: Master.R code to initialize application
### Initialize Application -------------------------------------------------------------------
# Start the rFreight application
source(file.path("lib", "scripts", "init_start_rFreight_model.R"))
cat("Running the", SCENARIO_DESCRIPTION, "Scenario from the ", SCENARIO_NAME , " Directory", "\n")
SCENARIO_RUN_START <- Sys.time()The structure of the code to run each of the model components is
relatively consistent, with only minor changes depending on the outputs
from the step (e.g., model databases, trip tables, or a dashboard). In
the example below, for the firm synthesis model, a check is first done
on whether the step should be run by checking the value of
SCENARIO_RUN_FIRMSYN, which is set during initialization
based on the arguments passed from TransCAD via the
run_semcog_cvtm_integrated.bat batch file.
Two key script files are then read in,
firm_sim_process_inputs.R is the Input Processing script for firm synthesis,
while firm_sim.R is the Control
script for firm synthesis.
The input processing script is run first, which loads the main
tabular input for firm synthesis into an object called
Establishments and the rest of the inputs into an R
environment called firm_inputs.
The next block of code uses the rFreight function
run_sim to run the firm synthesis model using it’s control
function firm_sim, the Establishments table,
and the firm_inputs environment, and returns the results in
a list object called firm_sim_results.
The results are then saved to a .Rdata database, and also exported to
a set of .csv files, one for each table in
firm_sim_results.
Figure 32: Master.R code to run firm synthesis component
### 1. Firm Synthesis ------------------------------------------------------------------------
if (SCENARIO_RUN_FIRMSYN) {
cat("Starting Firm Synthesis Step", "\n")
# Load executive functions (process inputs and simulation)
source(file = file.path(SYSTEM_SCRIPTS_PATH, "firm_sim_process_inputs.R"))
source(file = file.path(SYSTEM_SCRIPTS_PATH, "firm_sim.R"))
# Process inputs
cat("Processing Firm Synthesis Inputs", "\n")
firm_inputs <- new.env()
Establishments <- firm_sim_process_inputs(envir = firm_inputs)
# Run simulation
cat("Running Firm Synthesis Simulation", "\n")
firm_sim_results <- suppressMessages(run_sim(FUN = firm_sim, data = Establishments,
packages = SYSTEM_PKGS, lib = SYSTEM_PKGS_PATH,
inputEnv = firm_inputs))
# Save inputs and results to Rdata file
cat("Saving Firm Synthesis Database", "\n")
save(firm_sim_results,
firm_inputs,
file = file.path(SCENARIO_OUTPUT_PATH,
SYSTEM_FIRMSYN_OUTPUTNAME))
# Export results to csv files
cat("Writing Firm Synthesis Results to CSV Files", "\n")
lapply(1:length(firm_sim_results),
function(x) fwrite(firm_sim_results[[x]],
file = file.path(SCENARIO_OUTPUT_PATH,
paste(SYSTEM_FIRMSYN_OUTPUTNAME,
names(firm_sim_results)[x],
"csv",
sep = "."))))
# Clean up work space
rm(firm_sim_results, firm_inputs, Establishments)
gc(verbose = FALSE)
}5.4 Intialization
Script names prefixed with “init”, e.g.,
init_start_rfreight_model.R, contain code to start the
model application and support functions such as loading the required R
package function libraries. The two initialization scripts are described
in this section:
- init_start_rFreight_model.R R Script
- [init_install_special_packages.R R Script]
5.4.1 init_start_rFreight_model.R R Script
This script is run from __Master.R and begins the model
initialization process. The content of the script is shown in the figure
below.
The script first reads in the four scripts that contain global variables, described below in Variables. These variables include information about the file structure of the model application, the location of various input files, and which R package libraries to load.
The script continues by installing the rFreight R
package, along with other packages that require a non-standard
installation process. This steps uses the
[init_install_special_packages.R R Script]. The rFreight R
package is then loaded, which provides R with access to many of the
custom functions requires to run the CV model.
Finally, the rFreight function initializeApp is used to
start the CV model application, which includes loading all of the other
R packages that are used to run the model and creating the output
directory for the model outputs if that has not already been
created.
Figure 33: init_start_rFreight_model.R code to initialize the model
# initialization script for rFreight applications
# Load global variables
source(file.path("lib", "scripts", "_SYSTEM_VARIABLES.R"))
source(file.path(SYSTEM_SCRIPTS_PATH, "_BASE_VARIABLES.R"))
source(file.path(SYSTEM_SCRIPTS_PATH, "_SCENARIO_VARIABLES.R"))
source(file.path(SYSTEM_SCRIPTS_PATH, "_USER_VARIABLES.R"))
# Install rFreight and any packages not available on CRAN
source(file.path(SYSTEM_SCRIPTS_PATH, "init_install_special_packages.R"))
# Load current rFreight installation
suppressWarnings(suppressMessages(library(rFreight,
lib.loc = SYSTEM_PKGS_PATH)))
# Check for new rFreight version, load rFreight and other packages, create output folder
initializeApp(rFreight.path = SYSTEM_RFREIGHT_PATH,
output.path = SCENARIO_OUTPUT_PATH,
lib = SYSTEM_PKGS_PATH,
packages = c(SYSTEM_PKGS),
reload.rFreight = FALSE)5.4.2 init_install_special_pacakges.R R Script
This script contains several steps that are required to install and load R packages used by the model.
Most R packages are available for download from the CRAN mirror system of remote servers (https://cran.r-project.org/mirrors.html). The first block of code sets a specific CRAN mirror location to use for any installation of packages that are available on CRAN.
The second part of the scripts deals with the installation of the
rFreight package. While the package is included with the CV
model, ready to be installed from a .zip file at
./CVM/lib/rFreight_0.1-32.zip, there are three packages
that must be installed from CRAN in order for rFreight functions to run
correctly. These are data.table, reshape, and
plyr. If those packages have not previously been installed
on this computer, they are downloaded from CRAN and installed. The
rFreight package is then installed from the .zip file if
that has not been done before.
Finally, two of the packages that support the .omx open matrix format
used for trip table are not available on CRAN but are available on an
alternative package repository, Bioconductor (https://www.bioconductor.org/). The script first
installs a package from CRAN called BiocManager that
contains functions to support installation of packages from Bioconductor
and then installs the required packages rhdf5 and
Rhdf5lib.
5.5 Variables
There are several scripts that contain global variables used through
the CV model. The script naming convention is upper case and starting
with an underscore, e.g. _BASE_VARIABLES.R. The variables
in these scripts are described in this section. The four variables
scripts are shown in the table below:
Table 7: List of model scripts that specify global variables
| StepNumber | StepName | Filename | Description | ScriptType |
|---|---|---|---|---|
| 0 | Initialization | _BASE_VARIABLES.R | List of base variables | Variables |
| 0 | Initialization | _SCENARIO_VARIABLES.R | List of scenario variables | Variables |
| 0 | Initialization | _SYSTEM_VARIABLES.R | List of system variables | Variables |
| 0 | Initialization | _USER_VARIABLES.R | List of user varibales | Variables |
5.5.1 _BASE_VARIABLES.R
The first of the global variables scripts primarily contains
variables that would be not be adjusted by the model user unless they
were making significant alterations to the model, such as updating the
model’s TAZ structure. All of the variables created in this file are
prefixed with BASE_.
The figure below shows the content of the script including descriptions of each of the variables.
Figure 34: BASE VARIABLES Script
# 1. Define TAZ ranges for different elements of the model region.
TAZ_System <- read.csv(file.path(SYSTEM_DATA_PATH, "TAZ_System.csv"))
BASE_TAZ_INTERNAL <- TAZ_System[TAZ_System$INTERNAL == 1, "TAZ"] #range of TAZs that are within the SEMCOG model region including the buffer
BASE_TAZ_EXTERNAL <- TAZ_System[TAZ_System$EXTERNAL == 1, "TAZ"] #range of external station TAZs
BASE_TAZ_MODEL_REGION <- TAZ_System[TAZ_System$SEMCOG == 1, "TAZ"] #range of internal TAZs that are within the SEMCOG model region not including the buffer
BASE_TAZ_BUFFER <- TAZ_System[TAZ_System$BUFFER == 1, "TAZ"] #range of internal TAZs that are in the buffer around the outside of the SEMCOG model region
BASE_TAZ_REGION_EXTERNAL <- c(BASE_TAZ_MODEL_REGION, BASE_TAZ_EXTERNAL) #range of SEMCOG model region TAZs (internal) and externals, buffer excluded
BASE_TAZ_US <- TAZ_System[TAZ_System$COUNTRY == "US" & TAZ_System$INTERNAL == 1, "TAZ"] #range of TAZs (not external stations) that are within the US
BASE_TAZ_CANADA <- TAZ_System[TAZ_System$COUNTRY == "Canada" & TAZ_System$INTERNAL == 1, "TAZ"] #range of TAZs (not external stations) that are in Canada
BASE_TAZ_COMPLETE <- TAZ_System$TAZ #full set of valid TAZs
rm(TAZ_System)
# 2. Define other application parameters
BASE_NEW_FIRMS_PROP <- 0.6 #proportion of growth in employment in already developed TAZs that comes from new firm formation as opposed to existing firm growth
BASE_SEED_VALUE <- 5 #seed for sampling to ensure repeatable results
BASE_TIME_PERIOD_TRIP_POINT <- "MIDDLE" #point in trip for time period allocation, from ("START", "MIDDLE", "END")
BASE_TOLL_SKIM_AVAILABLE <- FALSE #are there toll skims for use in the CVM?
# Penalties applies to skims to reduce likelihood of certain TAZs/externals being used
#(2897 is tunnel, this penalty moves trucks to Ambassador bridge)
BASE_TAZ_PENALTY <- data.frame(TAZ = c(2897),
Penalty = c(10))
# Penalty (units equivalent to miles) used to reduce likelihood of crossing International boundary between US and Canada
# Applied to in the CVTM (local commerical vehicle trips) in the scheduled stops model and intermediate stops model
BASE_INTERNATIONAL_PENALTY <- 100
# 3. Define time-of-day groupings for skims and trip tables
# Units are minutes after midnight
# (12:00 AM = 0 at the start of a range, 1440 at the end of a range)
# Five time periods in the SEMCOG E8 model:
# AM (6:30 AM - 8:59 AM)
# MD (9:00 AM - 2:29 PM)
# PM (2:30 PM - 6:29 PM)
# EV (6:30 PM - 9:59 PM)
# NT (10:00 PM - 6:29 AM)
BASE_TOD_RANGES <- list(NT = list(c(0, 390), c(1320, 1440)),
AM = list(c(390, 540)),
MD = list(c(540, 870)),
PM = list(c(870, 1110)),
EV = list(c(1110, 1320)))
# 4. Define settings used in Dashboard/spreadsheet report
# Column name from TAZ_System.csv that labels each TAZ with the desired group
# names for use in the dashboard. This also determines how TAZs will be grouped
# into larger regions for display in the dashboard.
BASE_DASHBOARD_GEOGRAPHY <- "CountyName"
# Unit used for display in dashboard
BASE_DASHBOARD_LENGTH_UNIT <- "miles"
# Default scenario to use for refence comparisons in the Dashboard/spreadsheet report
# The SCENARIO_REFERENCE_NAME can also be passed as a command argument
# A command argument will which takes precendence over the value entered, see run_semcog_cvtm.R
# Valid settings:
# "Validation" - (quoted string) this will compare the current scenario to observed data for validation purposes
# "<scenario name>" - (quoted string, case sensitive) this will the current scenario to another scenario
# the scenario name is the directory name for the scenario, and the scenario must have
# been run to completion already
BASE_SCENARIO_REFERENCE_NAME <- "Validation"5.5.2 _SCENARIO_VARIABLES.R
The second of the global variables scripts primarily contains
variables that deal with the components of the model to run during a
scenario and the location of files and folders within the scenario
folder. All of the variables created in this file are prefixed with
SCENARIO_.
The scenario control variables that are passed from the TransCAD
E8Plus model are processed in this file, either being read from the
SYSTEM_COMMAND_ARGS object that is created in the
run_semcog_cvtm.R script from the arguments passed to the
CV model, or assigned with default values.
This script also reads and parses into individual variables the
contents of the run_semcog_cvtm.txt configuration file. The
TransCAD macros that create this file for each of the CV model
components are described in GISDK Modifications for
Integration. Since the structure of the configuration file varies
for each CV model component, there is a block of code to process the
configuration file that is specific to each component. When the CV model
is run independently, the run_semcog_cvtm.txt is not
present and so the variables that would otherwise be created using the
values from the configuration file are created using default values
specified in this script.
The figure below shows the content of the script.
Figure 35: SCENARIO VARIABLES Script
# 1. Define any default variables: only used in model testing/stand alone operation,
# values for these will be derived from either command arguments or text configuration file from TransCAD
SCENARIO_DEFAULT_NAME <- "2015_KA15" # base year scenario name in E8 model
# 2. Remainder of the script processes command line and config settings for the set up of this scenario
# If the model was called from the command line
if (exists("SYSTEM_COMMAND_ARGS")) {
if (length(SYSTEM_COMMAND_ARGS) >= 1) SCENARIO_RUN_FIRMSYN <- as.logical(SYSTEM_COMMAND_ARGS[1]) else SCENARIO_RUN_FIRMSYN <- TRUE
if (length(SYSTEM_COMMAND_ARGS) >= 2) SCENARIO_RUN_LDM <- as.logical(SYSTEM_COMMAND_ARGS[2]) else SCENARIO_RUN_LDM <- TRUE
if (length(SYSTEM_COMMAND_ARGS) >= 3) SCENARIO_RUN_CVTM <- as.logical(SYSTEM_COMMAND_ARGS[3]) else SCENARIO_RUN_CVTM <- TRUE
if (length(SYSTEM_COMMAND_ARGS) >= 4) SCENARIO_RUN_TT <- as.logical(SYSTEM_COMMAND_ARGS[4]) else SCENARIO_RUN_TT <- TRUE
if (length(SYSTEM_COMMAND_ARGS) >= 5) SCENARIO_RUN_DB <- as.logical(SYSTEM_COMMAND_ARGS[5]) else SCENARIO_RUN_DB <- TRUE
# Read the scenario name and input paths file
if(file.exists("run_semcog_cvtm.txt")){
scenario_paths_file <- file("run_semcog_cvtm.txt")
scenario_paths <- readLines(scenario_paths_file)
close(scenario_paths_file)
SCENARIO_DESCRIPTION <- scenario_paths[1]
SCENARIO_INPUT_DIR <- scenario_paths[2]
SCENARIO_OUTPUT_DIR <- scenario_paths[3]
SCENARIO_ITERATION <- scenario_paths[4]
if (scenario_paths[5] == "Firm Synthesis"){
SCENARIO_TAZSE <- scenario_paths[6]
SCENARIO_TAZSEBUFFER <- scenario_paths[7]
SCENARIO_BASEFIRMS <- scenario_paths[8]
SCENARIO_BASESCENARIO <- scenario_paths[9]
# Check the value of SCENARIO_BASESCENARIO
stopifnot(as.integer(SCENARIO_BASESCENARIO) %in% c(0,1))
}
if (scenario_paths[5] == "Long Distance"){
SCENARIO_FACILITIES <- scenario_paths[6]
SCENARIO_CVM_IEEI <- scenario_paths[7]
SCENARIO_CVM_EE <- scenario_paths[8]
SCENARIO_CVM_EXTERNAL <- scenario_paths[9]
SCENARIO_LD_SKIM_DIST <- scenario_paths[10]
SCENARIO_LD_SKIM_PATH <- scenario_paths[11]
SCENARIO_BASESCENARIO <- scenario_paths[12]
# Check the value of SCENARIO_BASESCENARIO
stopifnot(as.integer(SCENARIO_BASESCENARIO) %in% c(0,1))
}
if (scenario_paths[5] == "CV Touring Model"){
SCENARIO_CV_SKIM_TIME <- scenario_paths[6]
SCENARIO_CV_SKIM_DIST <- scenario_paths[7]
SCENARIO_CV_SKIM_TOLL <- scenario_paths[8]
SCENARIO_CVM_EXTERNAL <- scenario_paths[9]
SCENARIO_CV_SKIM_PATHS <- sapply(1:length(BASE_TOD_RANGES),
function(x) scenario_paths[9 + x])
}
if (scenario_paths[5] == "CV Trip Tables"){
SCENARIO_ASSIGN_PATH <- dirname(scenario_paths[6])
}
if (scenario_paths[5] == "CV Dashboard"){
SCENARIO_DB_SPREADSHEET <- as.logical(scenario_paths[6])
SCENARIO_ASSIGN_FLOWS_PATH <- dirname(scenario_paths[7])
# The reference scenario is not currently being passed from TransCAD via the text file so set here
if(SCENARIO_DB_SPREADSHEET) SCENARIO_REFERENCE_NAME <- BASE_SCENARIO_REFERENCE_NAME
}
# Split the input and output directory paths and create path components
SCENARIO_INPUT_DIR_SPLIT <- unlist(strsplit(SCENARIO_INPUT_DIR, split = "\\\\"))
SCENARIO_OUTPUT_DIR_SPLIT <- unlist(strsplit(SCENARIO_OUTPUT_DIR, split = "\\\\"))
SCENARIO_NAME <- SCENARIO_INPUT_DIR_SPLIT[length(SCENARIO_INPUT_DIR_SPLIT)-1]
SCENARIO_INPUT_DIR_NAME <- SCENARIO_INPUT_DIR_SPLIT[length(SCENARIO_INPUT_DIR_SPLIT)]
SCENARIO_OUTPUT_DIR_NAME <- SCENARIO_OUTPUT_DIR_SPLIT[length(SCENARIO_OUTPUT_DIR_SPLIT)]
SCENARIO_FOLDER_PATH <- paste(SCENARIO_INPUT_DIR_SPLIT[1:(length(SCENARIO_INPUT_DIR_SPLIT)-2)], collapse = "/")
rm(scenario_paths)
} else {
# Additional command arguments can be passed instead of using text file
if (length(SYSTEM_COMMAND_ARGS) >= 6) SCENARIO_NAME <- as.character(SYSTEM_COMMAND_ARGS[6]) else SCENARIO_NAME <- SCENARIO_DEFAULT_NAME
if (length(SYSTEM_COMMAND_ARGS) >= 7) SCENARIO_DESCRIPTION <- as.character(SYSTEM_COMMAND_ARGS[7]) else SCENARIO_DESCRIPTION <- SCENARIO_DEFAULT_NAME
if (length(SYSTEM_COMMAND_ARGS) >= 8) SCENARIO_ITERATION <- as.integer(SYSTEM_COMMAND_ARGS[8]) else SCENARIO_ITERATION <- 1L
if (length(SYSTEM_COMMAND_ARGS) >= 9) SCENARIO_REFERENCE_NAME <- SYSTEM_COMMAND_ARGS[9] else SCENARIO_REFERENCE_NAME <- BASE_SCENARIO_REFERENCE_NAME
}
} else {
# No command line arguments (e.g. default call to run base scenario)
SCENARIO_NAME <- SCENARIO_DEFAULT_NAME
SCENARIO_DESCRIPTION <- SCENARIO_DEFAULT_NAME
SCENARIO_ITERATION <- 1L
SCENARIO_RUN_FIRMSYN <- TRUE
SCENARIO_RUN_LDM <- TRUE
SCENARIO_RUN_CVTM <- TRUE
SCENARIO_RUN_TT <- TRUE
SCENARIO_RUN_DB <- TRUE
SCENARIO_REFERENCE_NAME <- BASE_SCENARIO_REFERENCE_NAME
}
# Build the paths to scenario input, scenario output, base output, and log file locations
# If paths not passed assume that scenarios are in "Model Runs" directory
if(!exists("SCENARIO_FOLDER_PATH")) SCENARIO_FOLDER_PATH <- file.path(SYSTEM_INTEGRATED_PATH, "Model Runs")
if(!exists("SCENARIO_INPUT_DIR")) SCENARIO_INPUT_DIR <- file.path(SCENARIO_FOLDER_PATH, SCENARIO_NAME, "Input")
if(!exists("SCENARIO_OUTPUT_DIR")) SCENARIO_OUTPUT_DIR <- file.path(SCENARIO_FOLDER_PATH, SCENARIO_NAME, "Output")
if(!exists("SCENARIO_OUTPUT_DIR_NAME")) SCENARIO_OUTPUT_DIR_NAME <- "Output"
SCENARIO_INPUT_PATH <- file.path(SCENARIO_INPUT_DIR, "SED")
SCENARIO_SKIM_PATH <- file.path(SCENARIO_OUTPUT_DIR, "HSkm")
SCENARIO_OUTPUT_PATH <- file.path(SCENARIO_OUTPUT_DIR, "CVM")
SCENARIO_DEFAULT_OUTPUT_PATH <- file.path(SCENARIO_FOLDER_PATH, SCENARIO_DEFAULT_NAME, SCENARIO_OUTPUT_DIR_NAME, "CVM")
SCENARIO_LOG_PATH <- file.path(SCENARIO_OUTPUT_PATH, "log")
# Create file specific paths if those are missing
if (!exists("SCENARIO_TAZSE") & SCENARIO_RUN_FIRMSYN) {
if(SCENARIO_NAME == SCENARIO_DEFAULT_NAME) {
SCENARIO_TAZSE <- file.path(SCENARIO_INPUT_PATH, "base_transcad_taz.csv")
} else {
SCENARIO_TAZSE <- file.path(SCENARIO_INPUT_PATH, paste0(substr(SCENARIO_NAME,1,4), "_transcad_taz.csv"))
}
}
if (!exists("SCENARIO_TAZSEBUFFER") & SCENARIO_RUN_FIRMSYN) SCENARIO_TAZSEBUFFER <- file.path(SCENARIO_INPUT_PATH, "TAZSocioEconomicsBuffer.csv")
if (!exists("SCENARIO_BASEFIRMS") & SCENARIO_RUN_FIRMSYN) SCENARIO_BASEFIRMS <- file.path(SCENARIO_DEFAULT_OUTPUT_PATH, SYSTEM_FIRMSYN_OUTPUTNAME)
if (!exists("SCENARIO_BASESCENARIO") & (SCENARIO_RUN_FIRMSYN | SCENARIO_RUN_LDM)) SCENARIO_BASESCENARIO <- 1L
if (!exists("SCENARIO_FACILITIES") & SCENARIO_RUN_LDM) SCENARIO_FACILITIES <- file.path(SCENARIO_INPUT_PATH, "Facilities.csv")
if (!exists("SCENARIO_CVM_IEEI") & SCENARIO_RUN_LDM) SCENARIO_CVM_IEEI <- file.path(SCENARIO_INPUT_PATH, "CVM_IEEI_Region.csv")
if (!exists("SCENARIO_CVM_EE") & SCENARIO_RUN_LDM) SCENARIO_CVM_EE <- file.path(SCENARIO_INPUT_PATH, "CVM_EE_Region.csv")
if (!exists("SCENARIO_CVM_EXTERNAL") & (SCENARIO_RUN_LDM | SCENARIO_RUN_CVTM)) SCENARIO_CVM_EXTERNAL <- file.path(SCENARIO_INPUT_PATH, "CVM_Externals.csv")
if (!exists("SCENARIO_LD_SKIM_DIST") & SCENARIO_RUN_LDM) SCENARIO_LD_SKIM_DIST <- "Miles (orig)"
if (!exists("SCENARIO_LD_SKIM_PATH") & SCENARIO_RUN_LDM) SCENARIO_LD_SKIM_PATH <- file.path(SCENARIO_SKIM_PATH, "MD_HwySkim.omx")
if (!exists("SCENARIO_CV_SKIM_TIME") & SCENARIO_RUN_CVTM) SCENARIO_CV_SKIM_TIME <- "Trav_Time"
if (!exists("SCENARIO_CV_SKIM_DIST") & SCENARIO_RUN_CVTM) SCENARIO_CV_SKIM_DIST <- "Miles (orig)"
if (!exists("SCENARIO_CV_SKIM_TOLL") & SCENARIO_RUN_CVTM) SCENARIO_CV_SKIM_TOLL <- NA
if (!exists("SCENARIO_CV_SKIM_PATHS") & SCENARIO_RUN_CVTM) {
SCENARIO_CV_SKIM_PATHS <- file.path(SCENARIO_SKIM_PATH,
paste(names(BASE_TOD_RANGES), "HwySkim.omx", sep = "_"))
}
if (!exists("SCENARIO_ASSIGN_PATH") & SCENARIO_RUN_TT) SCENARIO_ASSIGN_PATH <- file.path(SCENARIO_OUTPUT_DIR, "HAssign")
if (!exists("SCENARIO_ASSIGN_FLOWS_PATH") & SCENARIO_RUN_DB) SCENARIO_ASSIGN_FLOWS_PATH <- file.path(SCENARIO_OUTPUT_DIR, "HAssign")
if (!exists("SCENARIO_DB_SPREADSHEET") & SCENARIO_RUN_DB) SCENARIO_DB_SPREADSHEET <- TRUE
if (!exists("SCENARIO_REFERENCE_NAME") & SCENARIO_RUN_DB) SCENARIO_REFERENCE_NAME <- BASE_SCENARIO_REFERENCE_NAME
# Switches for the Dashboard components: TRUE and the content for that model step is rendered in the dashboard
SCENARIO_DB_FIRMSYN <- TRUE
SCENARIO_DB_LDM <- TRUE
SCENARIO_DB_CVTM <- TRUE
SCENARIO_DB_TT <- TRUE5.5.3 _SYSTEM_VARIABLES.R
The third of the global variables scripts primarily contains
variables that deal with the setup of the modeling system. This includes
R packages used in different parts of the model, the folder structure
used in the model, and the naming of the main outputs from the model.
All of the variables created in this file are prefixed with
SYSTEM_.
The figure below shows the content of the script.
Figure 36: SYSTEM VARIABLES Script
# R packages required by the model
SYSTEM_PKGS <- c("bit", "bit64", "data.table", "fastcluster", "withr", "ggplot2", "pscl",
"reshape", "reshape2", "rFreight", "TSP", "sp", "rgdal", "rgeos", "rhdf5",
"leaflet", "jsonlite", "apollo")
SYSTEM_REPORT_PKGS <- c("DT", "flexdashboard", "leaflet", "plotly", "scales",
"pander", "geojsonlint", "stringr", "kableExtra", "openxlsx")
SYSTEM_DEV_PKGS <- c("sf", "lwgeom", "dplyr", "ggrepel",
"ggspatial", "leaps", "tools", "foreign")
# combine list of system and system report packages so all install if needed on call to initializeApp
# leave the list of packages used in development seperate, not required for application
SYSTEM_PKGS <- c(SYSTEM_PKGS, SYSTEM_REPORT_PKGS[!SYSTEM_REPORT_PKGS %in% SYSTEM_PKGS])
# paths to main application directories
SYSTEM_APP_PATH <- getwd()
SYSTEM_INTEGRATED_PATH <- dirname(getwd())
SYSTEM_RFREIGHT_PATH <- file.path(SYSTEM_APP_PATH, "lib", "rFreight_0.1-33.zip")
SYSTEM_PKGS_PATH <- file.path(SYSTEM_APP_PATH, "lib", "pkgs", "library")
# add the library folder to the library search paths -- required for proper
# install and loading, especially with future
.libPaths(c(SYSTEM_PKGS_PATH, .libPaths()))
Sys.setenv(R_LIBS = paste(SYSTEM_PKGS_PATH, Sys.getenv("R_LIBS"),
sep=.Platform$path.sep))
SYSTEM_DATA_PATH <- file.path(SYSTEM_APP_PATH, "lib", "data")
SYSTEM_SCRIPTS_PATH <- file.path(SYSTEM_APP_PATH, "lib", "scripts")
SYSTEM_DEV_PATH <- file.path(SYSTEM_APP_PATH, "dev")
SYSTEM_DOCS_PATH <- file.path(SYSTEM_INTEGRATED_PATH, "docs")
# Standard model component databases
SYSTEM_FIRMSYN_OUTPUTNAME <- "1.Firms.RData"
SYSTEM_LDM_OUTPUTNAME <- "2.LongDistanceTrips.RData"
SYSTEM_CVTM_OUTPUTNAME <- "3.CommercialVehicleTrips.RData"
SYSTEM_TT_OUTPUTNAME <- "4.TripTables.RData"
SYSTEM_DB_OUTPUTNAME <- "5.DashboardTables.RData"5.5.4 _USER_VARIABLES.R
The fourth of the global variables scripts contains variables that
deal with how the user wishes to run the model on their computer, using
single or multiple cores for the parts of the model that are
multi-threaded, and whether they wish to run the model in calibration or
application mode. For most day to day model runs, this file would not be
changed after it was initially set up for the specific computer that the
model is installed on, as described in Configure the CV Model. All of the
variables created in this file are prefixed with USER_.
The figure below shows the content of the script.
Figure 37: USER VARIABLES Script
# User options to control hardware use, and run mode
USER_PROCESSOR_CORES <- 5L # How many processors should be used during model runs?
# (a number, followed by L to indicate integer)
USER_RUN_MODE <- "Application" # What type of run is being done?
# Options are "Application" or "Calibration"
# where calibration triggers certain model components to run iterative adjustments
USER_SAVE_COMBINED_OD_TABLES <- FALSE # Should OD tables also be saved in
# a single combined trip table in CVM outputs?
# This take ~5 minutes per iteration, not necessary typically
USER_SAVE_OUTPUTS_ITERATION <- FALSE # Should CVTM and Trip table outputs by exported for every system iteration?
# If TRUE, each CVTM and Trip table database CSV is saved
# with the filename appended with the iteration number
# (typically E8 model is run for 5 iterations)
USER_SPREADSHEET_SUMMARIES <- TRUE # Should dashboard component include additional reporting to a spreadsheet?
# If TRUE, a spreadsheet report is written.
# WARNING requires exported flow csv files for assignment summaries,
# these will only be up to date if dashboard step is run after
# full system model run is completed (can be run as a single step from the GUI)
# Note that call from integrated model to build dashboard in mid-run
# will override setting (if it is set to TRUE)
# as assignment results will be inconsistent with the rest of the results
# This is not the case when the reporting portion of the integrated model is used
# and the spreadsheet summaries are selected5.6 Control
Each CV model component is run by a control script that either runs a
model simulation or builds an output. In the case of a model simulation,
the script is named after the model component suffixed with “_sim”,
e.g., ld_sim.R for the long-distance model control script.
for the model steps that build an output, the script is named after the
model component suffixed with “_build”, e.g., tt_build.R
for the trip tables control script.
The five control scripts are shown in the table below:
Table 8: List of control scripts
| StepNumber | StepName | Filename | Description | ScriptType |
|---|---|---|---|---|
| 1 | Firm Synthesis | firm_sim.R | Firm synthesis control function | Control |
| 2 | Long Distance | ld_sim.R | Long distance control function | Control |
| 3 | CV Touring | cv_sim.R | CVTM control function | Control |
| 4 | Trip Tables | tt_build.R | Trip table building function | Control and Model Step |
| 5 | Dashboard | db_build.R | Dashboard control function | Control |
5.6.1 Control Script Example: ld_sim.R
An example control script is ld_sim.R, the control
script for the long-distance model. The script is organized as a
function that can be called from __Master.R and
sequentially run the individual model steps of the long distance model.
The model steps are each contained inside an R function, e.g.,
ld_sim_generation() is the long distance trip generation
step.
There are additional code elements in the control function:
- Progress logging with functions
progressStart()andprogressEnd() - Calibration support using the
run_stepobject that in calibration mode will be set so that just a single model step is run from the full set of steps, allowing for step by step calibration. - Return of results dependents on the run mode, where in calibration just the results from a particular step are returned and in normal application a list of model output tables is returned.
The figure below shows the code in ld_sim.R.
Figure 38: Control Script Example, ld_sim.R
# Simulate [l]ong [d]istance freight movements
ld_sim <- function(data = NULL) {
# Begin progress tracking
progressStart(action = "Simulating...", task = "Long Distance Movements", dir = SCENARIO_LOG_PATH, subtasks = FALSE)
# Define run_steps if it is not already in the environment (default to running all steps)
if(!exists("run_step")) run_step <- rep(TRUE, 4)
if(run_step[1]){
# Perform trip generation to produce productions and attractions
cat("Estimating Long Distance Productions and Attractions", "\n")
zoneProdsAttrs <- ld_sim_generation(TAZLandUseCVTM = TAZLandUseCVTM,
TAZ_System = TAZ_System,
Facilities = Facilities,
skims_dist = skims_dist,
ld_internal_gen_model = ld_internal_gen_model,
ld_external_model = ld_external_model,
ld_ieei_dailytrucks_region = ld_ieei_dailytrucks_region,
ld_externals = ld_externals)
gc()
}
if(run_step[2]){
# Perform trip distribution to produce IE and EI zone-to-zone trip totals
cat("Distributing Long Distance Trips", "\n")
IE_EI_Trips <- ld_sim_distribution(zoneProdsAttrs = zoneProdsAttrs,
skims_dist = skims_dist,
ld_distribution_model = ld_distribution_model)
gc()
}
if(run_step[3]){
# Scale observed truck flows from base year to find EE zone-to-zone trip totals
cat("Forecasting Long Distance External to External Trips", "\n")
EE_Trips <- ld_sim_throughtrips(ld_ee_trip_table = ld_ee_trip_table,
ld_ee_dailytrucks_region = ld_ee_dailytrucks_region,
ld_externals = ld_externals)
gc()
}
if(run_step[4]){
# Split long distance trips by vehicle type and time-of-day
cat("Allocating Long Distance Trips to Vehicles and Time of Day", "\n")
ld_trips <- ld_sim_veh_type_time_split(IE_EI_Trips = IE_EI_Trips,
EE_Trips = EE_Trips,
ld_trip_props = ld_trip_props)
gc()
}
# End progress tracking
progressEnd(dir = SCENARIO_LOG_PATH)
if(USER_RUN_MODE == "Calibration"){
return(get(submodel_results_name))
} else {
return(list(ld_trips = ld_trips,
zoneProdsAttrs = zoneProdsAttrs))
}
}5.7 Input Processing
Each CV model component contains a script that loads the input files
used by the model component into an environment in memory and does some
initial processing. This script is run as the first step of each model
component. These scripts have the same name as the control script for
the model component with the additional suffix of “_process_inputs”,
e.g., ld_sim_process_inputs.R for the long-distance model’s
input processing function.
The four input processing scripts are shown in the table below:
Table 9: List of inputs processing scripts
| StepNumber | StepName | Filename | Description | ScriptType |
|---|---|---|---|---|
| 1 | Firm Synthesis | firm_sim_process_inputs.R | Firm synthesis input processing function | Input Processing |
| 2 | Long Distance | ld_sim_process_inputs.R | Long distance input processing function | Input Processing |
| 3 | CV Touring | cv_sim_process_inputs.R | CVTM input processing function | Input Processing |
| 4 | Trip Tables | tt_build_process_inputs.R | Trip table input processing function | Input Processing |
| 5 | Dashboard | db_build_process_inputs.R | Dashboard input processing function | Input Processing |
5.7.1 Input Processing Script Example: firm_sim_process_inputs.R
An example input progressing script,
firm_sim_process_inputs.R, is the input processing script
for the firm synthesis model. The script is organized as a function that
can be called from __Master.R .
The sections of the script complete the following tasks:
- Loading common data and model parameters inputs, generally from
CVM/lib/data - Processing the common inputs, in this case creating a unique correspondence between the model’s employment categories and more aggregate employment groups.
- Loading the scenario input files, i.e., inputs that are specific to this scenario, such as the TAZ employment data.
- Processing the scenario inputs, in this case combining the internal employment and household data with the data for the buffer and creating a summarized version or modeling using the grouped employment categories.
- Organizing the data into a separate table object for the key simulation data (the establishment data, in this case), with the remainder of the data held together in an R environment. This organizational approach is required for parallelization, where the separate table is split into chunks that are simulated independently with the rest of the inputs copied to each of the parallel processes.
The figure below shows the code in
firm_sim_process_inputs.R
Figure 39: Input Processing Script Example, firm_sim_process_inputs.R
# This function loads all necessary inputs into envir, after any needed transformations
firm_sim_process_inputs <- function(envir) {
### Load project input files
project.files <- c(EstSizeCategories = file.path(SYSTEM_DATA_PATH, "EstSizeCategories.csv"),
Establishments = file.path(SYSTEM_DATA_PATH, "Establishments.csv"),
NAICS3_to_EmpCats = file.path(SYSTEM_DATA_PATH, "NAICS3_to_EmpCats.csv"),
TAZ_System = file.path(SYSTEM_DATA_PATH, "TAZ_System.csv"))
loadInputs(files = project.files, envir = envir)
### Process project input files
envir[["UEmpCats"]] <- unique(envir[["NAICS3_to_EmpCats"]][,.(EmpCatID, EmpCatName, EmpCatDesc, EmpCatGroupedName)])
### Load scenario input files
scenario.files <- c(TAZSocioEconomicsRawTAZData = SCENARIO_TAZSE,
TAZSocioEconomicsBuffer = SCENARIO_TAZSEBUFFER)
loadInputs(files = scenario.files, envir = envir)
### Process scenario input files
envir[["TAZSocioEconomicsInternal"]] <- envir[["TAZSocioEconomicsRawTAZData"]][, c("taz_id",
"tot_hhs",
"tot_pop",
sort(unique(envir[["UEmpCats"]]$EmpCatName))),
with = FALSE]
setnames(envir[["TAZSocioEconomicsInternal"]],
c("taz_id", "tot_hhs", "tot_pop"),
c("TAZ", "HH", "POP"))
envir[["TAZSocioEconomics"]] <- rbind(envir[["TAZSocioEconomicsInternal"]],
envir[["TAZSocioEconomicsBuffer"]])
envir[["TAZSocioEconomics"]] <- processTAZSocioEconomics(TAZSE = envir[["TAZSocioEconomics"]])
envir[["TAZEmployment"]] <- extractTAZEmployment(TAZSE = envir[["TAZSocioEconomics"]],
employment.regexpr = "^e[0-9][0-9]_")
envir[["TAZLandUseCVTM"]] <- summarizeTAZLandUse(envir[["TAZSocioEconomics"]],
EmpCats = envir[["UEmpCats"]]$EmpCatName,
GroupingCats = envir[["UEmpCats"]]$EmpCatGroupedName,
AddEmpTotal = TRUE,
TotalEmpName = "TotalEmp",
AddlFields = "HH",
AddlFieldNames = "TotalHHs")
### Return the Establishments table
Establishments <- envir[["Establishments"]]
rm(Establishments, envir = envir)
return(Establishments)
}5.8 Model Step
The control script for each CV model component can call additional
scripts that contain functions implementing individual model steps.
These scripts have the same name as the control script for the model
component with an additional suffix describing the model step, e.g.,
ld_sim_generation.R which contains the function to
implement the long-distance model’s trip generation model.
The four model step scripts are shown in the table below:
Table 10: List of model step scripts
| StepNumber | StepName | Filename | Description | ScriptType |
|---|---|---|---|---|
| 2 | Long Distance | ld_sim_distribution.R | Long distance trip distribution model | Model Step |
| 2 | Long Distance | ld_sim_generation.R | Long distance trip generation model | Model Step |
| 2 | Long Distance | ld_sim_throughtrips.R | Long distance through trip scaling function | Model Step |
| 2 | Long Distance | ld_sim_veh_type_time_split.R | Long distance vehicle type and time-of-day splitting function | Model Step |
| 3 | CV Touring | cv_sim_activities.R | CVTM activities model | Model Step |
| 3 | CV Touring | cv_sim_externalstations.R | CVTM external stations for I-E and E-I trips | Model Step |
| 3 | CV Touring | cv_sim_intermediatestops.R | CVTM non-scheduled stops model | Model Step |
| 3 | CV Touring | cv_sim_scheduledstops.R | CVTM stop generation model | Model Step |
| 3 | CV Touring | cv_sim_scheduledtrips.R | CVTM trip scheduling model | Model Step |
| 3 | CV Touring | cv_sim_stopduration.R | CVTM stop duration model | Model Step |
| 3 | CV Touring | cv_sim_tours.R | CVTM tour model (clustering and sequencing) | Model Step |
| 3 | CV Touring | cv_sim_vehicle.R | CVTM vehicle choice model | Model Step |
| 4 | Trip Tables | tt_build.R | Trip table building function | Control and Model Step |
| 5 | Dashboard | db_build_render.R | Dashboard rendering and post-processing | Model Step |
| 5 | Dashboard | db_build_spreadsheet.R | Spreadsheet summary of model outputs | Model Step |
5.8.1 Model Step Script Example: cv_sim_activities.R
An example model step script iscv_sim_activities.R, the
model step script for the firm activities step of the CVTM. The script
is organized as a function that can be called from cv_sim.R
.
The sections of the script complete the following tasks:
- Adds a progress log record using the
progressUpdate()function. - Merges the list of firms with the
cv_activities_modeltable to the activity proportions from the model. - Simulates the activity of each firm using a seeded random draw.
- Since some firms make both goods and service trips and some do neither, add duplicate records for those firms that do both and remove entirely those firms that do neither.
- Simplify the table to return just a list of firm IDs and their activities from the function.
The figure below shows the R code of
cv_sim_activities.R.
Figure 40: Model Step Script Example, cv_sim_activities.R
# Commercial Vehicle Scheduled Activities Simulation
cv_sim_activities <- function(firms, cv_activities_model) {
progressUpdate(subtaskprogress = 0, subtask = "Firm Activities", prop = 1/7, dir = SCENARIO_LOG_PATH)
# Add cumulative shares for goods and services by model employment category
firmActivities <- merge(firms[,.(BusID, EmpCatGroupedName)],
cv_activities_model[, .(EmpCatGroupedName, Goods, Service = Goods + Service, GoodsAndService = Goods + Service + GoodsAndService)],
by = "EmpCatGroupedName")
# Simulate a tour type (Goods, Service, or Both)
set.seed(BASE_SEED_VALUE)
firmActivities[, TMP_RAND := runif(.N)]
firmActivities[, Activity := ifelse(TMP_RAND < Goods, "Goods",
ifelse(TMP_RAND < Service, "Service",
ifelse(TMP_RAND < GoodsAndService, "GoodsAndService", "Other")))]
# Duplicate records for establishments making both goods and service tours
# Remove other, those are businesses that do not make any commercial vehicle trips covered by the model
firmActivities <- rbind(firmActivities[Activity %in% c("Goods", "Service")],
firmActivities[Activity == "GoodsAndService"][, Activity := "Goods"],
firmActivities[Activity == "GoodsAndService"][, Activity := "Service"])
# Reduce table to just BusID and activity, convert activity to factor, and key
firmActivities <- firmActivities[, .(BusID, Activity = factor(Activity))]
setkey(firmActivities, BusID, Activity)
progressUpdate(subtaskprogress = 1, subtask = "Firm Activities", prop = 1/7, dir = SCENARIO_LOG_PATH)
return(firmActivities)
}6 Data Inputs
6.1 Introduction
The tabular and parameter inputs to the CV model are in two locations:
Inputs that are common across all model scenarios are located in the
CVM/lib/datadirectoryInputs for a particular scenario are included in the
Model Runs/[scenario name]directory
The first group, the common inputs, are listed in the first table below, while the scenario specific inputs are listed in the second table below. The files are mainly comma separated variables (.csv) format files or encoded into one of R’s binary formats for either single objects (.rds) or a work space capable of holding multiple objects (.Rdata), but there are also ESRI shapefiles (.shp) files of TAZ layers and .omx matrices.
The scenario specific inputs include static input file prepared before a scenario is run and those produced during a run such as skims matrices.
The data inputs are described in turn in this chapter.
Table 11: List of inputs that are common across scenarios
| StepNumber | StepName | Filename | Description | InputType |
|---|---|---|---|---|
| 1 | Firm Synthesis | EstSizeCategories.csv | Establishment size categories for Firm Synthesis | Category definitions and correspondences |
| 1 | Firm Synthesis | Establishments.csv | List of establishments for Firm Synthesis | Tabular data |
| 1 | Firm Synthesis | NAICS3_to_EmpCats.csv | Correspondence between NAICS codes and employment Categories | Category definitions and correspondences |
| 1 | Firm Synthesis | TAZ_System.csv | TAZ descriptions, tabular form of shapefile attribute data | Tabular data |
| 2 | Long Distance | ld_distribution_model.csv | Long distance distribution model parameters | Model coefficients and parameters |
| 2 | Long Distance | ld_ee_trip_table.csv | Long distance static external-to-external trip table | Tabular data |
| 2 | Long Distance | ld_external_model.csv | Long distance external station allocation model | Model coefficients and parameters |
| 2 | Long Distance | ld_internal_gen_model.RDS | Long distance generations model coefficients for internal zones | Model coefficients and parameters |
| 2 | Long Distance | ld_trip_time_vehicle_proportions.csv | Long distance trip proportions by vehicle type and time-of-day | Model coefficients and parameters |
| 3 | CV Touring | TravelTimeSkimBuffer.omx | Travel time skims for the buffer TAZs | Matrix data |
| 3 | CV Touring | cv_activities_model.RDS | Commercial Vehicle activities model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_arrival_model.RDS | Commercial Vehicle arrival model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_goods_model.RDS | Commercial Vehicle goods stops model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_intermediate_deviations.rds | Commercial Vehicle non-scheduled stop deviation thresholds | Model coefficients and parameters |
| 3 | CV Touring | cv_intermediate_model.rds | Commercial Vehicle non-scheduled stop generation model | Model coefficients and parameters |
| 3 | CV Touring | cv_intermediate_model_attraction.rds | Commercial Vehicle non-scheduled stop location model | Model coefficients and parameters |
| 3 | CV Touring | cv_service_model.RDS | Commercial Vehicle service stops model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_settings.RData | Commercial Vehicle model parameters | Model coefficients and parameters |
| 3 | CV Touring | cv_stopduration_model.RDS | Commercial Vehicle stop duration model coefficients | Model coefficients and parameters |
| 3 | CV Touring | cv_tours_model.rds | Commerical Vehicle tour type model | Model coefficients and parameters |
| 3 | CV Touring | cv_vehicle_model.RDS | Commercial Vehicle vehicle choice model coefficients | Model coefficients and parameters |
| 5 | Dashboard | TAZ_System_Shape.dbf | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | TAZ_System_Shape.prj | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | TAZ_System_Shape.shp | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | TAZ_System_Shape.shx | Shapefile component for TAZ shapefile | Spatial data |
| 5 | Dashboard | counts_daily_class.csv | Count data containing daily counts for validation spreadsheet | Tabular data |
| 5 | Dashboard | counts_time_period.csv | Count data containing time period counts for validation spreadsheet | Tabular data |
Table 12: List of inputs that are scenario specific
| Filename | Location | Description |
|---|---|---|
| CVM_EE_Region.csv | Input/SED | External region to external region growth factors |
| CVM_Externals.csv | Input/SED | External stations active in the scenario, and external allocation factors for trucks |
| CVM_IEEI_Region.csv | Input/SED | Internal to external and external to internal external region control totals |
| Facilities.csv | Input/SED | Descriptions of high-intensity freight locations |
| TAZSocioEconomicsBuffer.csv | Input/SED | Employment, households, and population data by TAZ for the buffer TAZs |
| base_transcad_taz.csv | Input/SED | Employment, households, and population data by TAZ for the SEMCOG internal TAZs |
6.2 Common Inputs
In this section, each common input to the model, as contained in the
CVM/lib/data directory, is described.
6.2.1 counts_daily_class.csv
File description: Count data containing daily counts for validation spreadsheet
This file is lists the complete set of daily counts processed by
SEMCOG for the validation of the CV model. The file is used to create
validation summaries displayed in the validation summary spreadsheet.
The table below shows the first few rows of
counts_daily_class.csv.
Table 13: First few rows of counts_daily_class.csv
The table below describes the fields in
counts_daily_class.csv.
Table 14: Field Descriptions for counts_daily_class.csv
| Field | Description |
|---|---|
| LINKSERIAL | Network Link ID |
| USECOUNT | Flag for whether the county was used in data expansion |
| LOCAL_ID | Count station ID |
| AADT_YEAR | County year |
| AADT | Total AADT |
| AADT_CAR | AADT for light vehicles |
| AADT_TRUCK | AADT for trucks (medium and heavy |
| AADT_SUT | AADT for single unit trucks |
| AADT_MUT | AADT for multi-unit trucks |
| FCLASS | Functional class code |
| FCLASS_LABEL | Functional class label |
| AREATYPE | Area type code |
| AREATYPE_LABEL | Area type label |
| FCLASS_GROUP | Functional class grouping |
| COUNTY | County code |
| CountyName | County Name |
| VOLUME_GROUP_MUT | Volume group for multi-unit trucks |
| VOLUME_GROUP_SUT | Volume group for single unit trucks |
| VOLUME_GROUP_TRUCK | Volume group for total trucks |
6.2.2 counts_time_period.csv
File description: Count data containing time period counts for validation spreadsheet
This file is lists the complete set of time period counts processed
by SEMCOG for the validation of the CV model. The file is used to create
validation summaries displayed in the validation summary spreadsheet.
The table below shows the first few rows of
counts_time_period.csv.
Table 15: First few rows of counts_time_period.csv
The table below describes the fields in
counts_time_period.csv.
Table 16: Field Descriptions for counts_time_period.csv
| Field | Description |
|---|---|
| LINKSERIAL | Network Link ID |
| USECOUNT | Flag for whether the county was used in data expansion |
| LOCAL_ID | Count station ID |
| AADT_YEAR | County year |
| AADT | Total AADT |
| AADT_CAR | AADT for light vehicles |
| AADT_TRUCK | AADT for trucks (medium and heavy |
| AADT_SUT | AADT for single unit trucks |
| AADT_MUT | AADT for multi-unit trucks |
| FCLASS | Functional class code |
| FCLASS_LABEL | Functional class label |
| AREATYPE | Area type code |
| AREATYPE_LABEL | Area type label |
| FCLASS_GROUP | Functional class grouping |
| COUNTY | County code |
| CountyName | County Name |
| VOLUME_GROUP_MUT | Volume group for multi-unit trucks |
| VOLUME_GROUP_SUT | Volume group for single unit trucks |
| VOLUME_GROUP_TRUCK | Volume group for total trucks |
| TOD | Time Period |
| NumCountsParent | Number of Counts Parent |
| NumCounts | Number of Counts |
| SUT | Single Unit Truck Count for Iime Period |
| MUT | Multi-Unit Truck Count for Time Period |
| PctSUT | Percent of daily count for time period, Single Unit Truck |
| PctMUT | Percent of daily count for time period, Multi-Unit Truck |
6.2.3 cv_activities_model.RDS
File description: Commercial Vehicle activities model coefficients
This input contains the tables of proportions used in the CVTM’s
activities model. Each row shows the allocation between goods only,
service only, and both service and goods for a particularly industry
category. The table below shows all of the rows of
cv_activities_model.RDS.
The cv_activities_model.RDS file is produced by the
model estimation script,
CVM/dev/Estimation/cv_activities/Estimation_CV_Acitivities.R.
This script processes establishment information from the 2017 SEMECOG
Commercial Vehicle Survey (CVS).
Table 17: All rows of cv_activities_model.RDS
The table below describes the fields in
cv_activities_model.RDS.
Table 18: Field Descriptions for cv_activities_model.RDS
| Field | Description |
|---|---|
| EmpCatGroupedName | Grouped SEMCOG employment categories |
| Goods | Proportion of firms that provide goods delivery only |
| GoodsAndService | Proportion of firms whose activity includes both goods delivery and service calls |
| Other | Proportion of firms that made other stops (not goods or service stops) only |
| Service | Proportion of firms that provide service calls only |
6.2.4 cv_arrival_model.RDS
File description: Commercial Vehicle arrival model coefficients
This file contains an R model object that holds the estimation
results of the CVTM’s model to predict arrival time at the first stop on
a tour. The object is a list structure, one element of which is
estimates which contains the coefficients of the arrival
time model. The table below shows the coefficient estimates table from
cv_arrival_model.RDS.
The cv_arrival_model.RDS file is produced by the model
estimation script,
CVM/dev/Estimation/cv_arrival/model_estimation.R. This
model is estimated with data from the CVS.
Table 19: Estimates table in cv_arrival_model.RDS
| Coefficient Name | Coefficient |
|---|---|
| asc_overnight | 0.000 |
| asc_0600 | 0.782 |
| asc_0700 | 0.950 |
| asc_0730 | 1.430 |
| asc_0800 | 1.900 |
| asc_0830 | 2.026 |
| asc_0900 | 2.293 |
| asc_0930 | 2.080 |
| asc_1000 | 2.012 |
| asc_1030 | 1.379 |
| asc_1100 | 1.357 |
| asc_1130 | 1.379 |
| asc_1200 | 1.357 |
| asc_1230 | 1.007 |
| asc_1300 | 1.184 |
| asc_1330 | 0.792 |
| asc_1400 | 1.069 |
| asc_1430 | 0.406 |
| asc_1500 | 0.792 |
| asc_1530 | 0.463 |
| asc_1600 | 0.518 |
| asc_1630 | -0.636 |
| asc_1700 | -0.111 |
| asc_1730 | -0.885 |
| asc_1800 | -0.111 |
| asc_1900 | -1.502 |
| asc_2000 | -2.064 |
| asc_2100 | -1.252 |
| asc_begin_not_base_0900_1630 | -1.102 |
| asc_end_not_base_1700_2100 | 1.267 |
| asc_single_stop_1030_1800 | 0.430 |
| asc_is_no_stop_0600_0730 | 1.176 |
| asc_is_med_hvy_veh_0600 | -1.066 |
| asc_is_med_hvy_veh_0700 | -0.714 |
| asc_is_med_hvy_veh_0730 | -1.355 |
| asc_is_med_hvy_veh_0800 | -1.522 |
| asc_is_med_hvy_veh_0830 | -1.510 |
| asc_is_med_hvy_veh_0900 | -1.844 |
| asc_is_med_hvy_veh_0930 | -1.919 |
| asc_is_med_hvy_veh_1000 | -1.899 |
| asc_is_med_hvy_veh_1030 | -1.707 |
| asc_is_med_hvy_veh_1100 | -2.074 |
| asc_is_med_hvy_veh_1130 | -2.197 |
| asc_is_med_hvy_veh_1200 | -2.480 |
| asc_is_med_hvy_veh_1230 | -1.824 |
| asc_is_med_hvy_veh_1300 | -1.810 |
| asc_is_med_hvy_veh_1330 | -2.356 |
| asc_is_med_hvy_veh_1400 | -2.192 |
| asc_is_med_hvy_veh_1430 | -1.394 |
| asc_is_med_hvy_veh_1500 | -2.763 |
| asc_is_med_hvy_veh_1530 | -2.028 |
| asc_is_med_hvy_veh_1600 | -1.977 |
| asc_is_med_hvy_veh_1630 | -0.728 |
| asc_is_med_hvy_veh_1700 | -1.894 |
| asc_is_med_hvy_veh_1730 | -0.870 |
| asc_is_med_hvy_veh_1800 | -1.761 |
| asc_is_med_hvy_veh_1900_2100 | -0.855 |
6.2.5 cv_goods_model.RDS
File description: Commercial Vehicle goods stops model coefficients
This file contains an R model object that holds the estimation
results of the CVTM’s goods stop model. The object is a list structure,
one element of which is coefficients which contains the
coefficients of both the count model and zero hurdle model that comprise
the stop model. The table below shows the coefficients tables in
cv_goods_model.RDS.
The cv_goods_model.RDS file is produced by the model
estimation script,
CVM/dev/Estimation/cv_arrival/3 Goods models.R. This model
is estimated with data from the CVS.
Table 20: Coefficients table in cv_goods_model.RDS
|
|
6.2.6 cv_intermediate_deviations.rds
File description: Commercial Vehicle non-scheduled stop deviation thresholds
The CVTM’s non-scheduled stop model is constrained to search for
non-scheduled stop locations that do not deviate too far from the
shortest path between the last scheduled stop and the next scheduled
stop. The amount of deviation is dampened using a distance function
whereby locations with a higher deviation are less attractive. This
function uses a vehicle-type-specific parameter contained in this input
file. The table below shows all of the rows of
cv_intermediate_deviations.rds.
The cv_intermediate_deviations.rds file is produced by
the model estimation script,
CVM/dev/Estimation/cv_intermediate/data_preperation_deviations.R.
This parameters are estimated with data from the CVS.
Table 21: All rows of cv_intermediate_deviations.rds
The table below describes the fields in
cv_intermediate_deviations.rds.
Table 22: Field Descriptions for cv_intermediate_deviations.rds
| Field | Description |
|---|---|
| Vehicle | Vehicle type |
| deviation.dist | Deviation distance weighting parameter |
6.2.7 cv_intermediate_model.rds
File description: Commercial Vehicle non-scheduled stop generation model
This file contains an R model object that holds the estimation
results of the CVTM’s non-scheduled stop model, which is a choice
between a reference alternative (none, i.e., that there is no
non-scheduled stop between the last scheduled stop made and the next
scheduled stop) and alternatives describing three different types of
non-scheduled stop. The object is a list structure, one element of which
is estimates which contains the coefficients of the
non-scheduled stop model. The table below shows the coefficient
estimates table from cv_intermediate_model.rds.
The cv_intermediate_model.rds file is produced by the
model estimation script,
CVM/dev/Estimation/cv_intermediate/model_estimation.R. This
model is estimated with data from the CVS.
Table 23: Estimates table in cv_intermediate_model.rds
| Coefficient Name | Coefficient |
|---|---|
| asc_ns | 0.000 |
| asc_dn | -1.704 |
| asc_vs | -3.911 |
| asc_ot | -2.470 |
| b_log_duration_dn | -0.097 |
| b_log_duration_vs | 0.207 |
| b_log_duration_ot | 0.269 |
| b_med_veh_vs | -0.660 |
| b_med_veh_ot | -1.531 |
| b_hvy_veh_vs | -0.755 |
| b_hvy_veh_ot | -2.025 |
| b_is_lunch_dn | 0.287 |
| b_distance_1_2_dn | -0.414 |
| b_distance_2_10_dn | -0.228 |
| b_distance_10plus_dn | -0.333 |
| b_distance_1_10_vs | -0.505 |
| b_distance_10plus_vs | -0.218 |
| b_distance_1_2_ot | -0.128 |
| b_distance_2plus_ot | -0.333 |
6.2.8 cv_intermediate_model_attraction.rds
File description: Commercial Vehicle non-scheduled stop location model
This file contains an R model object that holds the estimation
results of the CVTM’s non-scheduled stop location choice model, which is
a choice among a set of potential stop TAZs in which to make the
non-scheduled stop. The object is a list structure, one element of which
is estimates which contains the coefficients of the
non-scheduled stop location choice model. The table below shows the
coefficient estimates table from
cv_intermediate_model_attraction.rds.
The cv_intermediate_model_attraction.rds file is
produced by the model estimation script,
CVM/dev/Estimation/cv_intermediate/model_estimation_attraction.R.
This model is estimated with data from the CVS.
Table 24: Estimates table in cv_intermediate_model_attraction.rds
| Coefficient Name | Coefficient |
|---|---|
| b_emp | 0.079 |
| b_pop | 0.000 |
| b_dist | -0.137 |
| b_retail | 0.911 |
| b_leisure | 0.000 |
| b_emp_vs | 0.000 |
| b_pop_vs | 0.000 |
| b_dist_vs | -0.049 |
| b_retail_vs | 0.000 |
| b_leisure_vs | 0.000 |
| b_emp_ot | 0.000 |
| b_pop_ot | 0.000 |
| b_dist_ot | 0.000 |
| b_retail_ot | 0.000 |
| b_leisure_ot | 0.636 |
6.2.9 cv_service_model.RDS
File description: Commercial Vehicle service stops model coefficients
This file contains an R model object that holds the estimation
results of the CVTM’s service stop model. The object is a list
structure, one element of which is coefficients which
contains the coefficients of both the count model and zero hurdle model
that comprise the stop model. The table below shows the coefficients
tables in cv_service_model.RDS.
The cv_service_model.RDS file is produced by the model
estimation script,
CVM/dev/Estimation/cv_arrival/4 Service models.R. This
model is estimated with data from the CVS.
Table 25: Coefficients tables in cv_service_model.RDS
|
|
6.2.10 cv_settings.RData
File description: Commercial Vehicle model parameters
This file is a compressed R work space (.Rdata file) that contains
several parameter objects used to support the application of the CVTM’s
stop generation, clustering, stop sequencing, and non-scheduled
(intermediate) stop models. The table below lists and describes the
contents of cv_settings.RData.
Table 26: Descriptions for objects in cv_settings.RData
| Object | Description |
|---|---|
| d_bars | Distance dampening values for CVTM stops model |
| hurdle_support | Parameter used in CVTM stops zero hurdle model |
| branch.limit | Parameter used in CVTM clustering model |
| cv_calibrated_parameters | Calibrated constants and other parameters for CVTM models |
| .Random.seed | Seed values for CVTM clustering model |
| numZones | Maximum zones to sample from for CVTM stops model |
| deviance.threshold | Maxiumum allowed deviation distance in the non-scheduled (intermediate) stop model |
The table below shows the values of the d_bars
parameters, which are distance dampening values for the CVTM stops
models, in cv_settings.RData.
Table 27: d_bars parameters in cv_settings.RData
| Coefficient Name | Coefficient |
|---|---|
| Production | 18 |
| Industrial | 18 |
| Retail | 18 |
| Transportation | 18 |
| Info_FIRE_Prof | 18 |
| Ed_Pub_Other_Ser | 18 |
| Leisure | 18 |
| Medical_Services | 18 |
The table below shows the values of the hurdle_support
parameter, which is used in the CVTM stops zero hurdle model to assert
an allowable set of output values from the model, in
cv_settings.RData.
Table 28: hurdle_support parameter in cv_settings.RData
## [1] 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
## [36] 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69
## [71] 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100The table below shows the values of the branch.limit
parameters, which are used in the CVTM clustering model to constrain the
size of the clusters created, in cv_settings.RData.
Table 29: branch.limit parameters in cv_settings.RData
| Coefficient Name | Coefficient.EmpCatName | Coefficient.brlim.mu | Coefficient.brlim.sd |
|---|---|---|---|
| EmpCatName | e01_nrm | 2.276 | 0.477 |
| brlim.mu | e02_constr | 2.276 | 0.477 |
| brlim.sd | e03_manuf | 2.143 | 0.493 |
| EmpCatName | e04_whole | 2.143 | 0.493 |
| brlim.mu | e05_retail | 1.890 | 0.724 |
| brlim.sd | e06_trans | 1.925 | 0.643 |
| EmpCatName | e07_utility | 1.890 | 0.724 |
| brlim.mu | e08_infor | 1.608 | 0.032 |
| brlim.sd | e09_finan | 1.651 | 0.809 |
| EmpCatName | e10_pstsvc | 2.320 | 0.503 |
| brlim.mu | e11_compmgt | 2.276 | 0.477 |
| brlim.sd | e12_admsvc | 2.276 | 0.477 |
| EmpCatName | e13_edusvc | 2.143 | 0.493 |
| brlim.mu | e14_medfac | 2.143 | 0.493 |
| brlim.sd | e15_hospit | 1.890 | 0.724 |
| EmpCatName | e16_leisure | 1.925 | 0.643 |
| brlim.mu | e17_othsvc | 1.890 | 0.724 |
| brlim.sd | e18_pubadm | 1.608 | 0.032 |
The table below shows the values of the
cv_calibrated_parameters parameters, which includes
calibrated constants and other calibration parameters for CVTM models,
in cv_settings.RData.
Table 30: cv_calibrated_parameters table in cv_settings.RData
| Variable | Value |
|---|---|
| CAL_CVTM_SCHED_GOODS | 4.75 |
| CAL_CVTM_SCHED_SERVICE | 5.00 |
| CAL_beta_v1_dist_00_02 | 1.00 |
| CAL_beta_v1_dist_02_05 | 1.00 |
| CAL_beta_v1_dist_05_10 | 1.50 |
| CAL_beta_v1_dist_10_20 | 3.00 |
| CAL_beta_v1_dist_20_p | 4.00 |
| CAL_beta_v2_dist_00_02 | 1.00 |
| CAL_beta_v2_dist_02_05 | 1.00 |
| CAL_beta_v2_dist_05_10 | 1.50 |
| CAL_beta_v2_dist_10_20 | 3.00 |
| CAL_beta_v2_dist_20_p | 4.00 |
| CAL_asc_light | 0.50 |
| CAL_asc_medium | -0.70 |
| CAL_asc_heavy | 0.00 |
| CAL_ARRIVAL_1 | -0.20 |
| CAL_ARRIVAL_2 | -0.20 |
| CAL_ARRIVAL_3 | -0.20 |
| CAL_ARRIVAL_4 | -0.20 |
| CAL_ARRIVAL_5 | -0.20 |
| CAL_ARRIVAL_6 | -0.20 |
| CAL_ARRIVAL_7 | -0.20 |
| CAL_ARRIVAL_8 | -0.10 |
| CAL_ARRIVAL_9 | 0.00 |
| CAL_ARRIVAL_10 | 0.00 |
| CAL_ARRIVAL_11 | 0.10 |
| CAL_ARRIVAL_12 | 0.10 |
| CAL_ARRIVAL_13 | 0.20 |
| CAL_ARRIVAL_14 | 0.20 |
| CAL_ARRIVAL_15 | 0.20 |
| CAL_ARRIVAL_16 | 0.20 |
| CAL_ARRIVAL_17 | 0.20 |
| CAL_ARRIVAL_18 | 0.20 |
| CAL_ARRIVAL_19 | 0.20 |
| CAL_ARRIVAL_20 | 0.10 |
| CAL_ARRIVAL_21 | 0.10 |
| CAL_ARRIVAL_22 | 0.00 |
| CAL_ARRIVAL_23 | 0.00 |
| CAL_ARRIVAL_24 | -0.20 |
| CAL_ARRIVAL_25 | -0.20 |
| CAL_ARRIVAL_26 | -0.20 |
| CAL_ARRIVAL_27 | -0.20 |
| CAL_ARRIVAL_28 | -0.20 |
The table below shows the first few values of the
.Random.seed parameter, which are seed values for the CVTM
clustering model, in cv_settings.RData.
Table 31: .Random.seed parameter in cv_settings.RData
## [1] 403 10 879089675 -506660550 -1093539780 -1828438145 130069717 338348268 1582367298 -1968978531The value of the numZones parameter, which sets the
maximum number of zones to sample from in the CVTM stops model, in
cv_settings.RData, is 16.
The value of the deviance.threshold parameter, which
sets the maximum allowed deviation distance in the non-scheduled
(intermediate) stop model, in cv_settings.RData, is 9.
6.2.11 cv_stopduration_model.RDS
File description: Commercial Vehicle stop duration model coefficients
This file contains an R model object that holds the estimation
results of the CVTM’s stop duration model, which predicts time ranges
for the length of time a vehicle spends at a stop. The object is a list
structure, one element of which is estimates which contains
the coefficients of the stop duration model. The table below shows the
coefficient estimates table from
cv_stopduration_model.RDS.
The cv_stopduration_model.RDS file is produced by the
model estimation script,
CVM/dev/Estimation/cv_duration/model_estimation.R. This
model is estimated with data from the CVS.
Table 32: Estimates table in cv_stopduration_model.RDS
| Coefficient Name | Coefficient |
|---|---|
| asc_15 | 0.000 |
| asc_30 | -1.074 |
| asc_45 | -1.963 |
| asc_60 | -2.452 |
| asc_75 | -3.053 |
| asc_90 | -3.447 |
| asc_150 | -2.719 |
| asc_210 | -3.520 |
| asc_270 | -3.994 |
| asc_390 | -4.172 |
| asc_600 | -3.601 |
| b_is_med_veh_30_75 | -0.088 |
| b_is_med_veh_90_plus | -0.653 |
| b_is_hvy_veh_30_90 | 1.005 |
| b_is_hvy_veh_150_plus | 0.777 |
| b_activity_service_60_75 | 0.575 |
| b_activity_service_90_plus | 1.457 |
| b_activity_other_30 | 0.100 |
| b_activity_other_45 | 0.175 |
| b_activity_other_60_90 | 0.317 |
| b_activity_other_150_plus | 0.511 |
| b_activity_driver_needs_30 | 0.329 |
| b_activity_driver_needs_45 | 0.525 |
| b_activity_driver_needs_60_90 | 0.526 |
| b_activity_driver_needs_150_plus | 0.377 |
6.2.12 cv_tours_model.rds
File description: Commerical Vehicle tour type model
This file contains an R model object that holds the estimation
results of the CVTM’s tour type choice model, which predicts the type of
tour that each cluster of stop will be part of. The object is a list
structure, one element of which is estimates which contains
the coefficients of the tour type choice model. The table below shows
the coefficient estimates table from
cv_tours_model.rds.
The cv_tours_model.rds file is produced by the model
estimation script,
CVM/dev/Estimation/cv_tours/model_estimation.R. This model
is estimated with data from the CVS.
Table 33: Estimates table in cv_tours_model.rds
| Coefficient Name | Coefficient |
|---|---|
| asc_bbm | 0.000 |
| asc_bbs | 0.008 |
| asc_bnm | -1.754 |
| asc_bns | -1.762 |
| asc_nbm | -2.836 |
| asc_nb0 | -3.196 |
| asc_nbs | -3.315 |
| asc_nnm | -2.910 |
| asc_bbm_industry_retail | 0.000 |
| asc_bbs_industry_retail | 0.000 |
| asc_bnm_industry_retail | 0.000 |
| asc_bns_industry_retail | 0.000 |
| asc_nbm_industry_retail | 0.000 |
| asc_nb0_industry_retail | 0.000 |
| asc_nbs_industry_retail | 0.000 |
| asc_nnm_industry_retail | 0.000 |
| asc_bbm_industry_industrial | 0.000 |
| asc_bbs_industry_industrial | 0.000 |
| asc_bnm_industry_industrial | 0.329 |
| asc_bns_industry_industrial | 0.000 |
| asc_nbm_industry_industrial | 0.000 |
| asc_nb0_industry_industrial | 0.000 |
| asc_nbs_industry_industrial | 0.000 |
| asc_nnm_industry_industrial | 0.000 |
| asc_bbm_industry_production | 0.000 |
| asc_bbs_industry_production | 0.000 |
| asc_bnm_industry_production | 0.000 |
| asc_bns_industry_production | 0.000 |
| asc_nbm_industry_production | 0.000 |
| asc_nb0_industry_production | 0.000 |
| asc_nbs_industry_production | 0.000 |
| asc_nnm_industry_production | 0.000 |
| asc_bbm_industry_ed_pub_other_ser | 0.000 |
| asc_bbs_industry_ed_pub_other_ser | 0.000 |
| asc_bnm_industry_ed_pub_other_ser | 0.000 |
| asc_bns_industry_ed_pub_other_ser | 0.000 |
| asc_nbm_industry_ed_pub_other_ser | 0.000 |
| asc_nb0_industry_ed_pub_other_ser | 0.000 |
| asc_nbs_industry_ed_pub_other_ser | 0.000 |
| asc_nnm_industry_ed_pub_other_ser | 0.000 |
| asc_bbm_industry_transportation | 0.000 |
| asc_bbs_industry_transportation | -0.499 |
| asc_bnm_industry_transportation | 0.000 |
| asc_bns_industry_transportation | 0.000 |
| asc_nbm_industry_transportation | 0.000 |
| asc_nb0_industry_transportation | 0.000 |
| asc_nbs_industry_transportation | 0.000 |
| asc_nnm_industry_transportation | 0.000 |
| asc_bbm_industry_info_fire_prof | 0.000 |
| asc_bbs_industry_info_fire_prof | 0.000 |
| asc_bnm_industry_info_fire_prof | 0.000 |
| asc_bns_industry_info_fire_prof | 0.000 |
| asc_nbm_industry_info_fire_prof | 0.000 |
| asc_nb0_industry_info_fire_prof | 0.000 |
| asc_nbs_industry_info_fire_prof | 1.232 |
| asc_nnm_industry_info_fire_prof | 1.337 |
| asc_bbm_industry_medical_services | 0.000 |
| asc_bbs_industry_medical_services | -0.461 |
| asc_bnm_industry_medical_services | 0.000 |
| asc_bns_industry_medical_services | 0.000 |
| asc_nbm_industry_medical_services | 0.000 |
| asc_nb0_industry_medical_services | 0.000 |
| asc_nbs_industry_medical_services | 0.000 |
| asc_nnm_industry_medical_services | 0.000 |
| asc_bbm_industry_leisure | 0.000 |
| asc_bbs_industry_leisure | 1.252 |
| asc_bnm_industry_leisure | 0.000 |
| asc_bns_industry_leisure | 0.000 |
| asc_nbm_industry_leisure | 0.000 |
| asc_nb0_industry_leisure | 0.000 |
| asc_nbs_industry_leisure | 0.000 |
| asc_nnm_industry_leisure | 0.000 |
| asc_bbm_is_med_veh | 0.000 |
| asc_bbs_is_med_veh | 0.000 |
| asc_bnm_is_med_veh | -0.871 |
| asc_bns_is_med_veh | -1.401 |
| asc_nbm_is_med_veh | -1.138 |
| asc_nb0_is_med_veh | 0.000 |
| asc_nbs_is_med_veh | 0.000 |
| asc_nnm_is_med_veh | 0.000 |
| asc_bbm_is_hvy_veh | 0.000 |
| asc_bbs_is_hvy_veh | 0.000 |
| asc_bnm_is_hvy_veh | 0.000 |
| asc_bns_is_hvy_veh | -1.297 |
| asc_nbm_is_hvy_veh | 0.000 |
| asc_nb0_is_hvy_veh | 0.000 |
| asc_nbs_is_hvy_veh | 0.000 |
| asc_nnm_is_hvy_veh | 0.000 |
6.2.13 cv_vehicle_model.RDS
File description: Commercial Vehicle vehicle choice model coefficients
This file contains an R model object that holds the estimation
results of the CVTM’s vehicle choice model, which predicts the vehicle
type used for each stop. The object is a list structure, one element of
which is estimates which contains the coefficients of the
vehicle choice model. The table below shows the coefficient estimates
table from cv_vehicle_model.RDS.
The cv_vehicle_model.RDS file is produced by the model
estimation script,
CVM/dev/Estimation/cv_vehicle/model_estimation.R. This
model is estimated with data from the CVS.
Table 34: Estimates table in cv_vehicle_model.RDS
| Coefficient Name | Coefficient |
|---|---|
| asc_light | 1.912 |
| asc_medium | 0.636 |
| asc_heavy | 0.000 |
| beta_v1_industry_retail | 0.000 |
| beta_v1_industry_industrial | -2.774 |
| beta_v1_industry_production | -1.532 |
| beta_v1_industry_ed_pub_other_ser | 0.871 |
| beta_v1_industry_transportation | -3.014 |
| beta_v1_industry_info_fire_prof | -0.335 |
| beta_v1_industry_medical_services | 11.239 |
| beta_v1_industry_leisure | 11.392 |
| beta_v2_industry_retail | 0.000 |
| beta_v2_industry_industrial | -1.064 |
| beta_v2_industry_production | -0.865 |
| beta_v2_industry_ed_pub_other_ser | 1.578 |
| beta_v2_industry_transportation | -1.666 |
| beta_v2_industry_info_fire_prof | 0.378 |
| beta_v2_industry_medical_services | 14.063 |
| beta_v2_industry_leisure | 10.939 |
| beta_v1_activity_deliver_pickup | 0.000 |
| beta_v1_activity_service | 3.787 |
| beta_v2_activity_deliver_pickup | 0.000 |
| beta_v2_activity_service | 2.366 |
| beta_v1_dist_00_02 | 0.000 |
| beta_v1_dist_02_05 | 0.681 |
| beta_v1_dist_05_10 | 0.620 |
| beta_v1_dist_10_20 | -0.300 |
| beta_v1_dist_20_p | -1.114 |
| beta_v2_dist_00_02 | 0.000 |
| beta_v2_dist_02_05 | 0.859 |
| beta_v2_dist_05_10 | 0.990 |
| beta_v2_dist_10_20 | -0.015 |
| beta_v2_dist_20_p | -0.277 |
6.2.14 Establishments.csv
File description: List of establishments for Firm Synthesis
This file contains the processed business establishment database that
is used as a seed for the firm synthesis model. It is a list of business
establishments by NAICS industrial codes and employment in each TAZ. The
table below shows the first few rows of
Establishments.csv.
The file was processed from an establishment database provided to
SEMCOG by MDOT. The script CVM/dev/data_firm_sim.R was used
to process the database into the Establishments.csv file
format.
Table 35: First few rows of Establishments.csv
The table below describes the fields in
Establishments.csv.
Table 36: Field Descriptions for Establishments.csv
| Field | Description |
|---|---|
| TAZ | TAZ code |
| EmpCatID | 2 or 3 digit NAICS industrial code |
| Emp | Employment at the establishment |
6.2.15 EstSizeCategories.csv
File description: Establishment size categories for Firm Synthesis
This input file defines the employment size categories used in the
firm synthesis model. The table below shows all of the rows in
EstSizeCategories.csv.
Table 37: All rows of EstSizeCategories.csv
The table below describes the fields in
EstSizeCategories.csv.
Table 38: Field Descriptions for EstSizeCategories.csv
| Field | Description |
|---|---|
| ID | Firm size category number |
| Label | Size category label |
| LowerBound | Lower bound number of employees |
| Midpoint | Category midpoint number of employees |
6.2.16 ld_distribution_model.csv
File description: Long distance distribution model parameters
This file contains the parameters of the long-distance model’s trip
distribution model, which is a gravity model that uses a gamma function
to estimate friction factor values. The gravity model was calibrated
using the ATRI truck GPS data that was processed and expanded to truck
counts. The table below shows all of the rows of
ld_distribution_model.csv.
The ld_distribution_model.csv file was created using the
script
CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R.
Table 39: All rows of ld_distribution_model.csv
6.2.17 ld_ee_trip_table.csv
File description: Long distance static external-to-external trip table
This file contains a trip list of origin to destination trips between
external TAZs that represents external to external base year truck
trips. The list was derived from ATRI truck GPS data that was processed
and expanded to truck counts. The table below shows the first few rows
of ld_ee_trip_table.csv.
The file was processed from the final ATRI truck OD matrix into the
ld_ee_trip_table.csv file format using the script
CVM/dev/data_ee_base_growth.R.
Table 40: First few rows of ld_ee_trip_table.csv
The table below describes the fields in
ld_ee_trip_table.csv.
Table 41: Field Descriptions for ld_ee_trip_table.csv
| Field | Description |
|---|---|
| OTAZ | Origin TAZ code |
| DTAZ | Destination TAZ code |
| trips | Number of daily truck trips |
| xx_movement_num | External Origin-Destination group ID number |
6.2.18 ld_external_model.csv
File description: Long distance external station allocation model
This file contains the external trip allocation model, which is a
table of factors allocating regional commodity flows to and from the
SEMCOG region from the Transearch commodity flow data to scaled external
truck flows derived from the ATRI truck GPS data at each external
station. The table below shows the first few rows in
ld_external_model.csv.
The ld_external_model.csv file is produced by the model
estimation script,
CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R. The
external factors were developed with ATRI truck GPS data and Transearch
commodity flow data.
Table 42: First few rows of ld_external_model.csv
The table below describes the fields in
ld_external_model.csv.
Table 43: Field Descriptions for ld_external_model.csv
| Field | Description |
|---|---|
| TAZ | TAZ/External Station code |
| StationGroup | External Station Geographical Group |
| StationName | External Station Name |
| StationType | External Station Type (Other Road, Major Highway, International Crossing) |
| XI | Number of Daily External to Internal Trips (Expanded ATRI) |
| IX | Number of Daily Internal to External Trips (Expanded ATRI) |
| XIIX | Total XI and IX Trips from Expanded ATRI |
| ExtGen | Balanced External Generation (XIIX/2) |
| ExtAttr | Balanced External Attraction (XIIX/2) |
| F_BUFFER_MI | Factor for Buffer Michigan region |
| F_BUFFER_OH | Factor for Buffer Ohio region |
| F_BUFFER_CAN | Factor for Buffer Canada region |
| F_OTHER_MI | Factor for Other Michigan region |
| F_Mexico | Factor for Mexico |
| F_US_Central | Factor for US Central region |
| F_US_East | Factor for US East region |
| F_US_South | Factor for US South region |
| F_US_West | Factor for US West region |
| F_Canada_West | Factor for Canada West region |
| F_Canada_East | Factor for Canada East region |
| F_Canada_ON | Factor for Canada Ontario region |
| F_Canada_QC | Factor for Canada Quebec region |
| ExtLongHaul | External Long Distance Trips |
| ExtShortHaul | External Short Distance Trips (Between Buffer and SEMCOG REgion) |
6.2.19 ld_internal_gen_model.RDS
File description: Long distance generations model coefficients for internal zones
This file contains an R model object that holds the estimation
results of the long-distance model’s internal generation model. This
model was estimated using TAZ socio-economic data, freight facility
data, and transportation supply and accessibility data as explanatory
variables and origins and destinations from the expanded ATRI truck GPS
data as the dependent variable. The object is a list structure, one
element of which is coefficients which contains the
coefficients of the linear regression model. The table below shows the
coefficients table in ld_internal_gen_model.RDS. The model
is used estimate both the internal long-distance trips generated and
attracted by each TAZ, as they were balanced in the expanded ATRI truck
GPS data.
The ld_internal_gen_model.RDS file is produced by the
model estimation script,
CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R. This
model is estimated with ATRI truck GPS data.
Table 44: Coefficients table in ld_internal_gen_model.RDS
| Coefficient Name | Coefficient |
|---|---|
| IndustrialSc | 0.107 |
| ProductionSc | 0.031 |
| TransportationSc | 0.020 |
| NFac_RailTruckSc | 33.654 |
| NFac_LandfillSc | 29.241 |
| DistFloorSpace_LargeKFtSc | 0.053 |
| DistFloorSpace_MediumKFtSc | 0.187 |
| Dist_Airport_Prod_EmpSc | 18.002 |
| Dist_Interstate_EmpSc | 0.960 |
| Dist_I75_Ambassador_EmpSc | 3.561 |
| Dist_I75_Ambassador_Large_DistSc | 9.725 |
6.2.20 ld_trip_time_vehicle_proportions.csv
File description: Long distance trip proportions by vehicle type and time-of-day
This file contains a list of factors to proportions of long distance
truck trips that take place in each time period by each vehicle class.
They are used to allocate long distance truck trips to vehicle classes
and time periods. The table below shows the first few rows of
ld_trip_time_vehicle_proportions.csv.
The ld_trip_time_vehicle_proportions.csv file is
produced by the data processing script,
CVM/dev/data_ldm_tod_Factors.R. The factors in the file are
derived from time period traffic counts at external stations and on
major facilities on routes to and from external stations.
Table 45: First few rows of ld_trip_time_vehicle_proportions.csv
The table below describes the fields in
ld_trip_time_vehicle_proportions.csv.
Table 46: Field Descriptions for ld_trip_time_vehicle_proportions.csv
| Field | Description |
|---|---|
| start_minutes | Time interval start minutes |
| end_minutes | Time interval end minutes |
| Light | Light vehicle proportion of truck trips in this time interval |
| Medium | Medium truck proportion of truck trips in this time interval |
| Heavy | Heavy truck proportion of truck trips in this time interval |
6.2.21 NAICS3_to_EmpCats.csv
File description: Correspondence between NAICS codes and employment Categories
This correspondence file contains a crosswalk between 2 or 3 digit
NAICS industrial codes and the employment categories used in
socioeconomic data from SEMCOG. The crosswalk is used to link data in
SEMCOG categories to the other employment categorizations used in the
model, including the groupings of SEMCOG employment categories used in
several of the models estimated using the CVS. The table below shows all
rows in NAICS3_to_EmpCats.csv.
The NAICS3_to_EmpCats.csv file is produced by the data
processing script, CVM/dev/data_industry_corresp.R.
Table 47: All rows of NAICS3_to_EmpCats.csv
The table below describes the fields in
NAICS3_to_EmpCats.csv.
Table 48: Field Descriptions for NAICS3_to_EmpCats.csv
| Field | Description |
|---|---|
| NAICSn2n3 | 2 or 3 digit NAICS industrial code |
| EmpCatID | SEMCOG Employment Sector Number |
| EmpCatName | SEMCOG Employment Sector Code |
| EmpCatDesc | SEMCOG Employment Sector Name |
| NAICSCodes | NAICS Industrial Category Code Range |
| NAICSDesc | Name of the NAICS industrial category |
| EmpCatGroupedName | Grouped SEMCOG Employment Category |
The table below summarizes the correspondence between the full set of 18 SEMCOG employment categories and the grouped categories used in several of the models estimated using the SEMCOG Commercial Vehicle Survey
Table 49: Summary of SEMCOG Employment Groupings
| EmpCatID | EmpCatName | EmpCatDesc | EmpCatGroupedName |
|---|---|---|---|
| 1 | e01_nrm | Natural Resources & Mining | Production |
| 2 | e02_constr | Construction | Industrial |
| 3 | e03_manuf | Manufacturing | Production |
| 4 | e04_whole | Wholesale Trade | Retail |
| 5 | e05_retail | Retail Trade | Retail |
| 6 | e06_trans | Transportation & Warehousing | Transportation |
| 7 | e07_utility | Utilities | Industrial |
| 8 | e08_infor | Information | Info_FIRE_Prof |
| 9 | e09_finan | Financial Activities | Info_FIRE_Prof |
| 10 | e10_pstsvc | Professional, Scientific, & Technical Services | Info_FIRE_Prof |
| 11 | e11_compmgt | Management of Companies & Enterprises | Info_FIRE_Prof |
| 12 | e12_admsvc | Administrative, Support, & Waste Services | Ed_Pub_Other_Ser |
| 13 | e13_edusvc | Education Services | Ed_Pub_Other_Ser |
| 14 | e14_medfac | Medical Facilities | Medical_Services |
| 15 | e15_hospit | Hospitals | Medical_Services |
| 16 | e16_leisure | Leisure & Hospitality | Leisure |
| 17 | e17_othsvc | Other Services | Ed_Pub_Other_Ser |
| 18 | e18_pubadm | Public Administration | Ed_Pub_Other_Ser |
6.2.22 TAZ_System.csv
File description: TAZ descriptions, tabular form of shapefile attribute data
This file contains a description of the TAZs used in the model
including the definition of aggregations of the TAZs into Counties. The
table below shows the first few rows of TAZ_System.csv.
The TAZ_System.csv file is produced by the data
processing script, CVM/dev/data_spatial.R from the TAZ
layers for the SEMCOG E8Plus model and the MDOT Statewide Model.
Table 50: First few rows of TAZ_System.csv
The table below describes the fields in
TAZ_System.csv.
Table 51: Field Descriptions for TAZ_System.csv
| Field | Description |
|---|---|
| TAZ | TAZ code |
| ID | TAZ code |
| TAZ_MDOT | MDOT Statewide Model TAZ code |
| TAZ_TYPE | Type of TAZ (SEMCOG model region, EXTERNAL station, BUFFER region) |
| TAZ_IE | Text label for whether the TAZ is an INTERNAL TAZ or an EXTERNAL station |
| INTERNAL | TRUE/FALE for whether the TAZ is an Internal TAZ (Buffer or SEMCOG) |
| SEMCOG | TRUE/FALSE for whether the TAZ is in the SEMCOG model region |
| BUFFER | TRUE/FALSE for whether the TAZ is in the buffer region |
| EXTERNAL | 0/1, value of 1 for external stations |
| CountyCode | SEMCOG code for the County |
| CountyName | Names of the county |
| FIPS | FIPS code for the county |
| STATE_PROV | State or Province for the TAZ (Michigan, Ohio, or Ontario |
| COUNTRY | Country (US or Canada) |
| DISTRICT | SEMCOG District Code |
| SUPER_DIST | SEMCOG Super District Code |
| AREA | TAZ area in acres |
| ZCTA5CE10 | US ZIP Code that TAZ Centroid falls in |
| SummaryGeog | Summary Geography (combination of counties and buffer regions) |
| StationGroup | External station group |
| StationName | External station name |
| StationType | External station type (Major Highywa, International Crossing, Other Road) |
| AIRPORT | TRUE/FALSE for whether the TAZ contains a major Airport |
| DistInterstate | Distance from the centroid of the TAZ to an Interstate highway |
The maps and table below display the TAZ system contained in
TAZ_System.csv. The first map shows the extent of the
internal SEMCOG model region along with the external stations, with
major externals labelled. The long distance model component works with
this TAZ system. The second map shows the internal SEMCOG model region
and the buffer TAZs. The CVTM model component works with this TAZ system
(and also does route trips between the buffer and internal TAZs via the
external stations that lie on the boundary between the two parts of the
TAZ system). The table shows the number of TAZs in each part of the TAZ
system by county.
Figure 41: Maps of SEMCOG Internal TAZs and External Stations
Figure 42: Maps of
SEMCOG Internal TAZs and Buffer TAZs
Table 52: Summary of
TAZ_System.csv
| TAZ_TYPE | STATE_PROV | CountyCode | CountyName | TAZs |
|---|---|---|---|---|
| SEMCOG | Michigan | 1 | Detroit | 379 |
| SEMCOG | Michigan | 2 | Other Wayne | 567 |
| SEMCOG | Michigan | 3 | Oakland | 722 |
| SEMCOG | Michigan | 4 | Macomb | 373 |
| SEMCOG | Michigan | 5 | Washtenaw | 374 |
| SEMCOG | Michigan | 6 | Monroe | 101 |
| SEMCOG | Michigan | 7 | St Clair | 171 |
| SEMCOG | Michigan | 8 | Livingston | 124 |
| BUFFER | Michigan | Clinton | 37 | |
| BUFFER | Michigan | Eaton | 46 | |
| BUFFER | Michigan | Genesee | 134 | |
| BUFFER | Michigan | Ingham | 108 | |
| BUFFER | Michigan | Jackson | 73 | |
| BUFFER | Michigan | Lapeer | 63 | |
| BUFFER | Michigan | Lenawee | 84 | |
| BUFFER | Michigan | Sanilac | 52 | |
| BUFFER | Michigan | Shiawassee | 54 | |
| BUFFER | Ohio | Lucas | 30 | |
| BUFFER | Ohio | Ottawa | 1 | |
| BUFFER | Ohio | Sandusky | 1 | |
| BUFFER | Ohio | Wood | 4 | |
| BUFFER | Ontario | 52 | ||
| SEMCOG | 2811 | |||
| BUFFER | 739 | |||
| EXTERNAL | 88 | |||
| ALL | 3638 |
6.2.23 TravelTimeSkimBuffer.omx
File description: Travel time skims for the buffer TAZs
Preprocessed Open Matrix format skim matrices of zone to zone travel
distances, free flow travel times, and tolls from the MDOT Statewide
model for travel within the buffer region and to and from the SEMCOG
external stations. Upon being imported in to the Cv model, the matrices
are converted into lists in I-J-Value format. These matrices, which
represent internal travel within the buffer region and between the
buffer region and SEMCOG external stations, are combined with the time
of skim matrices for the SEMCOG region that are included in the scenario
inputs into a single table called skims_tod. The table
below shows the first few rows of skims_tod.
The TravelTimeSkimBuffer.omx file is produced by the
data processing script, CVM/dev/data_buffer_skims.R from
the MDOT Statewide Model’s base year skim matrices.
Table 53: First few rows of skims_tod
The table below describes the fields in skims_tod.
Table 54: Field Descriptions for skims_tod
| Field | Description |
|---|---|
| OTAZ | Origin TAZ |
| DTAZ | Destination TAZ |
| time.NT | Travel time, night time period |
| dist.NT | Distance, night time period |
| toll.NT | Toll, night time period |
| time.AM | Travel time, AM peak time period |
| dist.AM | Distance, AM peak time period |
| toll.AM | Toll, AM peak time period |
| time.MD | Travel time, midday time period |
| dist.MD | Distance, midday time period |
| toll.MD | Toll, midday time period |
| time.PM | Travel time, PM peak time period |
| dist.PM | Distance, PM peak time period |
| toll.PM | Toll, PM peak time period |
| time.EV | Travel time, evening time period |
| dist.EV | Distance, evening time period |
| toll.EV | Toll, evening time period |
| time.avg | Travel time, daily average |
| dist.avg | Distance, daily average |
| toll.avg | Toll, daily average |
6.3 Scenario Inputs
In this section, each scenario input to the commercial vehicle model,
as contained in the Model Runs/[scenario name] directory,
is described.
6.3.1 base_transcad_taz.csv
File description: Employment, households, and population data by TAZ for the SEMCOG internal TAZs
This file is the socioeconomic data for the internal SEMCOG portion
of the model region for the scenario. The data are combined with the
socioeconomic data for the buffer portion of the model region and are
used to control the CV model’s firm synthesis to ensure that the total
employment by TAZ and employment category matches with TAZ level
employment assumptions. The table below shows the first few rows of
base_transcad_taz.csv. The file is imported to the CV model
during a model run from the
Model Runs/[scenario name]/Input/SED directory.
Table 55: First few rows of base_transcad_taz.csv
The table below describes the fields in
base_transcad_taz.csv.
Table 56: Field Descriptions for base_transcad_taz.csv
| Field | Description |
|---|---|
| zoneid | TAZ Code |
| taz | Long TAZ identifier |
| tot_acres | TAZ area in acres |
| tot_hhs | Number of households |
| hhs_pop | Household population |
| workers | Number of workers |
| grppop | Group quarters population |
| tot_pop | Total population |
| k12enroll | K-12 school enrollment |
| tot_emp | Total employment |
| e01_nrm | Emplyoment in Natural Resources & Mining |
| e02_constr | Emplyoment in Construction |
| e03_manuf | Emplyoment in Manufacturing |
| e04_whole | Emplyoment in Wholesale Trade |
| e05_retail | Emplyoment in Retail Trade |
| e06_trans | Emplyoment in Transportation & Warehousing |
| e07_utility | Emplyoment in Utilities |
| e08_infor | Emplyoment in Information |
| e09_finan | Emplyoment in Financial Activities |
| e10_pstsvc | Emplyoment in Professional, Scientific, & Technical Services |
| e11_compmgt | Emplyoment in Management of Companies & Enterprises |
| e12_admsvc | Emplyoment in Administrative, Support, & Waste Services |
| e13_edusvc | Emplyoment in Education Services |
| e14_medfac | Emplyoment in Medical Facilities |
| e15_hospit | Emplyoment in Hospitals |
| e16_leisure | Emplyoment in Leisure & Hospitality |
| e17_othsvc | Emplyoment in Other Services |
| e18_pubadm | Emplyoment in Public Administration |
| resid_sf | Residential building square footage |
| retail_sf | Retail building square footage |
| entmt_sf | Entertainment building square footage |
| othcom_sf | Other commercial building square footage |
| finan_sf | Financial building square footage |
| office_sf | Office building square footage |
| indust_sf | Industrail building square footage |
| medcare_sf | Medical care building square footage |
| medical_sf | Medical building square footage |
| edu_sf | Educational building square footage |
| othinst_sf | Other institutional building square footage |
6.3.2 CVM_Externals.csv
File description: External stations active in the scenario, and external allocation factors for trucks
This file contains a complete list of the external stations coded in
the model and an indication of whether they are active or not in the
current scenario. The file also contains allocation factors to
reallocate truck flows from the base year allocation to the scenario
allocation to account for changes in the active scenario and the
resulting need to reallocate where trucks are likely to enter and exit
the region. The table below shows the first few rows in
CVM_Externals.csv.
Table 57: All rows in CVM_Externals.csv
The table below describes the fields in
CVM_Externals.csv.
Table 58: Field Descriptions for CVM_Externals.csv
| Field | Description |
|---|---|
| TAZ | ID for the External Station TAZ |
| BaseActive | Whether the external station is active in the base scenario |
| Year | Year for the scenario |
| ScenarioActive | Whether the external station is active in the current scenario |
| BaseTAZ | Base external station TAZ that has flow allocated to this scenario external station TAZ |
| ScenarioProp | Proportion to the base external station TAZ’s volume that is allocated to this scenario external station TAZ |
6.3.3 CVM_EE_Region.csv
File description: External region to external region growth factors
This file contains growth rates for external to external truck
traffic to grow the base year external to external truck trips in the
long-distance model. The growth rates were estimated using Transearch
commodity flow data and are specific to groups of origin-destination
movements (e.g., “US South/Central, Mexico, Ohio Buffer TO Canada
Buffer/ON/QC/East”). The table below shows all rows in
CVM_EE_Region.csv.
The CVM_EE_Region.csv file is produced by the model
estimation script,
CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R. This
file was developed using Transearch data.
Table 59: All rows in CVM_EE_Region.csv
The table below describes the fields in
CVM_EE_Region.csv.
Table 60: Field Descriptions for CVM_EE_Region.csv
| Field | Description |
|---|---|
| xx_movement_od | ID for the group of external to external origin-destination movements |
| xx_movement_num | ID for the group of external to external origin-destination movements, by direction |
| xx_movement | Label for the group of external to external origin-destination movements, by direction |
| DailyTrucks_2015 | Number of daily trucks on that OD in the base year, from Transearch |
| Year | Year for the scenario |
| DailyTrucks | Number of daily trucks on that OD in the scenario year, from Transearch |
| Growth | Growth factor (ratio of daily trucks in the scenario year and base year |
6.3.4 CVM_IEEI_Region.csv
File description: Internal to external and external to internal external region control totals
This file contains growth rates for internal to external and external
to internal truck traffic to grow the base year external control totals
at the external stations to the region that used in the long-distance
model. The growth rates were estimated using Transearch commodity flow
data and are specific to groups of external origin and destination
regions (e.g., “Buffer Michigan”, “US East”). The table below shows all
rows in CVM_IEEI_Region.csv.
The CVM_IEEI_Region.csv file is produced by the model
estimation script,
CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R. This
file was developed using Transearch data.
Table 61: All rows in CVM_IEEI_Region.csv
The table below describes the fields in
CVM_IEEI_Region.csv.
Table 62: Field Descriptions for CVM_IEEI_Region.csv
| Field | Description |
|---|---|
| Region | Region name for the group of external movements |
| DailyTrucks | Number of daily trucks to and from that region in the scenario year, from Transearch |
6.3.5 Facilities.csv
File description: Descriptions of high-intensity freight locations
This file is lists the significant distribution and intermodal
facilities assumed for this scenario. The locations are used as special
generators and attractors in the LDM and the CVTM. The table below shows
the first few rows of Facilities.csv.
Table 63: First few rows of Facilities.csv
The table below describes the fields in
Facilities.csv.
Table 64: Field Descriptions for Facilities.csv
| Field | Description |
|---|---|
| ID | sequential ID code for the facility |
| SHPName | Data source name |
| FacilityTy | Facility type |
| City | City the facility is located in |
| FacilityNm | Name of the facility |
| YearBuilt | Year facility was built |
| FloorSpace | Square footage of the facility |
| Acres | Acreage of the facility |
| FacilityID | Facility ID from the data source |
| TAZ | TAZ code of the facility |
| COUNTY | SEMCOG county code for the county facility is located in |
| CountyName | County that the facility is located in |
6.3.6 TAZSocioEconomicsBuffer.csv
File description: Employment, households, and population data by TAZ for the buffer TAZs
This file is the socioeconomic data for the buffer portion of the
model region for the scenario. The data are combined with the
socioeconomic data for the internal portion of the SEMCOG region and are
used to control the CV model’s firm synthesis to ensure that the total
employment by TAZ and employment category matches with TAZ level
employment assumptions. The table below shows the first few rows of
TAZSocioEconomicsBuffer.csv.
The file was processed from TAZ socioeconomic data in the MDOT
Statewide Model. The script CVM/dev/data_firm_sim.R was
used to process the database into the
TAZSocioEconomicsBuffer.csv file format.
Table 65: First few rows of TAZSocioEconomicsBuffer.csv
The table below describes the fields in
TAZSocioEconomicsBuffer.csv.
Table 66: Field Descriptions for TAZSocioEconomicsBuffer.csv
| Field | Description |
|---|---|
| TAZ | TAZ Code |
| HH | Number of Households |
| POP | Population |
| e01_nrm | Emplyoment in Natural Resources & Mining |
| e02_constr | Emplyoment in Construction |
| e03_manuf | Emplyoment in Manufacturing |
| e04_whole | Emplyoment in Wholesale Trade |
| e05_retail | Emplyoment in Retail Trade |
| e06_trans | Emplyoment in Transportation & Warehousing |
| e07_utility | Emplyoment in Utilities |
| e08_infor | Emplyoment in Information |
| e09_finan | Emplyoment in Financial Activities |
| e10_pstsvc | Emplyoment in Professional, Scientific, & Technical Services |
| e11_compmgt | Emplyoment in Management of Companies & Enterprises |
| e12_admsvc | Emplyoment in Administrative, Support, & Waste Services |
| e13_edusvc | Emplyoment in Education Services |
| e14_medfac | Emplyoment in Medical Facilities |
| e15_hospit | Emplyoment in Hospitals |
| e16_leisure | Emplyoment in Leisure & Hospitality |
| e17_othsvc | Emplyoment in Other Services |
| e18_pubadm | Emplyoment in Public Administration |
6.3.7 AM_HwySkim.omx, EV_HwySkim.omx, MD_HwySkim.omx, NT_HwySkim.omx, PM_HwySkim.omx
File description: Time period specific time, distance, and toll skims
These five scenario inputs contain the five time period zone to zone travel distance matrices, travel time matrices, and toll matrices. At run time, the matrices are exported from the TransCAD .mtx format matrices into the .omx format file.
Upon being imported in to the CV model, the matrices are converted
into lists in I-J-Value format and tagged with the TAZ numbering instead
of matrix indices. These matrices which represent internal travel with
the SEMCOG model region and travel to, from, and between the external
stations to the SEMCOG model region, are combined with the skim matrices
for the buffer region that are included in the common data inputs into a
single table called skims_tod. The table below shows the
first few rows of skims_tod.
Table 67: First few rows of skims_tod
The table below describes the fields in skims_tod.
Table 68: Field Descriptions for skims_tod
| Field | Description |
|---|---|
| OTAZ | Origin TAZ |
| DTAZ | Destination TAZ |
| time.NT | Travel time, night time period |
| dist.NT | Distance, night time period |
| toll.NT | Toll, night time period |
| time.AM | Travel time, AM peak time period |
| dist.AM | Distance, AM peak time period |
| toll.AM | Toll, AM peak time period |
| time.MD | Travel time, midday time period |
| dist.MD | Distance, midday time period |
| toll.MD | Toll, midday time period |
| time.PM | Travel time, PM peak time period |
| dist.PM | Distance, PM peak time period |
| toll.PM | Toll, PM peak time period |
| time.EV | Travel time, evening time period |
| dist.EV | Distance, evening time period |
| toll.EV | Toll, evening time period |
| time.avg | Travel time, daily average |
| dist.avg | Distance, daily average |
| toll.avg | Toll, daily average |
7 Model Outputs
The main tabular outputs from the CV model for a particular scenario
are included in the Output/CVM folder for the scenario. The
output files are either .csv format files or R binary format files
(.Rdata) that contain compressed tables. In addition, the CV model adds
trip tables to the .omx trip tables in the Output/HAssign
folder for the scenario. These outputs are described in the table
below.
In addition to the main tabular and matrix outputs, the model produces summary outputs which are also listed in the table below. These tabular, matrix, and summary outputs are in addition to the downstream outputs such as vehicle link flow tables and HTML reports that the E8Plus model’s highway assignment and reporting steps produce.
Table 69: List of model outputs
| Filename | Location | Description |
|---|---|---|
| 1.Firms.RData | Output/CVM | Database of inputs and outputs from Firm Synthesis model component |
| 1.Firms.RData.BaseYearFirms.csv | Output/CVM | CSV export of firm synthesis base year firms table |
| 1.Firms.RData.ScenarioFirms.csv | Output/CVM | CSV export of firm synthesis scenario firms table |
| 1.Firms.RData.TAZLandUseCVTM.csv | Output/CVM | CSV export of processed land use database |
| 2.LongDistanceTrips.RData | Output/CVM | Dababase of inputs and outputs from Long Distance model component |
| 2.LongDistanceTrips.RData.ld_trips.csv | Output/CVM | CSV export of long distance model trip database |
| 2.LongDistanceTrips.RData.zoneProdsAttrs.csv | Output/CVM | CSV export of long distance model zonal productions and attractions |
| 3.CommercialVehicleTrips.RData | Output/CVM | Database of inputs and outputs from CVTM model component |
| 3.CommercialVehicleTrips.RData.cv_trips.csv | Output/CVM | CSV export of CVTM trips and tours table |
| 4.TripTables.RData | Output/CVM | Database of inputs and outputs from Trip Table model component |
| 4.TripTables.RData.TripTable.csv | Output/CVM | CSV export of trip table in list format |
| 5.DashboardTables.RData | Output/CVM | Database of inputs to the dashboard component |
| CVTM_Stops_Duration_Activity_Non-Scheduled.csv | Output/CVM | CVTM stop durations by activity, non scheduled (dashboard) |
| CVTM_Stops_Duration_Activity_Scheduled.csv | Output/CVM | CVTM stop durations by activity, scheduled (dashboard) |
| CVTM_Stops_Duration_Vehicle.csv | Output/CVM | CVTM stop durations by vehicle (dashboard) |
| CVTM_Stops_Duration_Vehicle_Activity.csv | Output/CVM | CVTM stop durations by activity, vehicle (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Activity.csv | Output/CVM | CVTM first stop arrival by activity (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Vehicle.csv | Output/CVM | CVTM first stop arrival by vehicle (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Vehicle_Activity.csv | Output/CVM | CVTM first stop arrival by activity, vehicle (dashboard) |
| CV_Trip_Generation_Summary.csv | Output/CVM | CV model trip generation summary (for HTML report) |
| CV_Trip_VMT_VHT_Summary.csv | Output/CVM | CV model trip, VMT, and VHT summary (dashboard) |
| Commercial_County_Vehicle_Segment.csv | Output/CVM | CVTM summary by count, vehicle, OD segment (dashboard) |
| Commercial_County_Vehicle_TOD.csv | Output/CVM | CVTM summary by count, vehicle, time of day (dashboard) |
| Commercial_Heavy_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, heavy vehicles (dashboard) |
| Commercial_Light_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, light vehicles (dashboard) |
| Commercial_Medium_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, medium vehicles (dashboard) |
| Commercial_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by activity (dashboard) |
| Commercial_Stop_Activity_by_County.csv | Output/CVM | CVTM stop activity by county (dashboard) |
| Commercial_Stop_Activity_by_County_Heavy.csv | Output/CVM | CVTM stop activity by county for heavy vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Light.csv | Output/CVM | CVTM stop activity by county for light vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Medium.csv | Output/CVM | CVTM stop activity by county for medium vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Vehicle.csv | Output/CVM | CVTM trip Activity by vehicle type (dashboard) |
| Commercial_Tour_NonScheduled_Stops_Tours.csv | Output/CVM | CVTM tours by number of non scheduled stops (dashboard) |
| Commercial_Tour_Scheduled_Stops_Tours.csv | Output/CVM | CVTM tours by number of scheduled stops (dashboard) |
| Commercial_Tour_Start_Locations_by_Region.csv | Output/CVM | CVTM tour start locations by region (dashboard) |
| Commercial_Tour_Stops_Tours.csv | Output/CVM | CVTM tours stops by tour (dashboard) |
| Commercial_Trip_Destinations_by_County_Buffer.csv | Output/CVM | CVTM trip destinations by SEMCOG county and buffer (dashboard) |
| Commercial_Trip_OD_Vehicle.csv | Output/CVM | CVTM trips by OD segment by vehicles (dashboard) |
| Commercial_Trip_Origins_by_County_Buffer.csv | Output/CVM | CVTM trip origins by SECMOG county and buffer (dashboard) |
| Commercial_Trip_TOD_Vehicle.csv | Output/CVM | CVTM trips by time of day and vehicle (dashboard) |
| Commercial_Trips_II_County_County.csv | Output/CVM | CVTM II trips by county OD (dashboard) |
| Commercial_Trips_IX_County_External.csv | Output/CVM | CVTM IX trips by county-external OD (dashboard) |
| Commercial_Trips_XI_External_County.csv | Output/CVM | CVTM XI trips by external-county OD (dashboard) |
| Commercial_Trips_XX_External_External.csv | Output/CVM | CVTM XX trips by external-external OD (dashboard) |
| Employees_by_County.csv | Output/CVM | Firm Synthesis: Employees by County (dashboard) |
| Employees_by_Industry.csv | Output/CVM | Firm Synthesis: Employees by Industry (dashboard) |
| Employees_by_Industry_and_County.csv | Output/CVM | Firm Synthesis: Employees by Industry and County (dashboard) |
| Employees_by_Industry_and_Size_Buffer.csv | Output/CVM | Firm Synthesis: Employees by Industry and Firm Size in the buffer (dashboard) |
| Employees_by_Industry_and_Size_SEMCOG.csv | Output/CVM | Firm Synthesis: Employees by Industry and Firm Size, SEMCOG (dashboard) |
| Employees_by_Size.csv | Output/CVM | Firm Synthesis: Employees by Firm Size (dashboard) |
| Facilities_by_Type.csv | Output/CVM | Firm Synthesis: Facilites by facility type (dashboard) |
| Firms_by_County.csv | Output/CVM | Firm Synthesis: Firms by County (dashboard) |
| Firms_by_Industry.csv | Output/CVM | Firm Synthesis: Firms by Industry (dashboard) |
| Firms_by_Industry_and_County.csv | Output/CVM | Firm Synthesis: Firms by Industry and County (dashboard) |
| Firms_by_Industry_and_Size_Buffer.csv | Output/CVM | Firm Synthesis: Firms by Industry and Firm Size in the buffer (dashboard) |
| Firms_by_Industry_and_Size_SEMCOG.csv | Output/CVM | Firm Synthesis: Firms by Industry and Firm Size, SEMCOG (dashboard) |
| Firms_by_Size.csv | Output/CVM | Firm Synthesis: Firms by Firm Size (dashboard) |
| Long-Distance_Trips_IXXIODSummary.csv | Output/CVM | Long Distance Trips IX and XI OD Summary (dashboard) |
| Long-Distance_Trips_IX_County_External.csv | Output/CVM | Long Distance IX Trips by County and External (dashboard) |
| Long-Distance_Trips_XI_External_County.csv | Output/CVM | Long Distance XI Trips by External and County (dashboard) |
| Long-Distance_Trips_XX_External_External.csv | Output/CVM | Long Distance XX Trips by External OD (dashboard) |
| Long-Distance_Trips_XX_ODSummary.csv | Output/CVM | Long Distance XX Trips OD Summary (dashboard) |
| Long-Distance_Trips_by_County.csv | Output/CVM | Long Distance Trips by County (dashboard) |
| Long-Distance_Trips_by_DistBin.csv | Output/CVM | Long Distance Trips by Distance range (dashboard) |
| Long-Distance_Trips_by_External.csv | Output/CVM | Long Distance Trips by External Station (dashboard) |
| Long-Distance_Trips_by_External_TAZ.csv | Output/CVM | Long Distance Trips by External and TAZ (dashboard) |
| Long-Distance_Trips_by_External_TOD.csv | Output/CVM | Long Distance Trips by External and Time Period (dashboard) |
| Long-Distance_Trips_by_External_TOD_ODSeg.csv | Output/CVM | Long Distance Trips by External, Time Period and OD segment (dashboard) |
| Long-Distance_Trips_by_TOD.csv | Output/CVM | Long Distance Trips by Time Period (dashboard) |
| ReportDashboard.html | Output/CVM | CVM dashboard for the scenario |
| TripTable_Heavy_PctTrips_by_County_OD.csv | Output/CVM | Trip table heavy percent trips by county OD (dashboard) |
| TripTable_Heavy_Trips_by_County_OD.csv | Output/CVM | Trip table heavy trips by county OD (dashboard) |
| TripTable_Light_PctTrips_by_County_OD.csv | Output/CVM | Trip table light percent trips by county OD (dashboard) |
| TripTable_Light_Trips_by_County_OD.csv | Output/CVM | Trip table light trips by county OD (dashboard) |
| TripTable_Medium_PctTrips_by_County_OD.csv | Output/CVM | Trip table medium percent trips by county OD (dashboard) |
| TripTable_Medium_Trips_by_County_OD.csv | Output/CVM | Trip table medium trips by county OD (dashboard) |
| TripTable_PctTrips_by_County_OD.csv | Output/CVM | Trip table percent trips by county OD (dashboard) |
| TripTable_Trips_IX_County_External.csv | Output/CVM | Trip table IX trips by county external OD (dashboard) |
| TripTable_Trips_XI_External_County.csv | Output/CVM | Trip table XI trips by external county OD (dashboard) |
| TripTable_Trips_XX_External_External.csv | Output/CVM | Trip table XX trips by external external OD (dashboard) |
| TripTable_Trips_by_County_OD.csv | Output/CVM | Trip table trips by county OD (dashboard) |
| TripTable_Trips_by_Vehicle_Segment_TOD.csv | Output/CVM | Trip table trips by vehicle, OD segment, and time period (dashboard) |
| Validation_2015_KA15_2022_07_04_08_15_17.xlsx | Output/CVM | Validation spreadsheet with time and date naming (dashboard) |
| log | Output/CVM | Folder of log files for model runs |
| skims_buffer.rds | Output/CVM | Combined skims table including buffer region |
| skims_tod.rds | Output/CVM | Combined skims table for all time periods for SEMCOG region |
| OD_AM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for AM time period |
| OD_DY.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for full day |
| OD_EV.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for EV time period |
| OD_MD.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for MD time period |
| OD_NT.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for NT time period |
| OD_PM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for PM time period |
7.1 Model Component Databases
This section provides documentation of the five model component databases that are exported at the end of each of the three main CV model components and then after the fourth and fifth model steps that creates the commercial vehicle trip tables and dashboard respectively. The contents of the databases are described in turn below.
7.1.1 1.Firms.RData
The firms database contains the outputs from the firm synthesis
model, which is three data tables called ScenarioFirms,
BaseYearFirms, and TAZLandUseCVTM contained
within an R list object called firm_sim_results, as well as
a copy of the input files used by the firm synthesis model contained
within an R environment called firm_inputs. The table below
lists the output tables in firm_sim_results.
Table 70: Tables in firm_sim_results in 1.Firms.RData
| Tables | Description |
|---|---|
| ScenarioFirms | Table of synthesized business establishments for the scenario year |
| BaseYearFirms | Table of synthesized business establishments for the base year |
| TAZLandUseCVTM | Table of processed TAZ land use data |
The ScenarioFirms data table contains one row for each
of the synthesized business establishments which are described with
characteristics such as location, industry, and size. The table below
shows the first few rows of the RegionalFirms table.
Table 71: First few rows of ScenarioFirms
The table below describes the fields in
ScenarioFirms.
Table 72: Field Descriptions for ScenarioFirms
| Field | Description |
|---|---|
| TAZ | TAZ that the business is located in |
| EmpCatID | SEMCOG employment category ID |
| Emp | Number of employees |
| BusID | Unique business identifier (created in the model) |
| EmpCatName | SEMCOG employment category name |
| EmpCatGroupedName | Grouped SEMCOG employment category name |
| esizecat | Employment size category |
The BaseYearFirms data table is a copy of the base year
firm synthesis outputs once they have been scaled to match the TAZ
control data. It is used in any future year scenarios as the basis on
which to add future growth (or decline) in each TAZ. The table below
shows the first few rows of the BaseYearFirms table.
Table 73: First few rows of BaseYearFirms
The table below describes the fields in
BaseYearFirms.
Table 74: Field Descriptions for BaseYearFirms
| Field | Description |
|---|---|
| TAZ | TAZ that the business is located in |
| EmpCatID | SEMCOG employment category ID |
| Emp | Number of employees |
| BusID | Unique business identifier (created in the model) |
| EmpCatName | SEMCOG employment category name |
| EmpCatGroupedName | Grouped SEMCOG employment category name |
| esizecat | Employment size category |
The TAZLandUseCVTM contains processed TAZ land use data,
with one row per TAZ, that is used in the CVTM to estimate stops per
TAZ. The table below shows the first few rows of the
TAZLandUseCVTM table.
Table 75: First few rows of TAZLandUseCVTM
The table below describes the fields in
TAZLandUseCVTM.
Table 76: Field Descriptions for TAZLandUseCVTM
| Field | Description |
|---|---|
| TAZ | TAZ that the business is located in |
| Ed_Pub_Other_Ser | Education, Public and Other Service employment |
| Industrial | Industrial employment |
| Info_FIRE_Prof | Information, FIRE, and Professional Services employment |
| Leisure | Leisure and Hospitality employment |
| Medical_Services | Medical Services employment |
| Production | Manufacturing employment |
| Retail | Retail employment |
| TotalEmp | Total employment |
| TotalHHs | Total households |
| Transportation | Transportation and Warehousing employment |
7.1.2 2.LongDistanceTrips.RData
The long distance trips database contains the outputs from the long
distance model, which is two data tables called ld_trips
and zoneProdsAttrs contained within an R list object called
ld_sim_results, as well as a copy of the input files used
by the long distance model contained within an R environment called
ld_inputs. The table below lists the output tables in
ld_sim_results.
Table 77: Tables in ld_sim_results in 2.LongDistanceTrips.RData
| Tables | Description |
|---|---|
| ld_trips | Table of long distance trips by OD pair, vehicle and time of day |
| zoneProdsAttrs | Table of zonal long-distance trip generation outputs |
The ld_trips table contains one row for each external to
internal, internal to external, and external to external zone pair, for
each vehicle class and time-of-day, and reports the number of trips for
that zone pair and vehicle type - time period combination. The table
below shows the first few rows of the ld_trips table.
Table 78: First few rows of ld_trips
The table below describes the fields in ld_trips.
Table 79: Field Descriptions for ld_trips
| Field | Description |
|---|---|
| OTAZ | Origin TAZ |
| DTAZ | Destination TAZ |
| trips | Number of trips |
| Vehicle | Vehicle Class |
| TOD | Model time-of-day period |
The zoneProdsAttrs table contains one row for each TAZ
and reports the number of long distance truck trips generated and
attracted to that zone. The table below shows the first few rows of the
zoneProdsAttrs table.
Table 80: First few rows of zoneProdsAttrs
The table below describes the fields in
zoneProdsAttrs.
Table 81: Field Descriptions for zoneProdsAttrs
| Field | Description |
|---|---|
| TAZ | TAZ code |
| tripGen | Trips generated in the zone |
| tripAttr | Trips attracted to the zone |
7.1.3 3.CommercialVehicleTrips.RData
The commercial vehicle database contains the outputs from the CVTM,
which is a data table called cv_trips contained within an R
list object called cv_sim_results, as well as a copy of the
input files used by the CVTM contained within an R environment called
cv_inputs. The table below lists the output table in
cv_sim_results.
Table 82: Tables in cv_sim_results in 3.CommercialVehicleTrips.RData
| Tables | Description |
|---|---|
| cv_trips | Table of commercial vehicle trips and tours |
The cv_trips table contains one row for each trip made
by a commercial vehicle which are described with characteristics such as
which tour they belong to, which business and vehicle made the trips,
and start, end, activity and timing information for the trip. The table
below shows the first few rows of the cv_trips table.
Table 83: First few rows of cv_trips
The table below describes the fields in cv_trips.
Table 84: Field Descriptions for cv_trips
| Field | Description |
|---|---|
| BusID | Unique business identifier (created in the model) |
| Vehicle | Vehicle type (light, medium, heavy) |
| TourID | Tour identification number |
| TripID | Trip identifier (order within the tour) |
| Scheduled | Flag to denote that the stop at the end of the trip was scheduled (vs. a non-scheduled stop) |
| OTAZ | Origin TAZ |
| DTAZ | Destination TAZ |
| Activity | Activity at the stop location at the end of the trip |
| MAMDepart | Departure time of the trip (minutes after midnight) |
| MAMArrive | Arrival time of the trip (minutes after midnight) |
| TravelTime | Travel time of the trip (minutes) |
| Distance | Distance of the trip (miles) |
| StopDuration | Stop duration for the activity at the end of the trip |
| TourType | Text field indicating if the tour started and ended at the base of the firm operating the vehicle |
| TourStartEndLoc | Text field indicating if the tour started and ended in the same TAZ or a different TAZ |
| MAMMid | Time at the midpoint of the trip (minutes after midnight) |
| TOD | Time of day period (SEMCOG time periods, (NI, AM, MD, PM, EV) |
| OTYPE | Origin TAZ Type (SEMCOG or Buffer) |
| DTYPE | Destination TAZ Type (SEMCOG or Buffer) |
| SKIMTYPE | Skim route type (Within SEMCOG, SEMCOG to Buffer, Buffer to SEMCOG, Buffer no traverse, Buffer traversing SEMCOG) |
| EXT | External Station for SEMCOG to Buffer and Buffer to SEMCOG trips |
| EXT1 | External station entering SEMCOG region for Buffer traversing SEMCOG trips |
| EXT2 | External station exiting SEMCOG region for Buffer traversing SEMCOG trips |
| dist.int | Distance of the trip within SEMCOG region (miles) |
| time.int | Travel time of the trip within SEMCOG region (minutes) |
| dist.buf | Distance of the trip within the buffer region (miles) |
| time.buf | Travel time of the trip within the buffer region (minutes) |
| dist.buf1 | Distance of the trip from the origin to the SEMCOG external station for Buffer traversing SEMCOG trips (miles) |
| time.buf1 | Travel time of the trip from the origin to the SEMCOG external station for Buffer traversing SEMCOG trips (minutes) |
| dist.buf2 | Distance of the trip from the SEMCOG external station to the destination for Buffer traversing SEMCOG trips (miles) |
| time.buf2 | Travel time of the trip from the SEMCOG external station to the destination for Buffer traversing SEMCOG trips (minutes) |
7.1.4 4.TripTables.RData
The trip tables databases contains the database output from the trip
table function which produces the trip tables that are exported into the
.omx format trip table files used in the E8Plus model. The .omx trip
tables are described in the Trip Tables
section. The database output is an R list object called
tt_list that contains 18 matrices in origin to destination
TAZ format, each named with the file path location of the .omx file that
they are written into. In addition, tt_list includes
TripTable which is a data table containing an OD list of
all trip produced by the CV model.
Table 85: List of Output Trip Matrices and Tables in tt_list in 4.TripTables.RData
| Name | Description |
|---|---|
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_NT.OMX | Trip Matrix for Light Vehicles and time period is Nighttime |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_AM.OMX | Trip Matrix for Light Vehicles and time period is AM Peak |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_MD.OMX | Trip Matrix for Light Vehicles and time period is Midday |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_PM.OMX | Trip Matrix for Light Vehicles and time period is PM Peak |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_EV.OMX | Trip Matrix for Light Vehicles and time period is Evening |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_DY.OMX | Trip Matrix for Light Vehicles and time period is Daily |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_NT.OMX | Trip Matrix for Medium Trucks and time period is Nighttime |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_AM.OMX | Trip Matrix for Medium Trucks and time period is AM Peak |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_MD.OMX | Trip Matrix for Medium Trucks and time period is Midday |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_PM.OMX | Trip Matrix for Medium Trucks and time period is PM Peak |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_EV.OMX | Trip Matrix for Medium Trucks and time period is Evening |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_DY.OMX | Trip Matrix for Medium Trucks and time period is Daily |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_NT.OMX | Trip Matrix for Heavy Trucks and time period is Nighttime |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_AM.OMX | Trip Matrix for Heavy Trucks and time period is AM Peak |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_MD.OMX | Trip Matrix for Heavy Trucks and time period is Midday |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_PM.OMX | Trip Matrix for Heavy Trucks and time period is PM Peak |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_EV.OMX | Trip Matrix for Heavy Trucks and time period is Evening |
| E:/Projects/Clients/SEMCOG_CVTM/Model Runs/2015_KA15/Output/HAssign/OD_DY.OMX | Trip Matrix for Heavy Trucks and time period is Daily |
| TripTable | Trip table data table |
The TripTable table combines the trip from both the
long-distance model and the CVTM aggregated by OD pair, vehicle type,
and time period. The table below shows the first few rows of the
TripTable table.
Table 86: First few rows of TripTable
The table below describes the fields in TripTable.
Table 87: Field Descriptions for TripTable
| Field | Description |
|---|---|
| OTAZ | Origin TAZ |
| DTAZ | Destination TAZ |
| Vehicle | Vehicle Type (Light, Medium, Heavy) |
| TOD | Time Period (NT, AM, MD, PM, EV) |
| trips | Number of trips |
| Source | Source model component (Long-distance model or CVTM) |
7.1.5 5.DashboardTables.RData
The dashboard tables databases contains the database output from the
dashboard step of the model. The database output is an R environment
object called db_inputs that contains the inputs and
processed tabulations used to create the tables, charts, and maps
displayed in the dashboard HTML file and output to CSV format output
tabulations.
7.2 Trip Tables
The CV model exports the 18 trip tables it produces (three vehicle
types by five time periods plus a daily table for each vehicle type)
into the five time period .omx trip table files and the daily .omx trip
table file. These six .omx files are located in the
Model Runs/[scenario name]/Output/Hassign folder, and are
listed in the table below.
Table 88: List of .omx trip table outputs
| Filename | Location | Description |
|---|---|---|
| OD_AM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for AM time period |
| OD_DY.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for full day |
| OD_EV.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for EV time period |
| OD_MD.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for MD time period |
| OD_NT.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for NT time period |
| OD_PM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for PM time period |
In addition to saving the trip tables into the set of six .omx file
used by the E8Plus model, there is also a user option to save a single
consolidated set of CV model trip tables in one .omx file for more
convenient analysis. In the
CVM/lib/scripts/_USER_VARIABLES.R script, this option can
be selected as shown in the figure below. If this option is used, the
trip tables will be saved in the
Model Runs/[scenario name]/Output/CVM folder, into a file
named CV_Trip_Tables.omx.
Figure 43: USER VARIABLES Script Saving Combined Trip Tables File
7.3 Visualization Dashboard and Spreadsheet Summary
The CV Model Dashboard is produced by the dashboard
component of the CV model. This visualization dashboard summarizes
results from the CV model only and is contained in a scenario specific
file called ReportDashboard.html. The dashboard is saved in
the Model Runs/[scenario name]/Output/CVM folder. The
dashboard is a single HTML file that is produced at the end of each
model run. Since it is an HTML file, it can be linked directly to a
website or distributed and opened in a web browser. It contains
interactive tabulations, charts, and maps embedded directly into the
HTML file so that users can explore the scenario results.
The code that produces the dashboards is, like the remainder of the
CV model, written in R, and specifically R Markdown, which combines text
and R code to create report outputs. The dashboard’s functionality
relies on several R packages, including flexdashboard to
convert the R Markdown to a navigable webpage layout,
leaflet to render interactive maps, and plotly
to produce dynamic charts.
Many of the results summaries that are included in the dashboard are
also exported to a summary spreadsheet called
ReportSpreadsheet.xlsx that is also saved in the
Model Runs/[scenario name]/Output/CVM folder.
7.3.1 Code and Files That Build the Dashboard and Spreadsheet
The visualization dashboards are created at the end of the commercial
vehicle model run. The block of R code at the end of the R script
__Master.R, at the comment “Produce Dashboard and
Spreadsheet Report” creates the dashboard. The figure below shows the
code block. Six R scripts are involved in producing the dashboards:
- db_build.R which is the control function for the dashboard.
- db_build_process_inputs.R which loads model outputs and then labels and summarizes them.
- db_build_graphics_functions.R contains support functions used in the creation of the charts and maps in the dashboard.
- db_build_render which processes the R markdown files into the dashboard.
- db_build_spreadsheet which produces the spreadsheet summary the accompanies the dashboard.
- db_build_spreadsheet_functions contains support functions used in the creation of the spreadsheet summary.
As noted in the code block, there is a global variable
SCENARIO_RUN_DB that controls whether the dashboard is
produced. A value of TRUE for this variable is passed to R
from the E8Plus model in TransCAD when the dashboard step is run.
*Figure 44: Visualization Dashboard Code in __Master.R*
### Produce Dashboard and Spreadsheet Report -------------------------------------------------------------------------
if (SCENARIO_RUN_DB) {
cat("Producing Commercial Vehicle Model Dashboard", "\n")
# Load executive functions
source(file = file.path(SYSTEM_SCRIPTS_PATH, "db_build.R"))
source(file = file.path(SYSTEM_SCRIPTS_PATH, "db_build_process_inputs.R"))
# Process inputs
cat("Processing Commercial Vehicle Model Dashboard Inputs", "\n")
db_inputs <- new.env()
db_build_process_inputs(envir = db_inputs)
# Generate dashboard and spreadsheet
cat("Rendering Commercial Vehicle Model Dashboard and Spreadsheet", "\n")
dashboardFileLoc <- suppressWarnings(suppressMessages(
run_sim(FUN = db_build, data = NULL,
packages = SYSTEM_REPORT_PKGS, lib = SYSTEM_PKGS_PATH,
inputEnv = db_inputs
)
))
# Save results to Rdata file
cat("Saving Dashboard Tabulations Database", "\n")
save(db_inputs,
file = file.path(SCENARIO_OUTPUT_PATH,
SYSTEM_DB_OUTPUTNAME))
}The R markdown files that are rendered (converted from R markdown to
HTML to produce the dashboard) are the .Rmd files in
CVM/lib/scipts/db_markdown. The main file is
ReportDashboard.Rmd, and there are then individual files
that produce the subsidiary pages for the firm synthesis model, etc. The
figure below shows the header section from
ReportDashboard.Rmd. This header explains that the output
is to be a flex dashboard, is to be structured in columns, and is styled
using the ReportDashboardStyles.css stylesheet. The
remainder of the .Rmd files contain a blend of R markdown to layout and
annotate the dashboards and R code to create the individual charts,
tables, and maps that are shown in the dashboards.
Figure 45: Visualization Dashboard .Rmd File Header
---
title: "<b><i>SEMCOG</i></b> | Commercial Vehicle Model | `r paste(SCENARIO_NAME, 'Scenario')`"
date: "`r format(SCENARIO_RUN_START, '%Y-%m-%d')`"
output:
flexdashboard::flex_dashboard:
orientation: columns
vertical_layout: fill
theme: yeti
css: "ReportDashboardStyles.css"
editor_options:
chunk_output_type: console
---7.3.2 Dashboard Content
The dashboard contain multiple tabs that can be accessed from the drop down menus along the top of the dashboard. In addition to the introductory overview tab, there is one tab for each of the main CV model components that are included in the model run. Within each of the component tabs, there are multiple pages that can be accessed using the drop down menus.
The individual pages contain a variety of tables, charts, and maps. Many of these are interactive. For example, the maps can be panned and zoomed in and out, and categories selected or deselected. Similarly, categories on the charts can be shown or hidden, and the charts can be exported to image files for including in a report.
7.3.3 Spreadsheet Summary
The spreadsheet summary produces validation, scenario output, or scenario comparison summaries in spreadsheet form. Since it can include assignment summaries, it is only produced at the end or after a complete model run of the E8Plus model has been made. The E8Plus model’s HTML report dialog includes a check box option to create the spreadsheet as part of the validation block of outputs, and the spreadsheet is created and saved when the HTML report is made.
Figure 46: E8Plus Model Reporting GUI
8 Running the Model
8.1 Introduction
This chapter of the user guide discusses:
- Hardware Requirements and Software Requirements and
Installation for the E8Plus model including the CVM.
- Installing the Model and Running the E8Plus Model from TransCAD: Installing and running the CV model as part of the E8Plus model, including how to run individual parts of the CV model.
- Model Configuration on Different Hardware describes how to configure the CV model to perform well given different hardware capabilities.
- Special Considerations for Running an Alternative or Future Year Scenario: The sequence of runs required to run a future year scenario (which requires a completed base year run first).
- Running the E8Plus Model with Static CV Model Trip Tables covers how to run the E8Plus model with a bypass of the CV model, instead using previously saved CV model trip tables as a model input.
The next chapter of the user guide deals with creating alternative scenarios.
8.2 Hardware Requirements
The E8Plus model including CV model runs on a windows computer. The set-up of the CV model is somewhat flexible (see Model Configuration on Different Hardware) but does have reasonably high minimum RAM requirements due to the way that R handles large data tables such as skims in memory. Two typical minimum and recommended specifications for computers to run the model are as follows. These are fairly typical modern modeling workstations (for the minimum requirements) or medium size server configurations (for the recommended) and are consistent with or smaller (in terms of RAM requirements) than an activity-based model for a region such as the SEMCOG region.
- Minimum requirements:
- Operating System: 64-bit Windows 7, 64-bit Windows 8 (8.1) or 64-bit Windows 10
- Processor: Intel CPU Core i5-2500K 3.3GHz / AMD CPU Phenom II X4 940 (or equivalent)
- Memory: 32 GB RAM
- Recommended specification:
- Operating System: 64-bit Windows 7, 64-bit Windows 8 (8.1) or 64-bit Windows 10
- Processor: Intel CPU Core i7 3770 3.4 GHz / AMD CPU AMD FX-8350 4 GHz (or equivalent)
- Memory: 128 GB RAM
8.3 Software Requirements and Installation
Two software applications, TransCAD and R, should be installed on the computer that will be used to run the E8Plus model including CV model. The specific version requirements are described here.
The overall model system, the SEMCOG E8Plus model with CV model, is
an integrated model that is controlled by and primarily runs in the
TransCAD transportation planning software platform produced by Caliper
(https://www.caliper.com/tcovu.htm). The E8Plus model
uses TransCAD version 8, build 22365. The E8Plus model has
a graphical user interface (GUI) programmed in TransCAD’s GISDK
scripting language that is used to manage scenarios, start model runs,
produce output reports, and run other modeling utilities.
The CV model has been incorporated into the E8Plus model’s file
structure. The CV model is programmed in R, an open source statistical
programming platform available from The Comprehensive R Archive Network
(https://cran.r-project.org/). The CV model uses
R version 4.1.2 for Windows (64 bit) (https://cran.r-project.org/bin/windows/base/old/4.1.2/R-4.1.2-win.exe).
This is, at the time of the release of this version of the CV model, the
current release of R for Windows and was published on 11/01/2021.
The installation process recommended for R is to simply use the standard windows installer with default options selected. The CV model does need to be collected to the installation of R via a file path to the installation location. This is described below in Installing the Model.
8.4 Installing the Model
The E8Plus model has been provided to SEMCOG in two parts:
A release zip file provided via GitHub that contains the model code and common model inputs files
A zip file of scenario inputs, called
Model Runs.zip, that was provided separately due to file size constraints for GitHub releases and data protection requirements.
The model installation process is described here.
8.4.1 Install the E8Plus Model Code
Download the release zip file
SEMCOG_CVTM-v1.0.0.zipfile from GitHub, https://github.com/SEMCOG/SEMCOG_CVTM/archive/v1.0.0.zipExtract the contents of
SEMCOG_CVTM-v1.0.0.zipinto the parent directory that you wish to use for the E8Plus model.
8.4.2 Set up the TransCAD Add-in
Set up the TransCAD add-in in TransCAD using the GIS Developers Kit. Open TransCAD and go to Tools > GIS Developer’s Kit > GISDK Toolbar to load the toolbar.
On the toolbar, click on the Compile to UI button, and compile the files in
SEMCOG_CVTM-v1.0.0/AddIn/rscinto a set ofDBDfiles inSEMCOG_CVTM-v1.0.0/AddIn. Compile:
SEMCOG Helper Utilities.rscintoutil_ui.dbdSEMCOG Scenario Manager.rscintoscen_ui.dbdSEMCOG.rscintosemcog_e8_ui.dbdSEMCOG_Dashboard.rscintodash_ui.dbdSEMCOG_Interop.lstintointerop_ui.dbdSEMCOG_Report.rscintoreport_ui.dbd- Note:
SEMCOG_Interop.lstcompilesSEMCOG_AQ.rsc,SEMCOG_AT_Toolbox.rsc, andSEMCOG_EJ.rsc.
- Go to Tools > GIS Developer’s Kit > Setup Add-Ins… and click “Add”.
- In Settings, select Dialog Box,
- In Description type “SEMCOG E8Plus”
- In Name type “SEMCOG Model”
- In UI Database browse to
SEMCOG_CVTM-v1.0.0/AddIn/semcog_e8_ui.dbd
8.4.3 Add the Scenario Inputs to the Model
Extract the contents of
Model Runs.zip(provided separately from the GitHub release). It contains a folder calledModel Runsand within that several scenario directories and the Input folders for those scenarios.Copy the
Model Runsfolder into the root directory of the model. The folder structure of the E8Plus model should now look like in the structure shown in the figure below.
Figure 47: SEMCOG E8Plus Model Root Level Directories
semcog_cvtm_vX.X.X
* AddIn TransCAD add-in directory
* CVM CV Model directory
* docs Documentation directory
* Model Runs Scenarios directory
* README.md Model repository README file
8.4.4 Install Pandoc
- If this is the first time that this machine has been used to run a
version of the CV model, install
pandocusing the installer package provided,SEMCOG_CVTM-v1.0.0/CVM/lib/pandoc-1.19-windows.msi. Pandoc is an application that converts text files from one format to another. In this case, Pandoc is required by the CV model’s dashboard component to convert output from an R markdown (.Rmd) format file into an HTML format file.
Figure 48: Pandoc Installer
8.4.5 Configure the CV Model
- Set up the connection between the CV model and R:
- Open the batch file
SEMCOG_CVTM-v1.0.0/CVM/run_semcog_cvtm.bat - Check the
pathtorinsidemodelandpathtorcdrivevariables. R can be installed in a variety of locations on the machine: inC:\Program Files\R\R-4.1.2\bin\x64for example, or inside the model itself (the recommended location in this case is inSEMCOG_CVTM-v1.0.0/CVM/lib/pkgs. - Update the path for the location that R is installed at on this computer – only one of the two paths, either inside the model or in another location on the machine – needs to be set. The relevant section in the batch file is shown below.
Figure 49: Setting path to R installation in run_semcog_cvtm.bat
:: User Defined Paths to R installation
:: for R installed inside the model, e.g. ".\lib\pkgs\R-4.1.2\bin\x64"
:: (if there is not an R installation at this location inside the model,
:: the C drive location specified in "pathtorcdrive" below will be checked)
set pathtorinsidemodel=".\lib\pkgs\R-4.1.2\bin\x64"
:: for R installed elsewhere such as the typical program files location, for example "C:\Program Files\R\R-4.1.2\bin\x64"
:: (if there is not an R installation at this location either, the batch file will pause with a message and then exit)
set pathtorcdrive="C:\Program Files\R\R-4.1.2\bin\x64"
- Set the appropriate number of cores for the size of computer that the model is installed on:
- The file
SEMCOG_CVTM-v1.0.0/CVM/lib/scripts/_USER_VARIABLES.Rcontains a settingUSER_PROCESSOR_CORES, as shown the figure below.
Figure 50: USER VARIABLES Script, Setting for USER_PROCESSOR_CORES
# User options to control hardware use, and run mode
USER_PROCESSOR_CORES <- 5L # How many processors should be used during model runs? (a number, followed by L to indicate integer)- If
USER_PROCESSOR_CORESis set to a value more than 1, several of the steps in the model will be parallelized. While this reduces run time, it requires more memory, and, in certain circumstances, the overhead of parallel process data copying and management can reduce or remove the benefit of parallelization. This is discussed in more detail in the section on Model Configuration on Different Hardware, but the general recommendations based on computer RAM are: - For a computer that just meets the minimum memory requirements for
the model, 32GB of RAM: set
USER_PROCESSOR_CORESto 1. - For a larger modeling server, with 128GB of RAM or more: set
USER_PROCESSOR_CORESto 5. (More than 5 is unnecessary as it has not been found to provide further reductions in run time). - For a computer with an amount of RAM in between those values, such
as 64GB or 80GB of RAM, experimentation has not found run time savings
with values of 2 or 3, so it recommended to set
USER_PROCESSOR_CORESequal to 1.
8.5 Running the E8Plus Model from TransCAD
This section of the user guide provides instructions on how to run the CV model as part of the E8Plus model from TransCAD, as either a complete system run or running the CV model step by step.
- Launch the E8Plus Model GUI: During installation, the TransCAD Add-In was configured (Set up the TransCAD Add-in) and the scenario inputs were added to the file system (Add the Scenario Inputs to the Model). Therefore, when TransCAD is opened the model’s GUI can be launched by selecting Tools > GIS Developer’s Kit > Add-Ins > SEMCOG E8Plus. The E8Plus model’s GUI is shown in the screenshot below. This shows the modified main GUI, which includes the CV model step, labelled “Commercial Vehicle”, between the “Mode Choice” and “Trip Assignment” steps.
Figure 51: E8Plus Model GUI
- Step Selection: As with the other main model component in the E8Plus model, the CV model is comprised of a series of steps. These can be selected in order to run one or more steps individually using the step settings buttons along the left side of the GUI. The all step settings button and the button for the commercial vehicle step both allow access to the CV model’s steps as shown the in following screenshots. For a complete run of the E8Plus model, all steps should be selected.
Figure 52: E8Plus Model GUI, All Step Settings
Figure 53: E8Plus Model GUI, Commercial Vehicle Step Settings
- The scenario GUI has also been modified to include CV model inputs. The screenshot shows several of the new inputs. Before running the model, ensure that all of the paths to the input for the scenario are correct and the file can be found by TransCAD (i.e., it shows as green). To change the path to a file, double click on the row of the file in the scenario GUI and browse to the file.
Figure 54: E8Plus Model GUI, Scenario Inputs
Running a complete E8Plus model run: once all of the scenario inputs have been set in the scenario GUI, a complete run of the E8Plus model can be started by clicking on the “Initialization” button on the Run Model tab of the E8Plus model GUI. Before clicking the “Initialization” button, ensure that the “Stop After stages” check box just above the button is unchecked, or the model will only run the initialization step. If you want a model report to be produced at the end of the run, check the “Create Report” check box and select the sections of the report required in the report dialog box. A complete run of the model with 5 feedback loops will take 12-24 hours depending on the size and speed of the computer.
Running a partial E8Plus model run: the model can be run in steps. If the “Stop After stages” check box is selected, individual steps of the model can be run. This generally requires that this scenario has been run to the stage before, so upstream inputs to a stage are available. The step selection menus can be used to selected particular steps within a larger stage. Again, the model must be run in sequence and so, generally, particular steps cannot be skipped over without leading to errors or anomalous results.
8.6 Model Configuration on Different Hardware
The _USER_VARIABLES.R script includes settings for the
USER_PROCESSOR_CORES global variable that indicates how
many processors should be used during the model runs. The CVTM
components of the CV model runs entirely as a multithreaded model. The
other portions of the CV model that are parallelized are the time of day
skim reading and processing into the combined time of day and buffer
skim. As this loops over time of day period, it works well with 5 cores
as there are 5 periods to loop over. The skim processing occurs in the
cv_sim_process_inputs.R script. Finally, the writing of
trip tables into the .omx files is also parallelized, which occurs in
the trip table step of the model.
Overall, running the CV model with 5 cores for these three portions of the model reduces run time significantly compared to a single thread. The overall run time is about 25 minutes (varies plus or minus a few minute depending on computer performance and other conflicting resource use during the run) with the CVTM + trip tables taking most of that, about 23 minutes, and with around two minutes for firm syn and LD. Therefore, total run time over 5 feedback iterations would be around two hours in total.
Running the model with more (say 10) cores only reduces the run time of the CVTM + trip table steps marginally, and total RAM required including the calling process is much higher in total as each process is working with a copy of the skim tables in memory.
Similarly, using 2 or 3 cores on a smaller machine does not seem to improve run time, likely due to the overhead now being comparable to any time savings of two rounds of moving data to and from the worker processors.
Therefore, it is recommended to leave the model running on a single thread except on larger computers (e.g., a modeling workstation or server that meets the recommend specifications described in Hardware Requirements) where 5 cores can be used.
8.7 Special Considerations for Running an Alternative or Future Year Scenario
Running an alternative or future year scenario, where the scenario
year is later than the base year, requires an additional input in
addition to those for running a base year run that is produced by the
base year run. This is because the firm synthesis for future years
pivots from the base year firm synthesis outputs (specifically the
firm_sim_results$FirmsScaled table in
scenarios/base/outputs/1.Firms.RData). The path to the base
year firm synthesis output is an optional scenario input,
Base Year Firms, that is listed in the scenario GUI. For
future year runs this must be present for the scenario to run, and
therefore the base year scenario must have already been run.
8.8 Running the E8Plus Model with Static CV Model Trip Tables
Since the E8Plus model including the CV model is more computationally intensive than previous versions of SEMCOG’s trip based travel demand model, and in particular the RAM requirements for hardware have increased, a feature has been added to allow the E8Plus model to be run on computers with lower amounts of RAM. This feature sets up the model scenario to bypass the CV model and instead uses static trip tables as a model input (i.e., the “static” trip tables are trip table that were saved from an earlier run of the CV model and will not be updated during this model run). During a model run, the previously saved CV model trip tables are added to the time period and daily trip tables in place of the CV model being run to generate new trip tables.
8.8.1 Settings for Running a Scenario with Static CV Model Trip Tables
The E8Plus model’s scenario GUI has been modified to include one
additional parameter under the CVM stage,
UseCVMStaticTrips. As noted in the parameter description
field in the scenario GUI, this parameter should be set to 1 to use
static CVM trips tables and 0 to run the CV model normally.
Figure 55: E8Plus Model GUI, and the UseCVMStaticTrips CV Model Parameter
8.8.2 Input Requirements for Running a Scenario with Static CVM Trip Tables
The E8Plus model’s scenario GUI has also been modified to react to
the setting of the UseCVMStaticTrips parameter to either
require the presence of the typical CV model inputs if it is set to 0 or
require two previously saved inputs representing the CV model trip
tables and a summary of the results used in reporting. The figures below
show the state of the scenario GUI’s input tab when
UseCVMStaticTrips is set to 1, first showing the gray
colored standard inputs market “Not used” and second showing the now
required “CVM Static Trip Table” and “CVM Trip Gen Summary”.
While the location of the two files is flexible, it is recommended to
locate them in the Model Runs\[scenario name]\Input\SED
directory along with the other CV model specific inputs.
Figure 56: E8Plus Model Scenario Inputs with UseCVMStaticTrips Parameter set to 1, CV model inputs not used
Figure 57: E8Plus Model Scenario Inputs with UseCVMStaticTrips Parameter set to 1, CV model static outputs required as inputs
8.8.2.1 CVM Static Trip Table
The CV model static trip tables file is an .omx file and contains all of the time period and daily vehicle class CV model trip tables. It is generated in a normal run of the E8Plus model including running the CV model by selecting the user option to save a single consolidated set of CV model trip tables in one .omx file.
In the CVM/lib/scripts/_USER_VARIABLES.R script, this
option can be selected as shown in the figure below. If this option is
used, the trip tables will be saved in the
Model Runs/[scenario name]/Output/CVM folder, into a file
named CV_Trip_Tables.omx. The .omx file can then be
manually copied to the Model Runs\[scenario name]\Input\SED
directory to set up scenario inputs correctly for a static trip table
run.
Figure 58: USER VARIABLES Script Saving Combined Trip Tables File
8.8.2.2 CVM Trip Gen Summary
An output from the CV model is
CV_Trip_Generation_Summary.csv, which contains the number
of trip origins and destinations by commercial vehicle class for each
TAZ. It is generated by default in a normal run of the E8Plus model
including running the CV model. The .csv file will be saved in the
Model Runs/[scenario name]/Output/CVM folder and can then
be manually copied to the
Model Runs\[scenario name]\Input\SED directory to set up
scenario inputs correctly for a static trip table run.
8.8.3 Executing a Run of the E8Plus Model with Static CV Model Trip Tables
Once the scenario parameter UseCVMStaticTrips is set to
1 and the CVM Static Trip Table and CVM Trip Gen Summary files are
referenced in the scenario GUI, the E8Plus model can be run as normal as
described above in Running the E8Plus Model
from TransCAD.
9 Creating and Running Scenarios
9.1 Introduction
This chapter of the user guide describes how to use the CV model to evaluate alternative scenarios. This includes the resources provided with the model to support creating and updating CV model inputs, and how to use the E8Plus model’s scenario GUI to create and run alternative scenarios.
The chapter also describes how to make some slightly more significant changes to CV model inputs beyond what is typically required to create a scenario to deal with cases where changes are made to the overall E8Plus model system and changes are required in the CV model to ensure that the model’s remain consistent in the scenario. Larger and longer term model modifications, where changes would require re-calibration of the base year model, are beyond the scope of this chapter and are discussed in the model documentation.
9.2 Contents of the chapter
The chapter is divided into several sections:
- Creating Scenarios
in the E8Plus Model
- General scenario creation in TransCAD and the CV model
- File structure for scenarios
- Input files needed to create a complete CV model scenario
- Settings options specific to a scenario
- Running a scenario
- Reviewing scenario results
- About Scenario Testing
- What is scenario testing?
- What types of scenarios can the CV model test?
- Scenario testing considerations (e.g., reasonable expectations around level of detail/granularity and uncertainty)
- Input Development
Resources
- Documentation of development files provided with the model to support model input development and modification.
- Making Changes to the Model System
9.3 Creating Scenarios in the E8Plus Model
This section explains step by step instructions for using a combination of the E8Plus Model’s GUI and the file system to create scenarios, and then discusses several CV model specific items related to scenario creation, including the CV model specific scenario inputs and how to modify them, scenario parameters, and run options that affect how the CV model runs.
9.3.1 General scenario creation in TransCAD and the CV model
During installation, the TransCAD Add-In was configured (Set up the TransCAD Add-in) and scenario inputs for the base and a set of interim and future year scenarios were added to the file system (Add the Scenario Inputs to the Model). Therefore, when TransCAD is opened the model’s GUI can be launched by selecting Tools > GIS Developer’s Kit > Add-Ins > SEMCOG E8Plus.
The GUI can be used to add, copy, and delete scenarios by using the scenario buttons below the scenario window on the GUI. The buttons are highlighted in red. - The button with a green plus sign adds a new scenario using the default scenario settings - The button with two documents copies the selected scenario - The button with a red cross deletes the selected scenario
Figure 59: E8Plus Model GUI Scenario Buttons
Clicking on the green plus sign button to add a scenario launches the scenario dialog box populated with default scenario paths to inputs and outputs and the default values for parameters. It does not create the underlying file structure for the new scenario. The figures below show the inputs and general tabs in the scenario GUI before any changes are made.
Figure 60: E8Plus Model GUI Scenario GUI, New Scenario
Figure 61: E8Plus Model GUI Scenario GUI, New Scenario
The inputs tab in the scenario GUI can then be used to browse to the locations of an alternative scenario’s input files. However, that structure first needs to be manually created by, for example, copying an existing scenario and replacing input files with new versions of the file that represent the scenario to be tested. The File structure for scenarios explains the file structure requirements for a scenario, while Input files needed to create a complete CV model scenario focuses on the input files required by the CV model.
A slightly more straightforward method to start the scenario creation process is to use the second of the scenario buttons that copies a selected scenario. This also does not create the underlying file structure for the scenario but does bring over the various settings from the selected scenario. The figures below show the inputs and general tabs in the scenario GUI after copying the current 2045 scenario and before any changes are made.
Figure 62: E8Plus Model GUI Scenario GUI Input Tab, Copied Scenario
Figure 63: E8Plus Model GUI Scenario GUI General Tab, Copied Scenario
The new scenario appears in the scenario GUI. When copying an existing scenario, if the existing scenario is complete, the scenario will already show as ready to run (i.e., there will be a green check mark next to the scenario name). However, the input and output paths to the new scenario’s location still need to be updated using the scenario dialog before making a run of the new scenario.
Figure 64: E8Plus Model GUI Scenario Added
9.3.2 File structure for scenarios
The scenario input and output file and folder structure has been modified slightly in the E8Plus model to incorporate the CV model. At the folder level, the input structure is unchanged with a new folder, “CVM”, added for CV model outputs. The figure below shows the folder structure, and how a new alternative scenario can be structured by following the structure currently used for scenarios.
The figure calls out new scenario inputs required by the CV model, which are described in turn in the section Input files needed to create a complete CV model scenario, and shows where they are typically located. It also shows new or modified outputs produced by the E8Plus model including highway skims and trip tables in .omx format and other CV model outputs.
The actual locations and filenames of scenario inputs to the CV model are flexible as they are not hard coded in the CV model but rather the files are selected using the scenario GUI and their file path and name are passed as variables to the CV model. The full list of CV model inputs and outputs are shown in the sections on CV Model Data Inputs and CV Model Outputs.
Figure 65: SEMCOG E8Plus Model Scenarios File Structure
semcog_cvtm_vX.X.X
* Model Runs Scenarios directory
+ 2015_KA15 Base year scenario directory
+ 2020_KC20 Future (2020) scenario directory
+ 2025_KC25 Future (2025) scenario directory
+ 2030_KC30 Future (2030) scenario directory
+ 2035_KC35 Future (2035) scenario directory
+ 2040_KC40 Future (2040) scenario directory
+ 2045_KC45 Future (2045) scenario directory
+ 2045_KC45_Alt Example alternative future (2045) scenario directory
- Input Alternative scenario inputs directory
+ AQ Air quality inputs
+ EJ EJ inputs
+ Network Networks
+ Params Parameters
+ SED Socioeconomic data
- CVM_Externals.csv External stations active in the scenario, used in the CVM
- CVM_EE_Region.csv Daily EE Trucks by External Region, used in the CVM
- CVM_IEEI_Region.csv Daily IE and EI Trucks by External Region, used in the CVM
- Facilities.csv Freight activity centers facilties used in the CVM
- TAZSocioEconomicsBuffer.csv TAZ data for the buffer region used in the CVM
- Output Alternative scenario outputs directory
+ AQ Air quality outputs
+ CVM CVM outputs
- 1.Firms.Rdata List of Firms from Frim Synthesis
- 2.LongDistanceTrips.Rdata Long Distance Truck Trips
- 3.CommercialVehicleTrips.Rdata Commercial Vehicle Tours and Trips
- 4.TripTables.Rdata Commercial Vehicle Trip Database
- CV_Trip_Generation_Summary.csv Commercial Vehicle Trip Generation
- ReportDashboard.html Commercial Vehicle Dashboard
- (additional output files) (See CV Model Outputs for comprehensive list)
+ EJ EJ outputs
+ HAssign Highway assignment outputs
- Flow_AM.csv AM time period assignment flow table in .csv format
- Flow_DY.csv Daily assignment flow table in .csv format
- Flow_EV.csv EV time period assignment flow table in .csv format
- Flow_MD.csv MD time period assignment flow table in .csv format
- Flow_NT.csv NT time period assignment flow table in .csv format
- Flow_PM.csv PM time period assignment flow table in .csv format
- OD_AM.omx AM time period trip tables in .omx format
- OD_DY.omx Daily trip tables in .omx format
- OD_EV.omx EV time period trip tables in .omx format
- OD_MD.omx MD time period trip tables in .omx format
- OD_NT.omx NT time period trip tables in .omx format
- OD_PM.omx PM time period trip tables in .omx format
+ HNet Highway network
+ HSkm Highway skims
- AM_HwySkim.omx AM time period highway skims in .omx format
- EV_HwySkim.omx EV time period highway skims in .omx format
- MD_HwySkim.omx MD time period highway skims in .omx format
- NT_HwySkim.omx NT time period highway skims in .omx format
- PM_HwySkim.omx PM time period highway skims in .omx format
+ ModeChoice Mode choice model outputs
+ TNet Transit network
+ Tripdist Trip distribution outputs
+ Tripgen Trip generation outputs
+ TrnAssign Transit assignment outputs
+ TSkm Transit skims
9.3.3 Input files needed to create a complete CV model scenario
9.3.3.1 Introduction
In this section, each scenario input to the commercial vehicle model,
as contained in the Model Runs/[scenario name] directory,
is described. The scenario specific inputs include static input file
prepared before a scenario is run and those produced during a run such
as skims matrices.
Table 89: List of inputs that are scenario specific
| Filename | Location | Description |
|---|---|---|
| CVM_EE_Region.csv | Input/SED | External region to external region growth factors |
| CVM_Externals.csv | Input/SED | External stations active in the scenario, and external allocation factors for trucks |
| CVM_IEEI_Region.csv | Input/SED | Internal to external and external to internal external region control totals |
| Facilities.csv | Input/SED | Descriptions of high-intensity freight locations |
| TAZSocioEconomicsBuffer.csv | Input/SED | Employment, households, and population data by TAZ for the buffer TAZs |
| base_transcad_taz.csv | Input/SED | Employment, households, and population data by TAZ for the SEMCOG internal TAZs |
9.3.3.2 YYYY_transcad_taz.csv
File description: Employment, households, and population data by TAZ for the SEMCOG internal TAZs
This file is the socioeconomic data for the internal SEMCOG portion
of the model region for the scenario year (the file’s title varies,
beginning with a four digit year followed by “_transcad_taz.csv”). The
data are combined with the socioeconomic data for the buffer portion of
the model region and are used to control the CV model’s firm synthesis
to ensure that the total employment by TAZ and employment category
matches with TAZ level employment assumptions. The table below shows the
first few rows of YYYY_transcad_taz.csv. The file is
imported to the CV model during a model run from the
Model Runs/[scenario name]/Input/SED directory.
Table 90: First few rows of YYYY_transcad_taz.csv
The table below describes the fields in
YYYY_transcad_taz.csv.
Table 91: Field Descriptions for YYYY_transcad_taz.csv
| Field | Description |
|---|---|
| zoneid | TAZ Code |
| taz | Long TAZ identifier |
| tot_acres | TAZ area in acres |
| tot_hhs | Number of households |
| hhs_pop | Household population |
| workers | Number of workers |
| grppop | Group quarters population |
| tot_pop | Total population |
| k12enroll | K-12 school enrollment |
| tot_emp | Total employment |
| e01_nrm | Emplyoment in Natural Resources & Mining |
| e02_constr | Emplyoment in Construction |
| e03_manuf | Emplyoment in Manufacturing |
| e04_whole | Emplyoment in Wholesale Trade |
| e05_retail | Emplyoment in Retail Trade |
| e06_trans | Emplyoment in Transportation & Warehousing |
| e07_utility | Emplyoment in Utilities |
| e08_infor | Emplyoment in Information |
| e09_finan | Emplyoment in Financial Activities |
| e10_pstsvc | Emplyoment in Professional, Scientific, & Technical Services |
| e11_compmgt | Emplyoment in Management of Companies & Enterprises |
| e12_admsvc | Emplyoment in Administrative, Support, & Waste Services |
| e13_edusvc | Emplyoment in Education Services |
| e14_medfac | Emplyoment in Medical Facilities |
| e15_hospit | Emplyoment in Hospitals |
| e16_leisure | Emplyoment in Leisure & Hospitality |
| e17_othsvc | Emplyoment in Other Services |
| e18_pubadm | Emplyoment in Public Administration |
| resid_sf | Residential building square footage |
| retail_sf | Retail building square footage |
| entmt_sf | Entertainment building square footage |
| othcom_sf | Other commercial building square footage |
| finan_sf | Financial building square footage |
| office_sf | Office building square footage |
| indust_sf | Industrail building square footage |
| medcare_sf | Medical care building square footage |
| medical_sf | Medical building square footage |
| edu_sf | Educational building square footage |
| othinst_sf | Other institutional building square footage |
9.3.3.3 CVM_Externals.csv
File description: External stations active in the scenario, and external allocation factors for trucks
This file contains a complete list of the external stations coded in
the model and an indication of whether they are active or not in the
current scenario. The file also contains allocation factors to
reallocate truck flows from the base year allocation to the scenario
allocation to account for changes in the active scenario and the
resulting need to reallocate where trucks are likely to enter and exit
the region. The table below shows the first few rows in
CVM_Externals.csv.
Table 92: All rows in CVM_Externals.csv
The table below describes the fields in
CVM_Externals.csv.
Table 93: Field Descriptions for CVM_Externals.csv
| Field | Description |
|---|---|
| TAZ | ID for the External Station TAZ |
| BaseActive | Whether the external station is active in the base scenario |
| Year | Year for the scenario |
| ScenarioActive | Whether the external station is active in the current scenario |
| BaseTAZ | Base external station TAZ that has flow allocated to this scenario external station TAZ |
| ScenarioProp | Proportion to the base external station TAZ’s volume that is allocated to this scenario external station TAZ |
9.3.3.4 CVM_EE_Region.csv
File description: External region to external region growth factors
This file contains growth rates for external to external truck
traffic to grow the base year external to external truck trips in the
long-distance model. The growth rates were estimated using Transearch
commodity flow data and are specific to groups of origin-destination
movements (e.g., “US South/Central, Mexico, Ohio Buffer TO Canada
Buffer/ON/QC/East”). The table below shows all rows in
CVM_EE_Region.csv.
The CVM_EE_Region.csv file is produced by the model
estimation script,
CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R. This
file was developed using Transearch data.
Table 94: All rows in CVM_EE_Region.csv
The table below describes the fields in
CVM_EE_Region.csv.
Table 95: Field Descriptions for CVM_EE_Region.csv
| Field | Description |
|---|---|
| xx_movement_od | ID for the group of external to external origin-destination movements |
| xx_movement_num | ID for the group of external to external origin-destination movements, by direction |
| xx_movement | Label for the group of external to external origin-destination movements, by direction |
| DailyTrucks_2015 | Number of daily trucks on that OD in the base year, from Transearch |
| Year | Year for the scenario |
| DailyTrucks | Number of daily trucks on that OD in the scenario year, from Transearch |
| Growth | Growth factor (ratio of daily trucks in the scenario year and base year |
9.3.3.5 CVM_IEEI_Region.csv
File description: Internal to external and external to internal external region control totals
This file contains growth rates for internal to external and external
to internal truck traffic to grow the base year external control totals
at the external stations to the region that used in the long-distance
model. The growth rates were estimated using Transearch commodity flow
data and are specific to groups of external origin and destination
regions (e.g., “Buffer Michigan”, “US East”). The table below shows all
rows in CVM_IEEI_Region.csv.
The CVM_IEEI_Region.csv file is produced by the model
estimation script,
CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R. This
file was developed using Transearch data.
Table 96: All rows in CVM_IEEI_Region.csv
The table below describes the fields in
CVM_IEEI_Region.csv.
Table 97: Field Descriptions for CVM_IEEI_Region.csv
| Field | Description |
|---|---|
| Region | Region name for the group of external movements |
| DailyTrucks | Number of daily trucks to and from that region in the scenario year, from Transearch |
9.3.3.6 Facilities.csv
File description: Descriptions of high-intensity freight locations
This file lists the significant distribution and intermodal
facilities assumed for the designated scenario. The locations are used
as special generators and attractors in the LDM and the CVTM. The table
below shows the first few rows of Facilities.csv.
Table 98: First few rows of Facilities.csv
The table below describes the fields in
Facilities.csv.
Table 99: Field Descriptions for Facilities.csv
| Field | Description |
|---|---|
| ID | sequential ID code for the facility |
| SHPName | Data source name |
| FacilityTy | Facility type |
| City | City the facility is located in |
| FacilityNm | Name of the facility |
| YearBuilt | Year facility was built |
| FloorSpace | Square footage of the facility |
| Acres | Acreage of the facility |
| FacilityID | Facility ID from the data source |
| TAZ | TAZ code of the facility |
| COUNTY | SEMCOG county code for the county facility is located in |
| CountyName | County that the facility is located in |
9.3.3.7 TAZSocioEconomicsBuffer.csv
File description: Employment, households, and population data by TAZ for the buffer TAZs
This file is the socioeconomic data for the buffer portion of the
model region for the scenario. The data are combined with the
socioeconomic data for the internal portion of the SEMCOG region and are
used to control the CV model’s firm synthesis to ensure that the total
employment by TAZ and employment category matches with TAZ level
employment assumptions. The table below shows the first few rows of
TAZSocioEconomicsBuffer.csv.
The file was processed from TAZ socioeconomic data in the MDOT
Statewide Model. The script CVM/dev/data_firm_sim.R was
used to process the database into the
TAZSocioEconomicsBuffer.csv file format.
Table 100: First few rows of TAZSocioEconomicsBuffer.csv
The table below describes the fields in
TAZSocioEconomicsBuffer.csv.
Table 101: Field Descriptions for TAZSocioEconomicsBuffer.csv
| Field | Description |
|---|---|
| TAZ | TAZ Code |
| HH | Number of Households |
| POP | Population |
| e01_nrm | Emplyoment in Natural Resources & Mining |
| e02_constr | Emplyoment in Construction |
| e03_manuf | Emplyoment in Manufacturing |
| e04_whole | Emplyoment in Wholesale Trade |
| e05_retail | Emplyoment in Retail Trade |
| e06_trans | Emplyoment in Transportation & Warehousing |
| e07_utility | Emplyoment in Utilities |
| e08_infor | Emplyoment in Information |
| e09_finan | Emplyoment in Financial Activities |
| e10_pstsvc | Emplyoment in Professional, Scientific, & Technical Services |
| e11_compmgt | Emplyoment in Management of Companies & Enterprises |
| e12_admsvc | Emplyoment in Administrative, Support, & Waste Services |
| e13_edusvc | Emplyoment in Education Services |
| e14_medfac | Emplyoment in Medical Facilities |
| e15_hospit | Emplyoment in Hospitals |
| e16_leisure | Emplyoment in Leisure & Hospitality |
| e17_othsvc | Emplyoment in Other Services |
| e18_pubadm | Emplyoment in Public Administration |
9.3.3.8 AM_HwySkim.omx, MD_HwySkim.omx, PM_HwySkim.omx, EV_HwySkim.omx, NT_HwySkim.omx
File description: Time period specific time, distance, and toll skims
These five scenario inputs contain the five time periods zone to zone travel distance matrices, travel time matrices, and toll matrices. At run time, the matrices are exported from the TransCAD .mtx format matrices into the .omx format file.
Upon being imported into the CV model, the matrices are converted
into lists in I-J-Value format and tagged with the TAZ numbering instead
of matrix indices. These matrices which represent internal travel with
the SEMCOG model region and travel to, from, and between the external
stations to the SEMCOG model region, are combined with the skim matrices
for the buffer region that are included in the common data inputs into a
single table called skims_tod. The table below shows the
first few rows of skims_tod.
Table 102: First few rows of skims_tod
The table below describes the fields in skims_tod.
Table 103: Field Descriptions for skims_tod
| Field | Description |
|---|---|
| OTAZ | Origin TAZ |
| DTAZ | Destination TAZ |
| time.NT | Travel time, night time period |
| dist.NT | Distance, night time period |
| toll.NT | Toll, night time period |
| time.AM | Travel time, AM peak time period |
| dist.AM | Distance, AM peak time period |
| toll.AM | Toll, AM peak time period |
| time.MD | Travel time, midday time period |
| dist.MD | Distance, midday time period |
| toll.MD | Toll, midday time period |
| time.PM | Travel time, PM peak time period |
| dist.PM | Distance, PM peak time period |
| toll.PM | Toll, PM peak time period |
| time.EV | Travel time, evening time period |
| dist.EV | Distance, evening time period |
| toll.EV | Toll, evening time period |
| time.avg | Travel time, daily average |
| dist.avg | Distance, daily average |
| toll.avg | Toll, daily average |
9.3.4 Settings options specific to a scenario
The CV model only has one scenario specific parameter. In general, any scenario specific values are contained in input files and parameters are set and calibrated for the base scenario and are not exposed to the scenario GUI for adjustment in a particular scenario.
The one scenario specific parameter that does affect the CV model is “BaseScenario” which indicates whether the scenario is the base scenario or not. Only one scenario (in this case the KA15 scenario for 2015) is a base scenario where the variable has a value of one and all others are not the base scenario and the variable should have a value of zero. The parameter is located in the parameters tab, under the “CVM” stage and the “Params” option.
Figure 66: E8Plus Model GUI Scenario GUI, BaseScenario Parameter
The E8Plus model does have various parameters available for adjustment in the scenario GUI. Since these primarily affect other components of the model than the CV model, they are not discussed here, but can be reviewed in the parameters tab of the scenario GUI.
One particular setting that should be checked prior to a run is whether feedback is activated and the number of feedback iterations that the model should run. This setting is located in the general tab of the scenario GUI. Checking the “run speed feedback” check box activates the feedback process and then the toggle box is used to enter the number of feedback iterations. The E8Plus model is typically run for 5 feedback iterations. When a fixed number of feedback iterations is entered, the convergence percentage should be left as zero percent to prevent an early termination to the model run.
Figure 67: E8Plus Model GUI Scenario GUI, BaseScenario Parameter
9.3.5 Running a scenario
The following instructions repeat those in Running the E8Plus Model from TransCAD, which explains how to run the E8Plus Model. In essence, running a scenario follows the same process, the key aspect being ensuring that the input files are set up and selected in the scenario GUI correctly, as discussed above.
- Launch the E8Plus Model GUI: During installation, the TransCAD Add-In was configured (Set up the TransCAD Add-in) and the scenario inputs were added to the file system (Add the Scenario Inputs to the Model). Therefore, when TransCAD is opened, the model’s GUI can be launched by selecting Tools > GIS Developer’s Kit > Add-Ins > SEMCOG E8Plus. The E8Plus model’s GUI is shown in the screenshot below. This shows the modified main GUI, which includes the CV model step, labelled “Commercial Vehicle”, between the “Mode Choice” and “Trip Assignment” steps.
Figure 68: E8Plus Model GUI
- Step Selection: As with the other main model components in the E8Plus model, the CV model is comprised of a series of steps. These can be selected in order to run one or more steps individually using the step settings buttons along the left side of the GUI. The all step settings button and the button for the commercial vehicle step both allow access to the CV model’s steps as shown the in following screenshots. For a complete run of the E8Plus model, all steps should be selected.
Figure 69: E8Plus Model GUI, All Step Settings
Figure 70: E8Plus Model GUI, Commercial Vehicle Step Settings
- The scenario GUI has also been modified to include CV model inputs. The screenshot shows several of the new inputs. Before running the model, ensure that all of the paths to the input files for the scenario are correct and that the files can be found by TransCAD (i.e., when TransCAD can find the file, it shows as green). To change the path to a file, double click on the row of the file in the scenario GUI and browse to the file.
Figure 71: E8Plus Model GUI, Scenario Inputs
- Running a complete E8Plus model run: once all of the scenario inputs have been set in the scenario GUI, a complete run of the E8Plus model can be started:
- Click on the scenario that is to be run, it will be highlighted
- The “Stop After stages” check box just on the Run Model tab of the E8Plus model GUI should be unchecked, or the model will only run the initialization step
- If you want a model report to be produced at the end of the run, check the “Create Report” check box on the Run Model tab of the E8Plus model GUI and select the sections of the report required in the report dialog box.
- Click on the “Initialization” button on the Run Model tab of the E8Plus model GUI. The run will then start. A complete run of the model with 5 feedback loops will take 12-24 hours depending on the size and speed of the computer.
- Running a partial E8Plus model run: the model can be run in steps. If the “Stop After stages” check box is selected, individual steps of the model can be run. This generally requires that this scenario has been run to the stage before, so upstream inputs to a stage are available. The step selection menus can be used to select particular steps within a larger stage. Again, the model must be run in sequence and so, generally, particular steps cannot be skipped over without leading to errors or anomalous results.
9.3.6 Reviewing scenario results
The outputs from the CV model are described in full in the chapter Model Outputs, and summarized in the table
below. The main tabular outputs from the CV model for a particular
scenario are included in the Output/CVM folder for the
scenario. The output files are either .csv format files or R binary
format files (.Rdata) that contain compressed tables. The tables in each
of the .Rdata files are exported to .csv format files so it is
unnecessary to open the databases in R if the model user prefers to used
Excel or another alternative software application for analysis.
In addition to the .csv and .Rdata files, the CV model adds trip
tables to the .omx trip tables in the Output/HAssign folder
for the scenario. The .omx trip tables can be opened in TransCAD and
summarized using TransCAD’s matrix manipulation tools in the same way as
trip tables that use Caliper’s .mtx matrix format.
In addition to the main tabular and matrix outputs, the model produces summary outputs which are also listed in the table below. These tabular, matrix, and summary outputs are in addition to the downstream outputs such as vehicle link flow tables and HTML reports that the E8Plus model’s highway assignment and reporting steps produce.
Table 104: List of model outputs
| Filename | Location | Description |
|---|---|---|
| 1.Firms.RData | Output/CVM | Database of inputs and outputs from Firm Synthesis model component |
| 1.Firms.RData.BaseYearFirms.csv | Output/CVM | CSV export of firm synthesis base year firms table |
| 1.Firms.RData.ScenarioFirms.csv | Output/CVM | CSV export of firm synthesis scenario firms table |
| 1.Firms.RData.TAZLandUseCVTM.csv | Output/CVM | CSV export of processed land use database |
| 2.LongDistanceTrips.RData | Output/CVM | Dababase of inputs and outputs from Long Distance model component |
| 2.LongDistanceTrips.RData.ld_trips.csv | Output/CVM | CSV export of long distance model trip database |
| 2.LongDistanceTrips.RData.zoneProdsAttrs.csv | Output/CVM | CSV export of long distance model zonal productions and attractions |
| 3.CommercialVehicleTrips.RData | Output/CVM | Database of inputs and outputs from CVTM model component |
| 3.CommercialVehicleTrips.RData.cv_trips.csv | Output/CVM | CSV export of CVTM trips and tours table |
| 4.TripTables.RData | Output/CVM | Database of inputs and outputs from Trip Table model component |
| 4.TripTables.RData.TripTable.csv | Output/CVM | CSV export of trip table in list format |
| 5.DashboardTables.RData | Output/CVM | Database of inputs to the dashboard component |
| CVTM_Stops_Duration_Activity_Non-Scheduled.csv | Output/CVM | CVTM stop durations by activity, non scheduled (dashboard) |
| CVTM_Stops_Duration_Activity_Scheduled.csv | Output/CVM | CVTM stop durations by activity, scheduled (dashboard) |
| CVTM_Stops_Duration_Vehicle.csv | Output/CVM | CVTM stop durations by vehicle (dashboard) |
| CVTM_Stops_Duration_Vehicle_Activity.csv | Output/CVM | CVTM stop durations by activity, vehicle (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Activity.csv | Output/CVM | CVTM first stop arrival by activity (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Vehicle.csv | Output/CVM | CVTM first stop arrival by vehicle (dashboard) |
| CVTM_Stops_First_Stop_Arrival_Time_Vehicle_Activity.csv | Output/CVM | CVTM first stop arrival by activity, vehicle (dashboard) |
| CV_Trip_Generation_Summary.csv | Output/CVM | CV model trip generation summary (for HTML report) |
| CV_Trip_VMT_VHT_Summary.csv | Output/CVM | CV model trip, VMT, and VHT summary (dashboard) |
| Commercial_County_Vehicle_Segment.csv | Output/CVM | CVTM summary by count, vehicle, OD segment (dashboard) |
| Commercial_County_Vehicle_TOD.csv | Output/CVM | CVTM summary by count, vehicle, time of day (dashboard) |
| Commercial_Heavy_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, heavy vehicles (dashboard) |
| Commercial_Light_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, light vehicles (dashboard) |
| Commercial_Medium_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by stop activity, medium vehicles (dashboard) |
| Commercial_Stop_Activity_Tour_Composition.csv | Output/CVM | CVTM tours by activity (dashboard) |
| Commercial_Stop_Activity_by_County.csv | Output/CVM | CVTM stop activity by county (dashboard) |
| Commercial_Stop_Activity_by_County_Heavy.csv | Output/CVM | CVTM stop activity by county for heavy vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Light.csv | Output/CVM | CVTM stop activity by county for light vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Medium.csv | Output/CVM | CVTM stop activity by county for medium vehicles (dashboard) |
| Commercial_Stop_Activity_by_County_Vehicle.csv | Output/CVM | CVTM trip Activity by vehicle type (dashboard) |
| Commercial_Tour_NonScheduled_Stops_Tours.csv | Output/CVM | CVTM tours by number of non scheduled stops (dashboard) |
| Commercial_Tour_Scheduled_Stops_Tours.csv | Output/CVM | CVTM tours by number of scheduled stops (dashboard) |
| Commercial_Tour_Start_Locations_by_Region.csv | Output/CVM | CVTM tour start locations by region (dashboard) |
| Commercial_Tour_Stops_Tours.csv | Output/CVM | CVTM tours stops by tour (dashboard) |
| Commercial_Trip_Destinations_by_County_Buffer.csv | Output/CVM | CVTM trip destinations by SEMCOG county and buffer (dashboard) |
| Commercial_Trip_OD_Vehicle.csv | Output/CVM | CVTM trips by OD segment by vehicles (dashboard) |
| Commercial_Trip_Origins_by_County_Buffer.csv | Output/CVM | CVTM trip origins by SECMOG county and buffer (dashboard) |
| Commercial_Trip_TOD_Vehicle.csv | Output/CVM | CVTM trips by time of day and vehicle (dashboard) |
| Commercial_Trips_II_County_County.csv | Output/CVM | CVTM II trips by county OD (dashboard) |
| Commercial_Trips_IX_County_External.csv | Output/CVM | CVTM IX trips by county-external OD (dashboard) |
| Commercial_Trips_XI_External_County.csv | Output/CVM | CVTM XI trips by external-county OD (dashboard) |
| Commercial_Trips_XX_External_External.csv | Output/CVM | CVTM XX trips by external-external OD (dashboard) |
| Employees_by_County.csv | Output/CVM | Firm Synthesis: Employees by County (dashboard) |
| Employees_by_Industry.csv | Output/CVM | Firm Synthesis: Employees by Industry (dashboard) |
| Employees_by_Industry_and_County.csv | Output/CVM | Firm Synthesis: Employees by Industry and County (dashboard) |
| Employees_by_Industry_and_Size_Buffer.csv | Output/CVM | Firm Synthesis: Employees by Industry and Firm Size in the buffer (dashboard) |
| Employees_by_Industry_and_Size_SEMCOG.csv | Output/CVM | Firm Synthesis: Employees by Industry and Firm Size, SEMCOG (dashboard) |
| Employees_by_Size.csv | Output/CVM | Firm Synthesis: Employees by Firm Size (dashboard) |
| Facilities_by_Type.csv | Output/CVM | Firm Synthesis: Facilites by facility type (dashboard) |
| Firms_by_County.csv | Output/CVM | Firm Synthesis: Firms by County (dashboard) |
| Firms_by_Industry.csv | Output/CVM | Firm Synthesis: Firms by Industry (dashboard) |
| Firms_by_Industry_and_County.csv | Output/CVM | Firm Synthesis: Firms by Industry and County (dashboard) |
| Firms_by_Industry_and_Size_Buffer.csv | Output/CVM | Firm Synthesis: Firms by Industry and Firm Size in the buffer (dashboard) |
| Firms_by_Industry_and_Size_SEMCOG.csv | Output/CVM | Firm Synthesis: Firms by Industry and Firm Size, SEMCOG (dashboard) |
| Firms_by_Size.csv | Output/CVM | Firm Synthesis: Firms by Firm Size (dashboard) |
| Long-Distance_Trips_IXXIODSummary.csv | Output/CVM | Long Distance Trips IX and XI OD Summary (dashboard) |
| Long-Distance_Trips_IX_County_External.csv | Output/CVM | Long Distance IX Trips by County and External (dashboard) |
| Long-Distance_Trips_XI_External_County.csv | Output/CVM | Long Distance XI Trips by External and County (dashboard) |
| Long-Distance_Trips_XX_External_External.csv | Output/CVM | Long Distance XX Trips by External OD (dashboard) |
| Long-Distance_Trips_XX_ODSummary.csv | Output/CVM | Long Distance XX Trips OD Summary (dashboard) |
| Long-Distance_Trips_by_County.csv | Output/CVM | Long Distance Trips by County (dashboard) |
| Long-Distance_Trips_by_DistBin.csv | Output/CVM | Long Distance Trips by Distance range (dashboard) |
| Long-Distance_Trips_by_External.csv | Output/CVM | Long Distance Trips by External Station (dashboard) |
| Long-Distance_Trips_by_External_TAZ.csv | Output/CVM | Long Distance Trips by External and TAZ (dashboard) |
| Long-Distance_Trips_by_External_TOD.csv | Output/CVM | Long Distance Trips by External and Time Period (dashboard) |
| Long-Distance_Trips_by_External_TOD_ODSeg.csv | Output/CVM | Long Distance Trips by External, Time Period and OD segment (dashboard) |
| Long-Distance_Trips_by_TOD.csv | Output/CVM | Long Distance Trips by Time Period (dashboard) |
| ReportDashboard.html | Output/CVM | CVM dashboard for the scenario |
| TripTable_Heavy_PctTrips_by_County_OD.csv | Output/CVM | Trip table heavy percent trips by county OD (dashboard) |
| TripTable_Heavy_Trips_by_County_OD.csv | Output/CVM | Trip table heavy trips by county OD (dashboard) |
| TripTable_Light_PctTrips_by_County_OD.csv | Output/CVM | Trip table light percent trips by county OD (dashboard) |
| TripTable_Light_Trips_by_County_OD.csv | Output/CVM | Trip table light trips by county OD (dashboard) |
| TripTable_Medium_PctTrips_by_County_OD.csv | Output/CVM | Trip table medium percent trips by county OD (dashboard) |
| TripTable_Medium_Trips_by_County_OD.csv | Output/CVM | Trip table medium trips by county OD (dashboard) |
| TripTable_PctTrips_by_County_OD.csv | Output/CVM | Trip table percent trips by county OD (dashboard) |
| TripTable_Trips_IX_County_External.csv | Output/CVM | Trip table IX trips by county external OD (dashboard) |
| TripTable_Trips_XI_External_County.csv | Output/CVM | Trip table XI trips by external county OD (dashboard) |
| TripTable_Trips_XX_External_External.csv | Output/CVM | Trip table XX trips by external external OD (dashboard) |
| TripTable_Trips_by_County_OD.csv | Output/CVM | Trip table trips by county OD (dashboard) |
| TripTable_Trips_by_Vehicle_Segment_TOD.csv | Output/CVM | Trip table trips by vehicle, OD segment, and time period (dashboard) |
| Validation_2015_KA15_2022_07_04_08_15_17.xlsx | Output/CVM | Validation spreadsheet with time and date naming (dashboard) |
| log | Output/CVM | Folder of log files for model runs |
| skims_buffer.rds | Output/CVM | Combined skims table including buffer region |
| skims_tod.rds | Output/CVM | Combined skims table for all time periods for SEMCOG region |
| OD_AM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for AM time period |
| OD_DY.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for full day |
| OD_EV.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for EV time period |
| OD_MD.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for MD time period |
| OD_NT.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for NT time period |
| OD_PM.omx | Output/Hassign | Trip table matrices in OMX format, all vehicle types for PM time period |
9.4 About Scenario Testing
This section discusses what scenario testing is and the types of scenario that can be evaluated using the CV model, along with some considerations for using a regional CV model.
9.4.1 What is scenario testing?
Scenario testing is the process of defining and testing the effects of several (or many) alternative futures using a structured, systematic process to vary input assumptions and compare outputs across scenarios. It is typically used in long term planning work in order to evaluate the effects of policies such as transportation investments as an input to policy selection and project prioritization processes.
A scenario is comprised of a complete set of inputs, assumptions, and outputs that represent a single alternative future. In the E8Plus model, a scenario represents a typical weekday of travel in the SEMCOG region during a specific year. The process of adding new scenarios to the E8Plus model was discussed above. In order to actually design scenarios, the model user needs to think about which specific inputs must be changed to describe the scenario and then consider the magnitude of change required in that input.
For example, to develop a scenario that evaluated a change in the future location of distribution centers, such as an increase in the development of distribution centers close to DFW airport, the model user would make some adjustments to the scenario input files that control the locations of freight facilities and also the input that describes employment by TAZ. Once the model user has designed and run the scenarios, the next step is to compare the outputs across scenarios for the same future time horizon to understand the relative impacts of different scenario assumptions. In this case, the expected effect is an increases in the number of truck trips starting and ending in the TAZs where the new distribution centers are located and an increase in truck volumes on the routes between the new distribution centers and entry points to the region, and also between the new distribution centers and the business and residential areas in the region that the shipments that move through the distribution centers are delivered in.
9.4.2 What types of scenarios can the CV model test?
The following are several types of scenario that can be evaluated using the CV model. Since the CV model is part of a system that also models passenger travel, many of the changes discussed here would also impact personal travel and the E8Plus model would capture the combined effects across all travel.
Change over Time: to understand temporal change, i.e., change over time, it is typical to start from a base year and review either a single set of assumed futures for a sequence of future years or, in a more complex analysis, compare more than one divergent future from the base year onward. The former is the initial case with the E8Plus model, with a sequence of scenarios set up for five year increments from the 2015 base year state to the 2045 horizon. The base year represents a calibrated model of an observed and recent condition, where there is confidence that the model adequately quantifies input values and demonstrates that key output measures are relatively close to real values. The assumption inherent in maintaining a single and common base year as the starting point for any temporal analysis is that we cannot change the past and we can only expect changes going into the future. These might be changes that are represented in the model system as the results of policy changes or investments made by government. However, the changes might also be the result of factors beyond the control of government such as new technological developments or business practices, and changes over time can also be used to capture inherent uncertainty in forecasted model inputs, such as population forecasts. As changes accumulate over time (e.g., the impacts of different assumptions about annual population growth incorporated in socioeconomic inputs), the output performance measures will often diverge away from each other.
Change due to Alternative Assumption: to understand the impacts of policy changes or other changes in the land use, economy, and transportation system in a region, it is common to select a particular future horizon and run a series of scenarios that vary the model inputs for that year. For example, the 30 year planning horizon is typically used in transportation planning. In order to isolate the effects of changes, a package of changes can be gradually introduced to the model system, such as keeping transportation inputs static at first and adding in population and employment changes in a first scenario, and then adding in the transportation system changes.
Socioeconomic Scenarios: The key socioeconomic inputs to the CV model are population and employment data at the TAZ level and commodity flow assumptions including growth forecasts, along with more detailed location information specific to the CV model such as freight facility locations.
The employment data for a scenario, in YYYY_transcad_taz.csv and TAZSocioEconomicsBuffer.csv control the magnitude of internally generated commercial vehicle flows within the SEMCOG region as well as the distribution of long distance external truck flows to and from TAZs within the SEMCOG region. The two sources of data cover the larger model region used by the CV model, with the first file describing employment in the SEMCOG region and the second covering the TAZs in the buffer region.
The population and household data for a scenarios, also in YYYY_transcad_taz.csv and TAZSocioEconomicsBuffer.csv, affect the magnitude of demand for and the distribution of internally generated commercial vehicle flows within the SEMCOG region.
The external commodity flow assumptions, in CVM_EE_Region.csv and CVM_IEEI_Region.csv, which were derived from Transearch data, provide growth assumptions for external to external and internal to external/external to internal truck flows by origin to destination region.
The freight facility location assumptions, in Facilities.csv, provide the locations of distribution centers and other truck intensive facilities. These locations affect the destinations of internal to external/external to internal truck flows, and when used with consistent employment data assumptions, also affects the generation of internal commercial vehicle tours.
Future year files (2020+) for each of these inputs can be updated in a scenario to represent higher or lower levels of employment and population change within and around the SEMCOG region, changes to commodity flows to, from, and through the region, and alternative assumptions about freight facility development. More complex changes can be made than simple overall growth rate changes. For example changing assumption about the distribution of employment growth within the region, changes in the rates of growth of particular industries, or changes in the spatial origin and destination of commodity flows outside of the region.
Transportation Network Scenarios: The key transportation network inputs to the CV model are travel time skims in time period skim matrix files produced during a model run, AM_HwySkim.omx, MD_HwySkim.omx, PM_HwySkim.omx, EV_HwySkim.omx, NT_HwySkim.omx, and information about the external stations that are active in the scenario, CVM_Externals.csv. The travel time skims capture the impacts on travel times of network changes such as new road construction projects as well as (in a model run with feedback) the effects of congestion (increase in which may be caused by, for example, growth in passenger travel). The external stations coded in the network, i.e., the locations at which the road network intersects with the boundary of the region, affect where commercial vehicles can enter and leave the region. The file provides the ability to assert changes in the distribution of external traffic compared to the base scenario.
9.4.3 Scenario testing considerations
When using regional travel models for scenario analysis and comparing results between scenarios, it is important that the model user has reasonable expectations around the level of detail and granularity of results and the inherent uncertainty in model inputs and the behavior that is being simulated. Some general guidance to consider:
Compare changes for larger spatial units, e.g., counties and subdivisions of counties instead of specific TAZs.
Small network or socioeconomic changes may results in insignificant changes and may result in counter intuitive (but small) directional changes due to small redistribution effects and other changes during model feedback.
Since truck and commercial vehicle volumes are often small, particularly on lower volume facilities, proportional changes can be large. This can be apparent in comparisons of assignment results when routing switches between closely competitive alternative routes along minor facilities. Comparing measures such as vehicle miles traveled between scenarios (for a region, county, or other spatial unit) may provide more certainty than volumes on minor facilities.
9.5 Input Development Resources
A package of files has been created and delivered along with the
model to support the creation of model inputs, model estimation,
calibration, and validation: semcog_cvtm\CVM\dev. The
package is intended to organize and document the development of the base
model inputs along with the estimation, calibration and validation of
the CV model. The data sources used in the model are described in the
model documentation and each individual file produced during model
development for use in the model is listed in the Data Inputs chapter of this user guide. The
scripts in semcog_cvtm\CVM\dev can also be used to modify
and update model inputs to support scenario testing. The contents of the
package are introduced below. More detailed information on the content
of individual scripts is contained within code comments.
The semcog_cvtm\CVM\dev package is divided into several
main directories, along with a sequence of scripts in the root of the
directory:
The
Calibrationfolder contains outputs from the model calibration process.The
Data Processedfolder contains the various inputs datasets and interim outputs used to create the model inputs.The
Estimationfolder contains inputs and outputs from the statistical estimation work undertaken to develop CV model components.The
User Guidefolder contains R and R markdown scripts, static images, and other files required to produce this model user guide.The
Validationfolder contains outputs from the model validation process.
The folder structure of semcog_cvtm\CVM\dev including
sub directories is shown in the figure below.
Figure 72: SEMCOG E8Plus Model Development File Structure
semcog_cvtm\CVM\dev
* Calibration Calibration outputs directory
* Data_Processed Raw and processed data directory
+ _Documentation Chart and tabular outputs for documentation
+ ATRI Truck GPS data from ATRI
+ Counts Traffic counts
+ Landuse Land use data for SEMCOG regiona and Michigan
+ Models Data extracted from existing models of the region
+ Spatial Spatial data
+ Streetlight Truck GPS data from Streetlight
+ Survey Commercial vehicle survey and other survey data
+ Transearch Transearch commodity flow data
* Estimation Model estimation directory
+ cv_activities CVTM activities model
+ cv_arrival CVTM tour arrival time model
+ cv_duration CVTM stop duration model
+ cv_intermediate CVTM intermediate stop models
+ cv_stops CVTM stop models
+ cv_tours CVTM tour type model
+ cv_vehicle CVTM vehicle choice model
+ ld_gen Long distance generation and distribution models
* User Guide User Guide directory
* Validation Model validation directory
Processing scripts are included in the root of the
semcog_cvtm\CVM\dev directory. The table below describes
the purpose of each of the scripts. Each individual script includes
information about the files used by and produced by each script and the
accompanying code comments explain each step in the script.
Table 105: List of model development scripts
| Filename | Description |
|---|---|
| __Master_Dev.R | Develoment master script |
| app_run_batch.R | Development script to run multiple scenarios from R |
| app_run_components.R | Development script to run the model component by component from R |
| calibration_control.R | Control of the calibration process |
| calibration_functions.R | Suppoprt function for the calibration process |
| calibration_targets_cv_sim.R | Generates targets for CVTM from processed data |
| calibration_targets_firm_sim.R | Generates targets for firm synthesis from processed data |
| calibration_targets_ld_sim.R | Generates targets for long distance from processed data |
| calibration_targets_tt.R | Generates targets for trip tables from processed data |
| data_atri.R | Processes and analyses ATRI OD data, creates tables for estimation |
| data_buffer_skims.R | Creates the buffers skims from the MDOT model, tests code to develop combined skim |
| data_counts.R | Processes and anaylses daily and hourly counts and creates summaries for use in validation |
| data_cvs.R | Processes and analyses CVS data, creates tables for estimation |
| data_cvtm_cv_settings.R | Loads and adjusts CV_Settings.Rdata |
| data_ee_base_growth.R | Creates the base year EE trip table from ATRI data and EE growth inputs |
| data_externals.R | Creates the external stations input file |
| data_facilities.R | Creates the facilities inputs file used in LD estimation and application |
| data_firm_sim.R | Creates the establishments input and the buffer SE data files |
| data_ieei_growth.R | Creates the growth inputs for IEEI flows |
| data_industry_corresp.R | Creates the employment correspondence file used in firm synthesis |
| data_ldm_tod_factors.R | Creates time period factors for the LD model from time of day counts |
| data_spatial.R | Creates the combined TAZ system used in the CV Model |
| data_transearch.R | Initial processing of Transearch data for use in development of LD inputs |
| estimatIon_control.R | Estimation control file to copy estimated models to correct location in application |
| init_dev.R | Development initialization script |
| review_bus_est.R | Initital review of business establishment data |
| review_cvs.R | Initital review of commercial vehicle survey data |
| review_hhs.R | Initital review of household travel survey data |
| review_sed.R | Initital review of socioeconomic data |
To use the scripts, a typical workflow using an integrated development environment for R such as R Studio, ([https://www.rstudio.com/]), would be something like the following steps:
- Run (source) the
init_dev.Rscript to start the CV model application and load the R packages used in the scripts. - Open the relevant script for the task, for example
data_buffer_skims.Rto modify the model input incorporates travel time skims from the MDOT statewide model. - Run and/or modify and run the updated script, saving any new outputs
in an appropriate location in the
semcog_cvtm\CVM\dev\Data_Processeddirectory. - Manually copy or use code to copy the updated output file(s) to the CV model.
Given that the semcog_cvtm\CVM\dev package provides an
archive of how the original versions of the model inputs were created,
it is recommended that when changes are made to the scripts this is done
either at the end of the script (retaining the original work) or in a
clearly named copy of the script.
9.6 Making Changes to the Model System
In addition to creating and evaluating scenarios, the model user might need to make various updates to the CV model as changes are made to the overall model system, particularly as part of more complex scenarios or planning work. Larger and longer term model modifications, where changes would require re-calibration of the base year model, are beyond the scope of this chapter and are discussed in the model documentation.
9.6.1 Updating the TAZ System
The TAZ system is an inherent part of the structure of a travel demand model, and many files reference the TAZs as index in order to join inputs together. Some scenarios may require an area with a more detailed TAZ system than included in the general model, or the TAZ system may be refined without the wish to fully update and re-calibrate the model.
Once the TAZ system has been updated and saved in the E8Plus model, the following steps should be followed to update the CVM. These steps assume that the fields in the TransCAD model’s TAZ files have not been adjusted.
Update the TAZ_System.csv file: This file contains a description of the TAZs used in the model including the definition of aggregations of the TAZs into Counties. The
TAZ_System.csvfile is produced by the data processing script,CVM/dev/data_spatial.Rfrom the TAZ layers for the SEMCOG E8Plus model and the MDOT Statewide Model. The script currently references the base year scenario input fileInput/SED/_zones_shp_and_csv/Zones10_2899.shpand should be updated to read in the new TAZ layer shape file.Update common input files that include the TAZ field:
- Update Establishments.csv: This
file contains the processed business establishment database that is used
as a seed for the firm synthesis model. It is a list of business
establishments by NAICS industrial codes and employment in each TAZ. The
file was processed from an establishment database provided to SEMCOG by
MDOT. The script
CVM/dev/data_firm_sim.Rwas used to process the database into theEstablishments.csvfile format. The TAZ field contained in the establishment database will need to be updated. If that is not possible, and in circumstances where only minor TAZ changes are made, the establishment file can be as is and the scaling process in the firm synthesis model will adjust business establishment employment to match the new TAZ level employment data correctly. - Update ld_ee_trip_table.csv:
This file contains a trip list of origin to destination trips between
external TAZs that represents external to external base year truck
trips. The list was derived from ATRI truck GPS data that was processed
and expanded to truck counts. The file was processed from the final ATRI
truck OD matrix into the
ld_ee_trip_table.csvfile format using the scriptCVM/dev/data_ee_base_growth.R. If updates to the TAZ numbering cause a renumbering of the external stations, then the numbering used in this file will need to be updated using new code added to the script. - Update ld_external_model.csv:
This file contains the external trip allocation model, which is a table
of factors allocating regional commodity flows to and from the SEMCOG
region from the Transeach commodity flow data to scaled external truck
flows derived from the ATRI truck GPS data at each external station. The
ld_external_model.csvfile is produced by the model estimation script,CVM/dev/Estimation/ld_gen/ld_trip_gen_estimation.R. The external factors were developed with ATRI truck GPS data and Transearch commodity flow data. If updates to the TAZ lead to a renumbering of the external stations, then the numbering used in this file will need to be updated using new code added to the script. - Update TravelTimeSkimBuffer.omx:
Preprocessed Open Matrix format skim matrices of zone to zone travel
distances, free flow travel times, and tolls from the MDOT Statewide
model for travel within the buffer region and to and from the SEMCOG
external stations. On import in to the commercial vehicle model, the
matrices are converted into lists in I-J-Value format. These matrices,
which represent internal travel within the buffer region and between the
buffer region SEMCOG external stations, are combined with the time of
skim matrices for the SEMCOG region that are included in the scenario
inputs into a single table called
skims_tod. TheTravelTimeSkimBuffer.omxfile is produced by the data processing script,CVM/dev/data_buffer_skims.Rfrom the MDOT Statewide Model’s base year skim matrices. If updates to the TAZ lead to a renumbering of the external stations, then this data processing script (which uses the TAZ_System.csv file) will need to be rerun to incorporate that change.
- Update scenario input files that include the TAZ field:
- Update the scenario input CVM_Externals.csv: This file contains a complete list of the external stations coded in the model and an indication of whether they are active or not in the current scenario. The file also contains allocation factors to reallocate truck flows from the base year allocation to the scenario allocation to account for changes in the active scenario and the resulting need to reallocate where trucks are likely to enter and exit the region. If updates to the TAZ lead to a renumbering of the external stations, then the numbering used in this file will need to be updated.
- Update the scenario input Facilities.csv: This file lists the
significant distribution and intermodal facilities assumed for this
scenario. The locations are used as special generators and attractors in
the LDM and the CVTM. The file is produced by the data processing
script,
CVM/dev/data_facilities.Rfrom a series of facilities shapefiles included inCVM/dev/Data_Processed/Landuse. If updated shapefiles are available, the file can be regenerated, or alternatively a correspondence between old and new TAZs could be used to allocated faculties into the new TAZ system.
Any remaining model inputs that are passed from the E8Plus model, including TAZ socioeconomic data and travel time skims, will also need to be coded with the new TAZ system and numbering.
9.6.2 Updating External Travel Times and Distances
One static input to the CV model that describes transportation network conditions outside of the SEMCOG region that might be updated separately from an update to inputs for the SEMCOG region itself is the file describing travel times and distance for the buffer region.
TravelTimeSkimBuffer.omx
contains preprocessed Open Matrix format skim matrices of zone to zone
travel distances, free flow travel times, and tolls from the MDOT
Statewide model for travel within the buffer region and to and from the
SEMCOG external stations. Upon importing in to the CV model, the
matrices are converted into lists in I-J-Value format. These matrices,
which represent internal travel within the buffer region and between the
buffer region SEMCOG external stations, are combined with the time of
skim matrices for the SEMCOG region that are included in the scenario
inputs into a single table called skims_tod. The
TravelTimeSkimBuffer.omx file is produced by the data
processing script, CVM/dev/data_buffer_skims.R from the
MDOT Statewide Model’s base year skim matrices.
To update the file with new data from the MDOT Statewide Model:
Extract a set of base year MDOT skims as an OMX file, which should conform to the format of
CVM/dev/Data_Processed/Models/MDOT/data_extracts/Network_Skim_2015.omx.Use the
CVM/dev/data_buffer_skims.Rscript to process the skims into the TravelTimeSkimBuffer.omx skims file.