Skip to content

TMIP-EMAT

Introduction

TMIP-EMAT, or the Travel Model Improvement Program's Exploratory Modeling and Analysis Tool, is a utility that facilitates exploratory modeling using travel demand models. Using TMIP-EMAT, modelers can specify ranges or categories of input parameters and an experiment design efficiently covering the defined ranges is created and run programmatically in TDM23.

Blow are the key terms in EMAT:

  • Core Model: The primary model, in this case, TDM23.
  • Meta-Model: Derived from core model outputs, estimated by EMAT.
  • Experiments: Defined sets of model inputs.
  • Uncertainties: Inputs outside the decision-maker's control.
  • Levers: Inputs that decision-makers can adjust.
  • Metrics: Model (core or meta) outputs.
  • Scope: Definition of input variables (uncertainties and levers).
  • Design: A structured set of experiments within a defined scope.

Installation

  1. Prerequisites: Ensure TDM23 is installed and configured. See the TDM23 Setup page for details.

  2. Download EMAT: Clone the EMAT repository from CTPSSTAFF/tmip-emat on GitHub to the same machine as TDM23.

  3. Install EMAT: Use pip to install EMAT from the cloned repository:

    pip install "path_to_repo"
    • If testing in a Jupyter Notebook, run it in administrator mode and restart the kernel between installations.

  4. Configure EMAT: Update the following configuration files in tdm23/code/utilities/emat_runner to ensure integration with TDM23:

    • emat_config.yaml
    • emat_postprocessor.bat

Running TDM23 with EMAT

Once EMAT is installed and configured, it can be used to run TDM23 efficiently by following these steps.

Example Jupyter notebooks and documentation are included in the code/utilities/emat_runner folder within the TDM23 distribution. Refer to tdm23/code/utilities/emat_runner/README.md for detailed guides.

  1. Set Up the Environment

    • Ensure that EMAT is installed and configured properly.
    • Verify that the required dependencies and TDM23 API are up to date.
    • Make sure that the scope file and database are correctly specified.

  2. Prepare the Experiments

    • Define the scope, including uncertainties, levers, and metrics.
    • Store the scope information in the EMAT database.
    • Create a design of experiments using a structured sampling method (e.g., Latin Hypercube Sampling).
    • Review and validate the experiment design before proceeding.

  3. Run TDM23 with EMAT

    • Set the input parameters in the model using the experiment design.
    • Initiate the model run, ensuring that all required components are functioning correctly.
    • Monitor the model execution to confirm that it completes successfully.

  4. Data Extraction

    • Run post-processing scripts to generate required metrics from tdm23 results.
    • Ensure that output metrics are stored in a structured format.
    • If necessary, extract archived data for additional analysis.

  5. Store and Archive Results

    • Load experiment results into the EMAT database for analysis.
    • Store model outputs and key files in an archive location.
    • Verify that all expected results have been recorded accurately.

Analyzing Model Outputs

EMAT offers multiple ways to analyze data, applicable to both core and meta-model outputs:

  • Scatter Plots: Use EMAT’s visualization tools to generate two-way scatter plots of inputs and metrics.

  • Feature Scoring: Apply feature scoring techniques to assess the relative importance of variables.

  • Comprehensive Visualization: Use EMAT’s built-in visualizer, including 2-Way Interactive Plot and PRIM (Patient Rule Induction Method), to explore relationships between variables interactively.

For additional analysis methods, refer to the EMAT Documentation.