Understanding NYC Traffic Congestion: A Machine Learning Interpretability Approach

Author

Lilly Loghmani

Published

May 5, 2025

1 Introduction

1.1 Project Overview

Traffic congestion affects daily commutes, emergency response, the economy, and the environment in New York City. This report uses SHAP, LIME, and feature importance methods to identify the main factors driving traffic patterns.

1.2 Research Questions

The primary questions this analysis seeks to answer include:

  1. What are the key factors that affect traffic congestion in NYC?
  2. Do different interpretability methods (SHAP, LIME, feature importance) identify consistent factors?
  3. How stable are these interpretations across different model types and data splits?
  4. Could these insights eventually inform policy decisions related to traffic management?

1.3 Data Sources

This analysis integrates multiple data sources:

  • Traffic Volume Data: Traffic count data from NYC Department of Transportation: https://data.cityofnewyork.us/Transportation/Automated-Traffic-Volume-Counts/7ym2-wayt/about_data
  • Weather Data: Historical temperature and precipitation information: https://www.ncei.noaa.gov/access/monitoring/climate-at-a-glance/city/time-series/USW00094728/tavg/12/0/2013-2024?base_prd=true&begbaseyear=2013&endbaseyear=2020
  • Emergency Response Data: Response time metrics as indicators of road network performance: https://data.cityofnewyork.us/Public-Safety/911-Open-Data-Local-Law-119/gpny-cuvw/about_data

1.4 Methodology

The analysis steps are:

  1. Data Preprocessing: Cleaned, integrated, and engineered features.
  2. Exploratory Analysis: Generated visual summaries and statistical checks.
  3. Model Development: Trained machine learning models to predict traffic volume.
  4. Interpretability Analysis: Applied SHAP, LIME, and feature importance methods.
  5. Stability Analysis: Evaluated result consistency across models and data splits.

1.5 Book Structure

The remainder of this book is organized as follows:

  • Chapter 2: Data - Description of data sources, cleaning procedures, and feature engineering
  • Chapter 3: Analysis - Methodological approach, model development, and interpretability techniques
  • Chapter 4: Results - Findings from the interpretability analysis and comparison of methods
  • Chapter 5: Conclusion - Summary of insights and recommendations for future research