NYC Yellow Taxi Tripdata Analytics | Microsoft Azure Data Engineering Project

Dataset Overview

Dataset Description

NYC Yellow trip records contain the following fields:

• Pickup and drop‑off dates and times
• Pickup and drop‑off locations
• Trip distance
• Fare details
• Rate type
• Payment type
• Driver‑reported passenger count

Dataset Source

• Original source: https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page
• Data dictionary: https://www.nyc.gov/assets/tlc/downloads/pdf/data_dictionary_trip_records_yellow.pdf

Dataset Usage

This dataset underpins a comprehensive Azure data engineering project aimed at ingesting, transforming, analyzing, and visualizing New York City taxi trip data.

Project Architecture

Azure Services Used

• Azure Data Factory (ADF)
• Azure Data Lake Storage Gen2 (ADLS Gen2)
• Azure Databricks
• Azure Synapse Analytics
• Key Vault
• Azure Active Directory
• Power BI

Languages Used

• Programming languages: Python, PySpark
• Scripting language: SQL

Data Model

Power BI Dashboard