Data Engineering
Marketplace Sales Data Engineering Pipeline.
Built an end-to-end data engineering pipeline for marketplace sales data using Pentaho Data Integration, MySQL, and Power BI. The project covers data ingestion, transformation, data quality validation, dimensional modeling, ETL orchestration, and business intelligence dashboard development.
Project Information
Project Details
Overview
This project builds an end-to-end data engineering pipeline for marketplace sales data, transforming raw CSV data into a structured data warehouse and interactive Power BI dashboard.
Process
The workflow covers data ingestion, data cleaning and transformation using Pentaho Data Integration, data quality validation, dimensional modeling with a star schema, ETL orchestration, and data consumption through Power BI.
Insights
Raw marketplace data was ingested into a staging layer using a full-load strategy.
Data quality was improved by handling invalid values, NULL values, data types, and duplicate records.
The data warehouse was structured using one fact table and three dimension tables.
The warehouse was connected to Power BI to analyze GMV, sold units, views, and conversion rate.
Next
All Work ↗