Email intent dataset pipeline
An automated pipeline that curated 6.9k+ emails with PII filters and LLM-generated intent labels for downstream classification.
Details
- Role
- Organisation
- Dates
- Stack
Overview
During my internship at American International Assurance, I built an automated data extraction and preprocessing pipeline with Python, Pandas, SQL and Azure Databricks.
The pipeline curated a dataset of 6.9k+ emails. It filtered out personally identifiable information and attached LLM-generated labels, so the dataset could be used downstream for email intent classification.