Boss Challenge 13: Data Pipeline Normalization
Concept & Overview
ETL DATA INCIDENT: A legacy mainframe exported /home/learner/messy_data.csv using unconventional formatting: fields are separated by colons and rows end with semicolons (alice:dev:engineering;).
The analytics ingestion engine rejects this format. Transform the file into clean CSV format at /home/learner/clean_users.csv, containing only the user and department (fields 1 and 3) separated by commas (alice,engineering), with semicolons stripped.
/home/learner/messy_data.csv and write clean records into /home/learner/clean_users.csv formatted as name,dept without semicolons.Command Syntax Reference
| Command / Syntax | Description |
|---|---|
cat messy_data.csv | View raw delimited data |
awk / sed / cut / tr | Filter, translate delimiters, and cut fields |
> clean_users.csv | Write normalized data to output file |
Verification Objectives
These conditions will be automatically verified when you run commands inside the interactive terminal simulator:
- ○Generate /home/learner/clean_users.csv with clean comma-separated name,dept records
Ready to complete Quest 114?
Test your skills live in the browser terminal with an emulated Linux filesystem, real-time command evaluation, and instant step-by-step hints.