$
Learn Linux handbook
en
XP:0
Quest 114Data Munging Boss
+500 XP

Boss Challenge 13: Data Pipeline Normalization

Launch Interactive Terminal for This Quest→No installation required • Runs fully in browser

Concept & Overview

ETL DATA INCIDENT: A legacy mainframe exported /home/learner/messy_data.csv using unconventional formatting: fields are separated by colons and rows end with semicolons (alice:dev:engineering;).

The analytics ingestion engine rejects this format. Transform the file into clean CSV format at /home/learner/clean_users.csv, containing only the user and department (fields 1 and 3) separated by commas (alice,engineering), with semicolons stripped.

Process /home/learner/messy_data.csv and write clean records into /home/learner/clean_users.csv formatted as name,dept without semicolons.

Command Syntax Reference

Command / SyntaxDescription
cat messy_data.csvView raw delimited data
awk / sed / cut / trFilter, translate delimiters, and cut fields
> clean_users.csvWrite normalized data to output file

Verification Objectives

These conditions will be automatically verified when you run commands inside the interactive terminal simulator:

  • ○
    Generate /home/learner/clean_users.csv with clean comma-separated name,dept records

Ready to complete Quest 114?

Test your skills live in the browser terminal with an emulated Linux filesystem, real-time command evaluation, and instant step-by-step hints.