Text Processing (Grep/Sed/Awk)

Stop wrangling messy CSV output by hand

🧩 The Challenge

Dealing with raw CSV exports from ancient billing software is a nightmare, especially when the data has commas inside the quoted fields that break everything. I’ve wasted hours trying to build complex regexes that just fall apart the second the format changes slightly.

💡 The Fix

Use AWK with a custom field separator to handle the quotes properly without relying on brittle regular expressions. It turns a manual cleanup job into a five-second task.

awk -v FS='","' '{ gsub(/"/, "", $1); gsub(/"/, "", $NF); print $1, $NF }' data.csv

⚙️ Why It Works

Setting the field separator to a specific string pattern allows you to bypass the standard comma delimiter that’s killing your script. It’s much cleaner than trying to escape characters until your eyes bleed.

🚀 Pro-Tip: If your files are strictly formatted, check out the csvkit tools; sometimes there is no shame in using a specialized binary.

Linux Tips & Tricks | © ngelinux.com | 7/16/2026