Text Processing (Grep/Sed/Awk)

Stop awk from mangling your CSV exports when commas hide inside fields

🧩 The Challenge

Dealing with CSV files that contain commas inside quoted strings is a nightmare because the default delimiter breaks your columns into total garbage. I once spent an hour trying to parse a database dump only to realize my awk script was splitting the names of companies that had commas in their titles.

💡 The Fix

Use the FPAT variable in gawk to define columns by a regex instead of a simple character delimiter. It lets you treat quoted fields as a single block so you don’t lose your mind trying to escape characters.

awk -v FPAT='([^,]+)|("[^"]+")' '{print $2}' data.csv

⚙️ Why It Works

Setting FPAT forces the engine to match specific patterns as fields rather than just chopping the line at every comma. It’s a complete life-saver when your input data is generated by sloppy spreadsheet software.

🚀 Pro-Tip: If your CSV uses different quoting characters, just swap the pattern in FPAT to match what you actually see in the file.

Linux Tips & Tricks | © ngelinux.com | 8/20/2026