Text Processing (Grep/Sed/Awk)

Stop awk from tripping over weird field delimiters

🧩 The Challenge

Dealing with logs where the delimiter isn’t a simple comma or space is a nightmare. I spent an entire afternoon trying to parse a CSV-like file that used a pipe character inside of some fields, and standard awk just blew up the column count.

💡 The Fix

You can actually tell awk to treat a sequence of characters as a single field separator using a regex, which saves you from writing complex loops or cleaning the file beforehand. It’s a massive time-saver when the data format is inconsistent.

awk -F ' *[|]+ *' '{print $2, $4}' logfile.txt

⚙️ Why It Works

Setting the FS variable to a regular expression lets awk consume the mess surrounding your delimiter, effectively normalizing the input on the fly. It’s much cleaner than trying to strip spaces out of every row manually.

🚀 Pro-Tip: If your input is truly horrific, pipe it through tr first to normalize the separators to a single character before hitting awk.

Linux Tips & Tricks | © ngelinux.com | 8/3/2026