IP Networking: Ping Output to Comma Separated Values (CSV) at Runtime
Posted on April 19, 2021 — 5 Minutes Read
At times one in the field of IP networking will inevitably come across projects that require manipulating and analysing output from network performance measurement tools such as ping, traceroute and iPerf to name a few. Some of these tools were built with readability as well as computability in mind and as such were designed by default to print output friendly to the human eyes, with an option to convert the output at runtime to Comma Separated Values (CSV), should it need to be redirected to a file or a downstream function for manipulation and analysis. Some however were not. Prime among those is ping which, considering it is the go-to tool for checking network Round Trip time (RTT) and as such one of the most essential utilities in networking, is all the more surprising. That said, converting at runtime the standard human-readable ping output to a more processable CSV is simply a matter of coding a wrapper Bash script to filter and manipulate the result.
The rest of the code is shared on Github for reference and further development.
Writing a Bash script
There are many tools or scripting languages that one can use to automate tasks on a server and Bash is one of them. It is one of the most widely-used scripting languages and is the default login shell for Linux. When you connect to your Linux server by Secure Shell (SSH, RFC 4523), you are already using Bash.
Use the nano text editor to open a new file and start writing the script.
$ nano
First line to add the script shall be to inform the executer that this is a Bash script and is to be interpreted with Bash. This line is elegantly called a shebang.
#!/bin/bash
Thereafter goes the script.
#!/bin/bash
trap echo 0
ping $* | while read line; do
[[ "$line" =~ ^PING ]] && continue
[[ ! "$line" =~ "bytes from" ]] && continue
dt=$(date +'%Y%m%d%H%M%S')
et=$(date +%s)
addr=${line##*bytes from }
addr=${addr%%:*}
rtt=${line##*time=}
rtt=${rtt%% *}
echo -n "$dt,$et,$addr,$rtt"
echo
done
Control + o to save the script. Enter the a name for the new file e.g. pingc, with a full path if it is to be saved in a location other than the current working directory e.g. /app/pingc/pingc.sh. Press enter to save.
Control + x to exit nano.
Ping with a Loop
At its core this script executes the ping command with the rest of the argument supplied to the script. The rest of the output is then pipelined to a while loop instructed to read it line by line and perform the filtering and manipulation as appropriate.
ping $* | while read line; do
...
done
Five operations are required in total and the first of which is to filter the header and footer lines from the standard ping output that are not RTT results from the ICMP packets.
ping $* | while read line; do
[[ "$line" =~ ^PING ]] && continue
[[ ! "$line" =~ "bytes from" ]] && continue
...
done
Date and Time in Two Perspectives
Following the filtering are two commands that construct the date and time in two different formats. Each of which has its own merit and both will prove handy comes data manipulation and analysis. First of the two is the date and time in the numerical format of YYYYMMDDHHMMSS e.g. 20210419024510 that strikes a balance between comprehensibility and operability, and the second is the number of seconds since the epoch that is since 1970-01-01 00:00:00 UTC which will be a rather handy numerical counter in addition to the embedded date and time value in relation to the epoch.
ping $* | while read line; do
...
dt=$(date +'%Y%m%d%H%M%S')
et=$(date +%s)
...
done
Extracting Destination IP and RTT
With the date and time set, all is left is to output the two most important pieces of information from the ping namely the destination IP address and its RTT. Both of which are printed albeit as unstructured text by the standard ping output, and the Bash script will simply need to extract them accordingly.
ping $* | while read line; do
...
addr=${line##*bytes from }
addr=${addr%%:*}
rtt=${line##*time=}
rtt=${rtt%% *}
...
done
Echoing the Outputs
At the end of the while loop, before moving on and reading the next line, the rest of the above will be printed out with the tried and trusted echo command followed by an empty echo to ensure the next print will be on a new line.
ping $* | while read line; do
...
echo -n "$dt,$et,$addr,$rtt"
echo
done
Testing and Troubleshooting
Now the script can be tested.
First thing is to do is to provide the script with execute permission.
chmod +x *full-path-to-script*
$ chmod +x /app/pingc/pingc.sh
To execute the script, simply call it in your Bash shell with a destination IP as required by the ping command e.g. the Cloudflare DNS server at 1.1.1.1 and other optional argument(s) e.g. -c5 to instruct ping to stop after 5 counts.
$ /app/pingc/pingc.sh -c5 1.1.1.1
If all goes well, you should see an output similar to the below
20210419024510,1618800310,1.1.1.1,3.62
20210419024511,1618800311,1.1.1.1,3.36
20210419024512,1618800312,1.1.1.1,2.84
20210419024513,1618800313,1.1.1.1,3.38
20210419024514,1618800314,1.1.1.1,3.38
Below are some of the common errors.
Bash: permission denied
-bash: full-path-to-script: Permission denied
Please make sure that your script has execute permission. To add execute permission.
chmod +x *full-path-to-script*
$ chmod +x /app/pingc/pingc.sh
ping: command not found
full-path-to-script: line n: ping: command not found
Please make sure ping is installed on your server.
$ apt install iputils-ping
Thoughts
This is just one example of using a simple Bash script to manipulate command output. There are plenty of other fascinating tasks that can be done with Bash. Explore and be amazed.