A linux command to group log messages by time

Given a log file of this form:

[11:29:22.271 INFO  pool-4-thread-1] Received: { Bladibla
[11:29:22.271 INFO  pool-4-thread-1] Received: { Bladibla
[11:29:22.271 INFO  pool-4-thread-1] Received: { Bladibla

To find out how many messages are received per minute: find the log lines containing the word ‘Received’, extract the hour+minute on each of these log lines, discard all duplicates and count the number of occurences.

Command:
grep ‘Received:’ mylogfile.log | cut -c2-18 | awk ‘{print substr($0,0,length()-12)}’ | uniq -c

Result will be something like:

100 08:31
93 08:32
91 08:33
73 08:34

i.e 100 messages logged at 08:31, 93 at 8:32…etc.

This is the beauty of composability: a few simple, well-defined functions, free of side-effects , which can be piped into one another to (easily) achieve (fairly elaborate) results.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s