Grep IP Addresses From Log File

Last updated on Dec 30, 2019 in Linux

For example, we have a webserver log file which contains access requests including IP addresses of the client. We want to list all unique IP addresses found in that file.

Let’s assume our log file is access.log which contains following data (Apache log):

$ tail access.log
71.37.145.101 - - [29/Dec/2019:19:28:44 +0000] "GET / HTTP/1.1" 400 311 "-" "-"
153.201.201.97 - - [29/Dec/2019:19:41:07 +0000] "GET / HTTP/1.1" 200 3899 "-" "Mozilla/5.0 zgrab/0.x"
37.139.84.78 - - [29/Dec/2019:20:15:13 +0000] "GET / HTTP/1.1" 200 3899 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
71.37.145.101 - - [29/Dec/2019:19:28:44 +0000] "GET / HTTP/1.1" 400 311 "-" "-"

grep File With Regexp to Get IP Addresses

To list all IP addresses, we can use grep and regexp IP addresses from the file

$ grep -oE "/[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}/" access.log
71.37.145.101
153.201.201.97
37.139.84.78
71.37.145.101
  • -o tells grep to echo only matched part, not the whole line
  • E tells grep that our search term is regexp format

The regexp part "/[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}/" is not a perfect pattern, and there are tons of more sophisticated patterns available in internet, but this will do the work in 99.9% of the cases.

Remove Duplicate IP addresses

However, as we can see in the output, there are duplicate IPs found, which may not be the desired outcome. Let’s dedup the output with uniq:

$ grep -oE "/[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}/" access.log | uniq
71.37.145.101
153.201.201.97
37.139.84.78

The uniq command dedups the input lines and echoes only unique lines.