I need a certain amount of enrichment on IP addresses multiple times per day. While certain tools and controls have built-in enrichment, I often find myself sitting at a shell prompt needing the same or similar info quickly.
So, I decided to write something that will take an IPv4 address as an argument, and return the data I need in an easily-parsable format.
The code is available here.
The code takes an IPv4 address as input, and outputs the following:
- IP: IP address, taken from argv
- GeoCC: country code according to MaxMind GeoIP2 country database
- ShoCC: country code according to Shodan
- rvAS: ASN, per RouteViews MRT/RIB BRP database
- ShoAS: ASN from Shodan
- rvCIDR: CIDR range for the AS, per RouteViews
- ShoORG: AS Organization name, per Shodan
- DNS: Reverse DNS lookup result
- ShoDNS: hostnames for IP, per Shodan
- ShoPorts: Open ports for IP, from Shodan
- ShoTags: Any associated Shodan tags
- ShoVulns: Any known positive or negative vulnerability test results, per Shodan
- up-to-date databases for geoip2 and pyasn (see their docs)
- API key for shodan
You'll need a copy of the RouteViews database. Thankfully, Pyasn comes with utilities that make this a snap.
The following will download the latest copy:
$ pyasn_util_download.py --latest Finding most recent archive in /bgpdata/2017.12/RIBS ... Downloading ftp://archive.routeviews.org//bgpdata/2017.12/RIBS/rib.20171201.1600.bz2 100%, 58KB/s Download complete.
Now, it must be converted into a usable format:
$ pyasn_util_convert.py --single rib.20171201.1600.bz2 ipasn.dat Parsing MRT/RIB archive .. MrtTD2Record (PEER-INDEX-TABLE, collector 2162111334, 62 peers) MRT record 100000 @14s WARNING: repeated prefix '22.214.171.124/24' maps to different origin (62793 vs 22299) WARNING: repeated prefix '126.96.36.199/24' maps to different origin (62793 vs 22299) WARNING: repeated prefix '188.8.131.52/24' maps to different origin (62793 vs 22299) WARNING: repeated prefix '184.108.40.206/24' maps to different origin (62793 vs 22299) MRT record 200000 @28s WARNING: repeated prefix '220.127.116.11/22' maps to different origin (56258 vs 58463) WARNING: repeated prefix '18.104.22.168/23' maps to different origin (56258 vs 58463) WARNING: repeated prefix '22.214.171.124/23' maps to different origin (56258 vs 58463) WARNING: repeated prefix '126.96.36.199/24' maps to different origin (45561 vs 198949) MRT record 300000 @42s WARNING: repeated prefix '188.8.131.52/16' maps to different origin (62793 vs 22299) WARNING: repeated prefix '184.108.40.206/16' maps to different origin (36534 vs 16623) MRT record 400000 @57s WARNING: repeated prefix '220.127.116.11/22' maps to different origin (53115 vs 262482) MRT record 500000 @74s WARNING: repeated prefix '18.104.22.168/24' maps to different origin (36534 vs 16623) WARNING: repeated prefix '22.214.171.124/24' maps to different origin (36534 vs 16623) WARNING: repeated prefix '126.96.36.199/24' maps to different origin (36534 vs 16623) WARNING: repeated prefix '188.8.131.52/24' maps to different origin (36534 vs 16623) WARNING: repeated prefix '184.108.40.206/24' maps to different origin (36534 vs 16623) MRT record 600000 @90s WARNING: repeated prefix '220.127.116.11/24' maps to different origin (55871 vs 17755) MRT record 700000 @104s IPASN database saved (731953 IPV4 + 0 IPV6 prefixes)
Place the resulting
ipasn.dat file wherever you'd like to store it, and ensure that the PATH variable
ipinfo.py code reflects this location.
Next, you'll need the MaxMind GeoLite GeoIP2 Country database.
$ wget http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz --2017-12-01 14:36:40-- http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.tar.gz Connecting to 127.0.0.1:9050... connected. Proxy request sent, awaiting response... 200 OK Length: 1611049 (1.5M) [application/gzip] Saving to: ‘GeoLite2-Country.tar.gz’ GeoLite2-Country.tar.gz 100%[===============================================================================>] 1.54M 2.44MB/s in 0.6s 2017-12-01 14:36:41 (2.44 MB/s) - ‘GeoLite2-Country.tar.gz’ saved [1611049/1611049]
Next, uncompress the archive and move the
GeoLite2-Country.mmdb file into the same location you stored the
ipasn.dat file above.
$ tar xf GeoLite2-Country.tar.gz $ cd GeoLite2-Country_20171107/ $ ls COPYRIGHT.txt GeoLite2-Country.mmdb LICENSE.txt
Now edit ipinfo.py, and add you Shodan API key. You'll need a valid Shodan account. You'll find it in your account settings.
Once you've done this, you should be able to get quick info on any IP:
$ ipinfo 18.104.22.168 IP GeoCC ShoCC rvAS ShoAS rvCIDR ShoORG DNS ShoDNS ShoPorts ShoTags ShoVulns 22.214.171.124|US|US|15169|AS15169|126.96.36.199/24|Google|google-public-dns-a.google.com.|[u'google-public-dns-a.google.com']|||None
I'd originally intended to show two variants of this script: One using the locally-stored databases, and one using Shodan. However, you should never trust a single source for your data. Particularly when it comes to things like geolocation. So, I decided to combine the two approaches into a single script.
Personally, I comment out the line that prints the data header, so I can call it from other scripts when processing a large number of IPs at once, to get a more easily-parsable output.
Be aware that this may use your various Shodan tokens. So keep an eye on them while testing, to ensure you're aware of how the script might affect your Shodan tokens.