SHODAN-PYTHON(1)

shodan-python

SHODAN-PYTHON(1)

NAME¶

shodan-python - shodan-python Documentation

This is the official Python wrapper around both the Shodan REST API as well as the experimental Streaming API. And as a bonus it also lets you search for exploits using the Shodan Exploits REST API. If you’re not sure where to start simply go through the “Getting Started” section of the documentation and work your way down through the examples.

For more information about Shodan and how to use the API please visit our official help center at:

https://help.shodan.io

INTRODUCTION¶

Getting Started¶

Installation¶

To get started with the Python library for Shodan, first make sure that you’ve received your API key. Once that’s done, install the library via the cheeseshop using:

$ easy_install shodan

Or if you already have it installed and want to upgrade to the latest version:

$ easy_install -U shodan

It’s always safe to update your library as backwards-compatibility is preserved. Usually a new version of the library simply means there are new methods/ features available.

Connect to the API¶

The first thing we need to do in our code is to initialize the API object:

import shodan
SHODAN_API_KEY = "insert your API key here"
api = shodan.Shodan(SHODAN_API_KEY)

Searching Shodan¶

Now that we have our API object all good to go, we’re ready to perform a search:

# Wrap the request in a try/ except block to catch errors
try:


        # Search Shodan


        results = api.search('apache')


        # Show the results


        print('Results found: {}'.format(results['total']))


        for result in results['matches']:


                print('IP: {}'.format(result['ip_str']))


                print(result['data'])


                print('')
except shodan.APIError as e:


        print('Error: {}'.format(e))

Stepping through the code, we first call the Shodan.search() method on the api object which returns a dictionary of result information. We then print how many results were found in total, and finally loop through the returned matches and print their IP and banner. Each page of search results contains up to 100 results.

There’s a lot more information that gets returned by the function. See below for a shortened example dictionary that Shodan.search() returns:

{


        'total': 8669969,


        'matches': [


                {


                        'data': 'HTTP/1.0 200 OK\r\nDate: Mon, 08 Nov 2010 05:09:59 GMT\r\nSer...',


                        'hostnames': ['pl4t1n.de'],


                        'ip': 3579573318,


                        'ip_str': '89.110.147.239',


                        'os': 'FreeBSD 4.4',


                        'port': 80,


                        'timestamp': '2014-01-15T05:49:56.283713'


                },


                ...


        ]
}

Please visit the REST API documentation for the complete list of properties that the methods can return.

It’s also good practice to wrap all API requests in a try/ except clause, since any error will raise an exception. But for simplicity’s sake, I will leave that part out from now on.

Looking up a host¶

To see what Shodan has available on a specific IP we can use the Shodan.host() function:

# Lookup the host
host = api.host('217.140.75.46')
# Print general info
print("""


        IP: {}


        Organization: {}


        Operating System: {}
""".format(host['ip_str'], host.get('org', 'n/a'), host.get('os', 'n/a')))
# Print all banners
for item in host['data']:


        print("""


                Port: {}


                Banner: {}


        """.format(item['port'], item['data']))

EXAMPLES¶

Basic Shodan Search¶

#!/usr/bin/env python
#
# shodan_ips.py
# Search SHODAN and print a list of IPs matching the query
#
# Author: achillean
import shodan
import sys
# Configuration
API_KEY = "YOUR_API_KEY"
# Input validation
if len(sys.argv) == 1:


        print 'Usage: %s <search query>' % sys.argv[0]


        sys.exit(1)
try:


        # Setup the api


        api = shodan.Shodan(API_KEY)


        # Perform the search


        query = ' '.join(sys.argv[1:])


        result = api.search(query)


        # Loop through the matches and print each IP


        for service in result['matches']:


                print service['ip_str']
except Exception as e:


        print 'Error: %s' % e


        sys.exit(1)

Collecting Summary Information using Facets¶

A powerful ability of the Shodan API is to get summary information on a variety of properties. For example, if you wanted to learn which countries have the most Apache servers then you would use facets. If you wanted to figure out which version of nginx is most popular, you would use facets. Or if you wanted to see what the uptime distribution is for Microsoft-IIS servers then you would use facets.

The following script shows how to use the shodan.Shodan.count() method to search Shodan without returning any results as well as asking the API to return faceted information on the organization, domain, port, ASN and country.

#!/usr/bin/env python
#
# query-summary.py
# Search Shodan and print summary information for the query.
#
# Author: achillean
import shodan
import sys
# Configuration
API_KEY = 'YOUR API KEY'
# The list of properties we want summary information on
FACETS = [


    'org',


    'domain',


    'port',


    'asn',


    # We only care about the top 3 countries, this is how we let Shodan know to return 3 instead of the


    # default 5 for a facet. If you want to see more than 5, you could do ('country', 1000) for example


    # to see the top 1,000 countries for a search query.


    ('country', 3),
]
FACET_TITLES = {


    'org': 'Top 5 Organizations',


    'domain': 'Top 5 Domains',


    'port': 'Top 5 Ports',


    'asn': 'Top 5 Autonomous Systems',


    'country': 'Top 3 Countries',
}
# Input validation
if len(sys.argv) == 1:


    print('Usage: %s <search query>' % sys.argv[0])


    sys.exit(1)
try:


    # Setup the api


    api = shodan.Shodan(API_KEY)


    # Generate a query string out of the command-line arguments


    query = ' '.join(sys.argv[1:])


    # Use the count() method because it doesn't return results and doesn't require a paid API plan


    # And it also runs faster than doing a search().


    result = api.count(query, facets=FACETS)


    print('Shodan Summary Information')


    print('Query: %s' % query)


    print('Total Results: %s\n' % result['total'])


    # Print the summary info from the facets


    for facet in result['facets']:


        print(FACET_TITLES[facet])


        for term in result['facets'][facet]:


            print('%s: %s' % (term['value'], term['count']))


        # Print an empty line between summary info


        print('')
except Exception as e:


    print('Error: %s' % e)


    sys.exit(1)
"""
Sample Output
=============
./query-summary.py apache
Shodan Summary Information
Query: apache
Total Results: 34612043
Top 5 Organizations
Amazon.com: 808061
Ecommerce Corporation: 788704
Verio Web Hosting: 760112
Unified Layer: 627827
GoDaddy.com, LLC: 567004
Top 5 Domains
secureserver.net: 562047
unifiedlayer.com: 494399
t-ipconnect.de: 385792
netart.pl: 194817
wanadoo.fr: 151925
Top 5 Ports
80: 24118703
443: 8330932
8080: 1479050
81: 359025
8443: 231441
Top 5 Autonomous Systems
as32392: 580002
as2914: 465786
as26496: 414998
as48030: 332000
as8560: 255774
Top 3 Countries
US: 13227366
DE: 2900530
JP: 2014506
"""

Access SSL certificates in Real-Time¶

The new Shodan Streaming API provides real-time access to the information that Shodan is gathering at the moment. Using the Streaming API, you get the raw access to potentially all the data that ends up in the Shodan search engine. Note that you can’t search with the Streaming API or perform any other operations that you’re accustomed to with the REST API. This is meant for large-scale consumption of real-time data.

This script only works with people that have a subscription API plan! And by default the Streaming API only returns 1% of the data that Shodan gathers. If you wish to have more access please contact us at support@shodan.io for pricing information.

#!/usr/bin/env python
#
# cert-stream.py
# Stream the SSL certificates that Shodan is collecting at the moment
#
# WARNING: This script only works with people that have a subscription API plan!
# And by default the Streaming API only returns 1% of the data that Shodan gathers.
# If you wish to have more access please contact us at sales@shodan.io for pricing
# information.
#
# Author: achillean
import shodan
import sys
# Configuration
API_KEY = 'YOUR API KEY'
try:


    # Setup the api


    api = shodan.Shodan(API_KEY)


    print('Listening for certs...')


    for banner in api.stream.ports([443, 8443]):


                if 'ssl' in banner:


                        # Print out all the SSL information that Shodan has collected


                        print(banner['ssl'])
except Exception as e:


    print('Error: {}'.format(e))


    sys.exit(1)

GIF Creator¶

Shodan keeps a full history of all the information that has been gathered on an IP address. With the API, you’re able to retrieve that history and we’re going to use that to create a tool that outputs GIFs made of the screenshots that the Shodan crawlers gather.

The below code requires the following Python packages:

arrow
shodan

The arrow package is used to parse the timestamp field of the banner into a Python datetime object.

In addition to the above Python packages, you also need to have the ImageMagick software installed. If you’re working on Ubuntu or another distro using apt you can run the following command:

sudo apt-get install imagemagick

This will provide us with the convert command which is needed to merge several images into an animated GIF.

There are a few key Shodan methods/ parameters that make the script work:

1.: shodan.helpers.iterate_files() to loop through the Shodan data file
2.: history flag on the shodan.Shodan.host() method to get all the banners for an IP that Shodan has collected over the years

#!/usr/bin/env python
# gifcreator.py
#
# Dependencies:
# - arrow
# - shodan
#
# Installation:
# sudo easy_install arrow shodan
# sudo apt-get install imagemagick
#
# Usage:
# 1. Download a json.gz file using the website or the Shodan command-line tool (https://cli.shodan.io).
#    For example:
#        shodan download screenshots.json.gz has_screenshot:true
# 2. Run the tool on the file:
#        python gifcreator.py screenshots.json.gz
import arrow
import os
import shodan
import shodan.helpers as helpers
import sys
# Settings
API_KEY = ''
MIN_SCREENS = 5 # Number of screenshots that Shodan needs to have in order to make a GIF
MAX_SCREENS = 24
if len(sys.argv) != 2:


        print('Usage: {} <shodan-data.json.gz>'.format(sys.argv[0]))


        sys.exit(1)
# GIFs are stored in the local "data" directory
os.mkdir('data')
# We need to connect to the API to lookup the historical host information
api = shodan.Shodan(API_KEY)
# Use the shodan.helpers.iterate_files() method to loop over the Shodan data file
for result in helpers.iterate_files(sys.argv[1]):


        # Get the historic info


        host = api.host(result['ip_str'], history=True)


        # Count how many screenshots this host has


        screenshots = []


        for banner in host['data']:


                # Extract the image from the banner data


                if 'opts' in banner and 'screenshot' in banner['opts']:


                        # Sort the images by the time they were collected so the GIF will loop


                        # based on the local time regardless of which day the banner was taken.


                        timestamp = arrow.get(banner['timestamp']).time()


                        sort_key = timestamp.hour


                        screenshots.append((


                                sort_key,


                                banner['opts']['screenshot']['data']


                        ))


                        # Ignore any further screenshots if we already have MAX_SCREENS number of images


                        if len(screenshots) >= MAX_SCREENS:


                                break


        # Extract the screenshots and turn them into a GIF if we've got the necessary


        # amount of images.


        if len(screenshots) >= MIN_SCREENS:


                for (i, screenshot) in enumerate(sorted(screenshots, key=lambda x: x[0], reverse=True)):


                        open('/tmp/gif-image-{}.jpg'.format(i), 'w').write(screenshot[1].decode('base64'))


                # Create the actual GIF using the  ImageMagick "convert" command


                os.system('convert -layers OptimizePlus -delay 5x10 /tmp/gif-image-*.jpg -loop 0 +dither -colors 256 -depth 8 data/{}.gif'.format(result['ip_str']))


                # Clean up the temporary files


                os.system('rm -f /tmp/gif-image-*.jpg')


                # Show a progress indicator


                print(result['ip_str'])

The full code is also available on GitHub: https://gist.github.com/achillean/963eea552233d9550101

API REFERENCE¶

shodan¶

Exceptions¶

AUTHOR¶

achillean

COPYRIGHT¶

2024, achillean

March 14, 2024

1.0

Source file:	shodan-3.11.1.en.gz (from python311-shodan 1.31.0-1.2)
Source last updated:	2024-03-14T08:46:24Z
Converted to HTML:	2024-04-28T01:29:22Z