Skip to content

Commit

Permalink
Merge pull request #120 from secynic/dev
Browse files Browse the repository at this point in the history
  • Loading branch information
@secynic
secynic committed Apr 19, 2016
2 parents 1109220 + 9b29d8a commit 8905043
Show file tree
Hide file tree
Showing 8 changed files with 492 additions and 33 deletions.
102 changes: 96 additions & 6 deletions CLI.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,110 @@ environment Scripts directory.
It currently has full ipwhois.py functionality. The others (net.py, rdap.py,
utils.py, whois.py) will be included in a future release.

Input
Usage
=====

TODO
ipwhois_cli.py [-h] --addr IP [--whois] [--hr] [--show_name] [--colorize]
[--timeout TIMEOUT] [--proxy_http "PROXY_HTTP"]
[--proxy_https "PROXY_HTTPS"] [--disallow_permutations]
[--inc_raw] [--retry_count RETRY_COUNT]
[--asn_alts "ASN_ALTS"] [--extra_org_map "ASN_ALTS"]
[--depth COLOR_DEPTH]
[--excluded_entities "EXCLUDED_ENTITIES"] [--bootstrap]
[--rate_limit_timeout RATE_LIMIT_TIMEOUT]
[--get_referral] [--extra_blacklist "EXTRA_BLACKLIST"]
[--ignore_referral_errors] [--field_list "FIELD_LIST"]

Output
======
ipwhois CLI interface

TODO
optional arguments:
-h, --help show this help message and exit
--whois Retrieve whois data via legacy Whois (port 43) instead
of RDAP (default).

Output options:
--hr If set, returns results with human readable key
translations.
--show_name If this and --hr are set, the key name is shown in
parentheses after its short value
--colorize If set, colorizes the output using ANSI. Should work
in most platform consoles.

IPWhois settings:
--timeout TIMEOUT The default timeout for socket connections in seconds.
--proxy_http PROXY_HTTP
The proxy HTTP address passed to request.ProxyHandler.
User auth can be passed like
"http://user:[email protected]:80"
--proxy_https PROXY_HTTPS
The proxy HTTPS address passed to
request.ProxyHandler. User auth can be passed like
"https://user:[email protected]:443"
--disallow_permutations
Disable additional methods if DNS lookups to Cymru
fail. This is the opposite of the ipwhois
allow_permutations, in order to enable
allow_permutations by default in the CLI.

Common settings (RDAP & Legacy Whois):
--inc_raw Include the raw whois results in the output.
--retry_count RETRY_COUNT
The number of times to retry in case socket errors,
timeouts, connection resets, etc. are encountered.
--asn_alts ASN_ALTS
A comma delimited list of additional lookup types to
attempt if the ASN dns lookup fails. Allow
permutations must be enabled. Defaults to all:
"whois,http"
--extra_org_map ASN_ALTS
Dictionary mapping org handles to RIRs. This is for
limited cases where ARIN REST (ASN fallback HTTP
lookup) does not show an RIR as the org handle e.g.,
DNIC (which is now the built in ORG_MAP) e.g.,
{\"DNIC\": \"arin\"}. Valid RIR values are (note the
case-sensitive - this is meant to match the REST
result): 'ARIN', 'RIPE', 'apnic', 'lacnic', 'afrinic'

RDAP settings:
--depth COLOR_DEPTH If not --whois, how many levels deep to run RDAP
queries when additional referenced objects are found.
--excluded_entities EXCLUDED_ENTITIES
If not --whois, a comma delimited list of entity
handles to not perform lookups.
--bootstrap If not --whois, performs lookups via ARIN bootstrap
rather than lookups based on ASN data. ASN lookups are
not performed and no output for any of the asn* fields
is provided.
--rate_limit_timeout RATE_LIMIT_TIMEOUT
If not --whois, the number of seconds to wait before
retrying when a rate limit notice is returned via
rdap+json.

Legacy Whois settings:
--get_referral If --whois, retrieve referral whois information, if
available.
--extra_blacklist EXTRA_BLACKLIST
If --whois, A list of blacklisted whois servers in
addition to the global BLACKLIST.
--ignore_referral_errors
If --whois, ignore and continue when an exception is
encountered on referral whois lookups.
--field_list FIELD_LIST
If --whois, a list of fields to parse: ['name',
'handle', 'description', 'country', 'state', 'city',
'address', 'postal_code', 'emails', 'created',
'updated']

Input (Required):
--addr IP An IPv4 or IPv6 address as a string.

Usage Examples
==============

Basic usage
-----------

TODO
::

ipwhois_cli.py --addr 74.125.225.229 --hr --show_name --colorize --depth 1

29 changes: 29 additions & 0 deletions README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,35 @@ ipwhois needs some outbound firewall ports opened from your host/server.
API
===

IPWhois (main class)
--------------------

ipwhois.IPWhois is the base class for wrapping RDAP and Legacy Whois lookups.
Instantiate this object, then call one of the lookup functions:

`RDAP (HTTP) - IPWhois.lookup_rdap() <#rdap-http>`_
OR
`Legacy Whois - IPWhois.lookup_whois() <#legacy-whois>`_

Input
^^^^^

+--------------------+--------+-----------------------------------------------+
| **Key** |**Type**| **Description** |
+--------------------+--------+-----------------------------------------------+
| address | String | An IPv4 or IPv6 address as a string, integer, |
| | | IPv4Address, or IPv6Address. |
+--------------------+--------+-----------------------------------------------+
| timeout | Int | The default timeout for socket connections |
| | | in seconds. |
+--------------------+--------+-----------------------------------------------+
| proxy_opener | Object | The urllib.request.OpenerDirector request for |
| | | proxy support or None. |
+--------------------+--------+-----------------------------------------------+
| allow_permutations | Bool | Allow net.Net() to use additional methods if |
| | | DNS lookups to Cymru fail. |
+--------------------+--------+-----------------------------------------------+

RDAP (HTTP)
-----------

Expand Down
13 changes: 13 additions & 0 deletions UTILS.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,19 @@ Use Legacy XML File::
>>>> from ipwhois.utils import get_countries
>>>> countries = get_countries(is_legacy_xml=True)

Human Readable Fields
=====================

Human readable translations are available for all result fields (RDAP and
Legacy Whois). Translations are currently limited to the short name (_short),
the name (_name), and the description (_description).

See the ipwhois CLI (ipwhois_cli.py) for an example. I plan to better document
this functionality in a future release.

Import the human readable translation dictionaries::

>>>> from ipwhois.hr import (HR_ASN, HR_RDAP_COMMON, HR_RDAP, HR_WHOIS)

Usage Examples
==============
Expand Down
100 changes: 97 additions & 3 deletions WHOIS.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ IPWhois.lookup() is deprecated as of v0.12.0 and will be removed. Legacy whois
lookups were moved to IPWhois.lookup_whois().

Parsing is currently limited to the keys in the output
`below <#results-dictionary>`_..
`below <#results-dictionary>`_.
This is assuming that those fields are present (for both whois and rwhois).

Some IPs have parent networks listed. The parser attempts to recognize this,
Expand Down Expand Up @@ -73,15 +73,109 @@ Results Dictionary

The output dictionary from IPWhois.lookup_whois().

TODO
+------------------+--------+-------------------------------------------------+
| **Key** |**Type**| **Description** |
+------------------+--------+-------------------------------------------------+
| query | String | The IP address input |
+------------------+--------+-------------------------------------------------+
| asn | String | Globally unique identifier used for routing |
| | | information exchange with Autonomous Systems. |
+------------------+--------+-------------------------------------------------+
| asn_cidr | String | Network routing block assigned to an ASN. |
+------------------+--------+-------------------------------------------------+
| asn_country_code | String | ASN assigned country code in ISO 3166-1 format. |
+------------------+--------+-------------------------------------------------+
| asn_date | String | ASN allocation date in ISO 8601 format. |
+------------------+--------+-------------------------------------------------+
| asn_registry | String | ASN assigned regional internet registry. |
+------------------+--------+-------------------------------------------------+
| nets | List | List of network dictionaries. |
| | | See `Network Dictionary <#network-dictionary>`_.|
+------------------+--------+-------------------------------------------------+
| raw | String | Raw whois results if inc_raw is True. |
+------------------+--------+-------------------------------------------------+
| referral | Dict | Referral whois information if get_referral |
| | | is True and the server isn't blacklisted. See |
| | | `Referral Dictionary <#referral-dictionary>`_. |
+------------------+--------+-------------------------------------------------+
| raw_referral | String | Raw referral whois results if the inc_raw |
| | | parameter is True. |
+------------------+--------+-------------------------------------------------+

Network Dictionary
^^^^^^^^^^^^^^^^^^

The dictionary mapped to the nets key in the
`Results Dictionary <#results-dictionary>`_.

TODO
+-------------+--------+------------------------------------------------------+
| **Key** |**Type**| **Description** |
+-------------+--------+------------------------------------------------------+
| cidr | String | Network routing block an IP address belongs to. |
+-------------+--------+------------------------------------------------------+
| range | String | Network range an IP address belongs to. |
+-------------+--------+------------------------------------------------------+
| name | String | The identifier assigned to the network registration |
| | | for an IP address. |
+-------------+--------+------------------------------------------------------+
| handle | String | Unique identifier for a registered network. |
+-------------+--------+------------------------------------------------------+
| description | String | Description for a registered network. |
+-------------+--------+------------------------------------------------------+
| country | String | Country code registered with the RIR in |
| | | ISO 3166-1 format. |
+-------------+--------+------------------------------------------------------+
| state | String | State for a registered network (if applicable). |
+-------------+--------+------------------------------------------------------+
| city | String | City for a registered network (if applicable). |
+-------------+--------+------------------------------------------------------+
| address | String | The mailing address for a registered network. |
+-------------+--------+------------------------------------------------------+
| postal_code | String | The postal code for a registered network. |
+-------------+--------+------------------------------------------------------+
| emails | String | The email addresses listed for a registered network, |
| | | separated by '\n\' |
+-------------+--------+------------------------------------------------------+
| created | String | Network registration date in ISO 8601 format. |
+-------------+--------+------------------------------------------------------+
| updated | String | Network registration updated date in ISO 8601 format.|
+-------------+--------+------------------------------------------------------+

Referral Dictionary
^^^^^^^^^^^^^^^^^^^

The dictionary mapped to the referral key in the
`Results Dictionary <#results-dictionary>`_.

+-------------+--------+------------------------------------------------------+
| **Key** |**Type**| **Description** |
+-------------+--------+------------------------------------------------------+
| cidr | String | Network routing block an IP address belongs to. |
+-------------+--------+------------------------------------------------------+
| range | String | Network range an IP address belongs to. |
+-------------+--------+------------------------------------------------------+
| name | String | The identifier assigned to the network registration |
| | | for an IP address. |
+-------------+--------+------------------------------------------------------+
| description | String | Description for a registered network. |
+-------------+--------+------------------------------------------------------+
| country | String | Country code registered in ISO 3166-1 format. |
+-------------+--------+------------------------------------------------------+
| state | String | State for a registered network (if applicable). |
+-------------+--------+------------------------------------------------------+
| city | String | City for a registered network (if applicable). |
+-------------+--------+------------------------------------------------------+
| address | String | The mailing address for a registered network. |
+-------------+--------+------------------------------------------------------+
| postal_code | String | The postal code for a registered network. |
+-------------+--------+------------------------------------------------------+
| emails | String | The email addresses listed for a registered network, |
| | | separated by \'\n\'. |
+-------------+--------+------------------------------------------------------+
| created | String | Network registration date in ISO 8601 format. |
+-------------+--------+------------------------------------------------------+
| updated | String | Network registration updated date in ISO 8601 format.|
+-------------+--------+------------------------------------------------------+

Usage Examples
==============
Expand Down
4 changes: 1 addition & 3 deletions ipwhois/ipwhois.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -120,9 +120,7 @@ def lookup_whois(self, inc_raw=False, retry_count=3, get_referral=False,
:raw: Raw whois results if the inc_raw parameter is True. (String)
:referral: Dictionary of referral whois information if get_referral
is True and the server isn't blacklisted. Consists of fields
listed in the ipwhois.whois.RWHOIS dictionary. Additional
referral server informaion is added in the server and port
keys. (Dictionary)
listed in the ipwhois.whois.RWHOIS dictionary.
:raw_referral: Raw referral whois results if the inc_raw parameter
is True. (String)
"""
Expand Down
5 changes: 4 additions & 1 deletion ipwhois/net.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -828,7 +828,10 @@ def lookup_asn(self, retry_count=3, asn_alts=None, extra_org_map=None):
log.debug('ASN WHOIS lookup failed, trying ASN via HTTP')
try:

asn_data = self.get_asn_http(retry_count)
asn_data = self.get_asn_http(
retry_count=retry_count,
extra_org_map=extra_org_map
)

except ASNRegistryError:

Expand Down
Loading

0 comments on commit 8905043

Please sign in to comment.