Threat Recon API Version 1.0
Threat Recon™ is a new threat-intelligence API developed by Wapack Labs and powered by GO. The Threat Recon™ API lets you search both open-sourced and proprietary intelligence to provide a more complete picture of the threat landscape.
Basics and Getting Started
Getting started is easy! First sign-up to receive your free API key and then read the Usage section for examples. The Threat Recon™ team hosts a github repository with example scripts so you can start using immediately. The examples provided are Python however any programming language that can parse JSON will work with the API.
Our API focuses exclusively on relationships between knowns and unknowns. This is because if you have a single indicator that you know is bad, then you may be missing a large part of the picture without examining its relationships.
Conversely you may have a seemingly benign indicator however if you investigated its relationships, then you might eventually find out that it is linked to a known bad guy. This research might consume valuable time; however, Threat Recon™ identifies these relationships instantly and with high confidence.
The Threat Recon™ analytic engine leverages human analysis along with machine generated metadata such as Whois records, historical and current DNS information, tagging, as well as a proprietary confidence algorithm in order to provide as much context as possible about a single indicator including…
- Indicator Reference – Where has this indicator been reported?
- Source – Is information available in open source?
- Kill Chain – In what phase of the attack has the indicator been observed?
- First and Last Seen – When was the indicator observed?
- Attribution – Who or what can the indicator be attributed to?
- Process Type – How was this indicator derived by Threat Recon™?
- Was it directly added by our staff? [Direct]
- Was it automatically derived from a host lookup? [Derived_pDNS]
- Was identified as sharing infrastructure with a known bad? [Derived_pDNS_net]
- Was it identified as a subdomain to a known bad domain? [Derived_subdomain]
- Was it derived from a Whois Record? [Derived_whois]
- RRNAME – If the indicator is an IP, what is(or was) the hosted domain(s)?
- RDATA – If the indicator is a networked domain, what is the hosting IP(s)?
- Country – What is the geo location of an IP indicator?
- Tags – Has the indicator been identified as any of the following?
- dynamic domain?
- fast flux domain?
- DGA (Domain Generation Algorithm) Domain?
- non-routable IP?
- Is the domain in the Alexa top 1 million?
- Comment – Is there any other relevant intelligence of value?
- An indicator with a Root Node value has been derived and inherits certain information (See API Usage section)
- Root Node – Defines the Relationship!
- Confidence – For “Direct” indicators, the confidence number defines the estimated level of badness. For Derived indicators, the confidence defines the level of connectivity with the Root Node or in other words, it’s the confidence of the relationship.
High confidence indicators of compromise (IOCs) are provided by trusted sources and then additional information and context is derived via automation. Both types of indicators are accessible via the API.
“Direct” aka RootNodes - are created by humans and are accordingly assigned a higher confidence. Every ‘Direct’ indicator is fed into our engine in order to derive additional information. The resulting ‘Derived’ indicators are programmatically generated based on certain rules. As such, derived indicators are lower confidence however they need not be discounted. The occasional false positive is always better than the occasional false negative…
The Threat Recon™ API currently supports searches on the Indicator and RootNode fields. Both fields are automatically searched for every API query so as to maximize usage. This allows a user receive any immediate indicator relationships if they exist in the database without expending an additional query (these can add up!). Let’s try an example search on indicator serval.essanavy.com using the following python code which is available on our github repository (simple_search.py):
The following is color-coded output from the above search. Three derived records are shown with serval.essanavy.com listed in the RootNode field – this means they were derived from serval.essanavy.com. Every derived record inherits the following metadata from the RootNode: Reference, Source, KillChain, Attribution, and Comment. Each derived indicator record will also contain metadata specific to itself: Type, FirstSeen, LastSeen, ProcessType, Rrname, Rdata, Country, TAGS, Confidence, and ID. Derived Confidence ratings are computed based on how connected the derived indicator is with the RootNode.
The records shown in the raw output are the textual representation of several relationships. These can easily be rendered visually with serval.essanavy.com as the root node.
For cleaner – more human readable results, one may use the threat_recon_request.py script. This provides the available metadata if the search is a Direct indicator and then simply lists all derived relationships. These are devoid of inherited or secondary metadata for a less noisy output:
Some requests have the potential to provide a lot of results, for example a whois search on a name server, a CIDR block, or a popular IP. For these requests, threat_recon_to_CSV.py may come in handy. This will write all results and metadata to a CSV for later inspection and/or processing. This script may also be easily modified for writing results to a database.
Threat Recon™ is visualization-friendly. If you want to experiment, try out the visualization scripts on our public github. For Maltego users, custom transforms are available here.
Last but not least, we provided a whois record parser, threat_recon_whois.py that compares whois record components against Threat Recon’s whois database and then writes to a CSV. This is useful for performing Whois lookups and then automatically finding potential correlations to a known-bad logged by Threat Recon. Example output:
If you have any questions or recommendations on how to make Threat Recon™ better, we want to hear them. Please send to firstname.lastname@example.org