These days, many security equipments such as Host based Intrusion Detection Systems (HIDS), Network Intrusion Detection Systems (NIDS) and Anti-Viruses (AV) rely on Indicators Of Compromission (IOC) to struggle against violation of security policy. An IOC relies on attributes such as malicious email addresses, domain names, file hashes and IP addresses to model a compromission. Among all these attributes, the malicious IP address is one of the simpler and therefore often use in the network defense process. For example, firewalls can leverage IOCs to forbid any communications between its assets and malicious IP addresses. Unfortunately, shared hosting make possible that multiple domain names match a single IP address. Consequently, an analyst may produce coarse grained IOCs if he didn’t identify that only a subset of the services hosted by an IP is malicious. Such IOC can be disastrous for a company if vital services get banned because of this false positive.
To limit the risk of false-positive, an analyst must determine precisely if the collected IP address is potentially exploited by one or many websites. Moreover ensure that all other associated web sites are malicious! This type work is tedious and can be reduce the following question: “Is it a shared IP address?”
This article details a solution to confirm that an IP address is used by a shared host.
With the rise of hosting models it becomes interesting be able to identify them. Those hosting models can be divided into two main families: dedicated hosting and shared hosting. Section 2 provides a brief description of the two hosting families (dedicated and shared hosting models) and present their differences. In the following (Section 3), we describe the Passive Domain Name System (PDNS) technique and utility in the identification process. We then leverage an example to understand how to identify hosting solution in Section 4. To conclude, Section 5 details out tools and the results of the identification.
2.Dedicated Hosting and Shared Hosting
This model of hosting service allows user to choose their server resources configuration such as number of CPU, the amount of RAM, the size of disk, etc... When using a dedicated server, a user has full control over his device, he/she can choose it Operating System (OS), which softwares to install and so forth. Into this dedicated model, we can distinguish two types of server: physical server and Virtual Private Server (VPS). Both have one public IP address. Each dedicated server has a different public IP address.
The shared model is not flexible as much as the dedicated model in terms of configuration, OS choice, etc… Even if some host providers tend to propose flexibility for their services at logical raw. A shared server means several applications sharing the same server resources. The shared model fits with small applications, little greedy in resources such as a website with hundred connections per day. A shared server is composed of multiple applications, each application belong to a different user and each one is isolated from other applications. Sharing model is generally the most economical option because of the shared resources.
With this model, all applications on a shared server are visible from outside through the same public IP address. However website applications perfectly fit in this model thanks to HTTP/1.1 which defines a Host field. This field can be translated into a VirtualHost parameter for a web server and therefore allowing several websites to be hosted on a single server. If an IP address appears with plenty of domain names, we can deduce that the server might be a shared one.
3.Passive Domain Name System
PDNS consists of tracking Domain Name System (DNS) queries and DNS answers. Basically, it can be compared as a DNS traffic history. PDNS solution is install on DNS server in order to record DNS requests and replies. Which gives the correspondence between IP address and domain names on a period. We used this traffic history in our identification process. Shared applications are on a single server visible with a single IP address but the shared server have several domain names registered on it’s public IP address. So a shared server is defined by a unique IP address and lots of domain names. A dedicated server is defined by a unique IP address and few domain names. Free PDNS solution such as PowerDNS can do the job. Otherwise there are other solutions such as CIRCL, Passivetotal or Farsight.
Before going further, here is the definition of network block: A network block is defined by a range of successive IP addresses that can be represented in the following format: 192.168.0.0/24 (first IP address of the block then a dash and finally a net-mask, here 24) which is all IP addresses between 192.168.0.0 and 192.168.0.255. A network block is register.
We put ourself in the case of an investigation to understand the identification process. In this investigation two domains looks suspicious: yahoo.com and sekoia.fr. First step of the investigation, we look at the IP addresses associated to these domains by requesting the DNS server with a simple dig command.
dig +short yahoo.com
dig +short sekoia.fr
What next? We want to get more information about these IP addresses such as their history. Now we take a look at the PDNS answers that have been observed on on Yahoo and Sekoia server.
PDNS records on 22.214.171.124,126.96.36.199,188.8.131.52 (Yahoo.com):
PDNS records on 184.108.40.206 (sekoia.fr):
The result of the PDNS record on yahoo IP address shows that only yahoo.com have been associated to this address, so there is one domain register on this IP address. We can deduce that server behind 220.127.116.11 can be a dedicated host.
Now we look at PDNS record of IP address behind sekoia.fr. It stands out that multiple domains (201 exactly) are accessible by the same public IP address. Furthermore if we take a look at whois record of some domain names of the list, each one belongs to different entity (person or company). It looks like a sharing host. Moreover if we take a look at IP Whois lookup of this IP address (18.104.22.168) it looks like the address belong to GANDIFR-NET4 network block, which belongs to Gandi company (a company that proposes web hosting). We completely fit with the case of shared host.
In our classifier, we make the assumption that the smallest network block associated with an IP address is used for the same purpose. So we have flag the network block GANDIFR-NET4 (name of the network block assigned by Gandi itself) as a sharing host zone.
The next section introduce the tools developed by InThreat to automate the identification process.
For InThreat services, we have developed a tool to collect OSINT data from various sources. We use these data flow to feed our identification services especially the IP address. We look at PDNS results of the collected IP address to get the association: IP addresses, domain names. If the number of domain names is above the threshold that determine shared host we flag the network block associated to this IP address as shared host. Afterwards we store the identified network block in our database. The scheme of our identification process is showed below.
On our side, this solution is made to evaluate in nearly real time if domains or IP addresses are attached on a dedicated server or on a shared server. We update regularly our database by refreshing currently known network blocks and by trying to discover and identify new network blocks.
If we take a look at the results (link given below), some network blocks identify as sharing zone might not be so. For instance, Prolexic is a solution for anti DDoS that redirect network traffic to their IP addresses (using BGP), then reverse-proxying the targeted domain is made possible easily and it adds a new (even temporary) domain associated with one of their IP addresses. This leads to a semi-wrong identification because they are not really a shared hosting provider. Using manual identification, we can tag network blocks for their use. CDN (Content Delivery Network) also match with this misinterpretation.
Another limitation could come from unreliable RIR/LIR declaration of network blocks or also insufficiently subdivided network blocks used by some Internet companies.
We use this tool on our daily job and we thought it could be interesting to share a snapshot of the resulting database within our community in order to help on “is it shared or not?”. This snapshot is the extraction of our classifiers made on 2017-03-24.
This information can be valuable into the threat intelligence community so we decided to make it available for free and we also hope that people with tech/admin roles will take a few minutes to fill a comment when they identify a block from their company. A simple (“ok”, “security purpose”, “cluster host” or whatever will be helpful for the community)
The results are available on the: Sekoia Github repository. If you want to improve the results about a network block, feel free to create a pull request for the comment field.