A Novel Botnet Detection System for P2P Networks

: Botnets remain an active security problem on the Internet and various computer networks. They are continuously developing with regard to protocols, structure and quality of attacks. Many botnet detection programs are currently available, but only few can detect bots in real-time. The sooner bots are detected the lesser damage they can cause. In this paper, a novel botnet detection system, is proposed to detect peer-to-peer bots. The system consists of three-phases filtering, P2P detection and P2P botnet detection phases. For the third phase, P2P network behavior analysis is performed to detect P2P bots. Experimental results showed that the system exhibits high average true positive rate and extremely low average false positive rate during botnet detection.


Introduction
P2P botnets are among the most common types of P2P malwares (Obeidat, 2016). Botnets are composed of many computers with high bandwidth and computing capabilities, which increase with time. The bot master node controls the other bots by initiating various activities such as, email spamming, distributed denial of service attacks, key-logging, Bitcoin mining, click-fraud scamming and password cracking.
Command-and-Control (C&C) communications in P2P botnets are executed through the exchange of files (resources) shared by nodes in a network. For example, the master node of a P2P botnet can create a file of commands and share it with the bots. Subsequently, the bot master periodically shares the file of the C&C with the bots. Notably, C&C communications are similar to the file download traffic for benign nodes. Thus, constructing a detection system capable of distinguishing the difference between benign and malicious nodes based on network traffic analysis is of great importance.
Numerous methods can provide metrics for the inference or differentiation between benign and malicious networks (Strayer et al., 2008;Dillon, 2014). In this paper, a new metric is proposed based on the behaviors of P2P networks, where members exchange data repeatedly over different time intervals. In benign P2P networks, the repetition of uploaded or downloaded data is minimal. In malicious P2P networks, malicious peers share data several times.
A set of characteristics is extracted from the network flow and then used to derive the new metric for the detection phase. These characteristics include timestamp, source and destination IP addresses, protocol and packet size. This metric is based on forming a group for each and every peer in the network, where each group contains all the peers that communicate with this peer either by sending or receiving packets from it. The flow behavior between members of each group are studied and analyzed separately in consecutive and short time intervals according to the following criteria: the rate of change in the size of the group through successive time windows, rate of change in the members forming the group through successive time windows and rate of change in the size of the data transferred between members of the groups through successive time windows. The contribution of this research involves the use of the rate of change in the size (RCS) of the group to distinguish benign peers from malicious ones.
In addition, the proposed system can be characterized by the following features: • The system uses the behavioral features of the network traffic without the use of the payload in individual packets. Thus, it is not affected by encrypted traffic • It doesn't require any training to give accurate results. Thus, it can detect a botnet in real time through an efficient approach that works along with a short detection time window • In real time, the system accurately detects the presence of a bot activity during a significant part of its life during the C&C or attack phase This paper is organized as follows: the related work section to classify study the related methods. The section of the proposed system discusses proposed method and presents its mechanism in detail. Then the experimental results are discussed and illustrated in following section. The conclusion is presented in the following section. Finally, the future work is discussed.

Review of Related Literature
Botnet detection remains an active research topic. Although many methods were suggested in literature, most of them cannot efficiently detect botnets. P2P botnet detection techniques can be broadly classified according to the type of detection method (Obeidat and Bawaneh, 2016). One such method is botnet detection based on flow analysis (Barthakur et al., 2013;Zhang et al., 2014). In this method, network flow between the nodes are studied. However, flow-based approaches have two key limitations. First, most of the flows between nodes belong to benign network processes. Second, the flow features must be calculated at runtime and flow analysis requires a high computational overhead at runtime in the absence of an efficient filter. Meanwhile, detection methods based on resource-sharing behavior monitoring (Rodríguez-Gómez et al., 2014) model the evolution of the number of peers sharing a resource in a P2P network. The limitation of these methods is its requirement to build a normality model of legitimate resources during the training phase. These resources do not necessarily contain all cases. Node-based detection (He et al., 2014;Yin, 2014) examines input and output flow for every node where the approaches aggregate behavioral metrics for each P2P node seen in network communications and use them to distinguish benign P2P hosts from those infected by P2P botnets. The key limitation of this solution is in the use of machine learning, which relies on learning a set of extracted features from real P2P botnets. Conversation-based detection (Dillon, 2014;Fan and Xu, 2014;Narang et al., 2014) does not rely on deep packet inspection or signature-based mechanisms. This approach requires a training phase to detect botnets. Thus, the use of new or unknown P2P applications cannot be detected because they do not belong to known classes.
Botnet detection methods based on flow analysis can be classified into two sets. The first set is based on payload inspection. In this set, the methods are usually resource intensive and slow because they require the analysis of big packet data. New bots also frequently utilize encryption and other methods to conceal communication and packet inspection. The second set is based on flow analysis. In this set, encrypted C&C channels are used.
The proposed method belongs to the second set and the following literature reviews the most recent works closely related to this method. In these studies, P2P botnets are detected by analyzing the behavioral characteristics of the network traffic (Saad et al., 2011;Kheir and Wolley, 2013;Dillon, 2014;He et al., 2014;Almutairi et al., 2016).
PeerDigger (He et al., 2014) is a real-time system capable of detecting stealthy P2P bots. At the end of each time window, the system finds the set of destination IP addresses generated by each detected P2P host in Aggregation Flow (AF). The bot detection process is based on the Reconnection Number (RCN) of the AF. The RCN represents the number of repeated elements in the AF. The Reconnection Ratio (RCR) of Host (H) is defined as the maximum RCN of each AF and is used to determine whether the host is a bot. The problem with this approach is that it uses the maximum value of RCR for each AF at the end of each time window and neglects the relationship between consecutive time windows as a metric to identify the botnet network. In addition, the RCN is calculated by counting the number of destination IP addresses for each P2P host ignoring the received packets from other IP addresses to that host with a probability that this behavior is similar to that of normal networks. Thus, this metric cannot measure the temporal behavior of networks accurately. By contrast, the proposed method determines the P2P botnets by analyzing their network behaviors based on the RCS values between consecutive interval windows.
In 2014, the study (Dillon, 2014) on P2P bot detection within a local network was presented on the basis of the communications with the P2P overlay network of the P2P bots. The work used the NetFlow protocol to gain insight in all traffic within the network. The study analyzed and tested the behavior of Zeus as a P2P malware. Detecting this malware is based on either packet ratio (i.e., the sum of up packets divided by the sum of down packets) or traffic pattern. The experiment had limited access to the external network and with the limited data set, predictions cannot be made for results with real data.
The authors in (Kheir and Wolley, 2013) propose a system that detects active P2P bots through network analysis. Through the use of 1,317 distinct malware samples from eight malware families that communicate via P2P, a malware classifier is developed as part of the botnet detection system. P2P botnet traffic can be distinguished by three characteristics, namely, time, space and flow size. Using these characteristics, the authors used machine learning to differentiate P2P botnet traffic from benign P2P traffic with low FPRs. Their approach uses different characteristics with machine learning for botnet detection and thus greatly differs from our approach.
The Proposed system overcomes the previous limitations by analyzing traffic in real time without studying individual packets. The system analyzes network traffic in each phase, filtering out the unlikely flow along each step, so that the most computationally intensive analysis is done on a dramatically reduced traffic set. First, individual flows are subjected to a series of filters and classifiers to filter out as much traffic as possible. In this process, botnet traffic is cautiously prevented from being eliminated. The flows are then correlated with one another to determine the groups of flows that may be related and those that are parts of the same botnet. Finally, the detector module is examined for the presence of malicious networks based on the measurement of temporal node groups.

The Proposed System
The proposed system monitors the traffic in the network to analyze the flow in real time in order to reveal P2P botnets. The process of revealing malicious networks faces a major problem in the small differences between the behaviors of bots and benign networks. The process undergoes three phases, namely, filtering, P2P detection and botnet detection phases, as shown in Fig. 1.
In the first phase, the packets are filtered according to the transport layer protocol used. TCP and UDP traffic flows are extracted from the overall network traffic. The extraction filters out network flows that are unlikely to be generated by P2P network activities (Perényi et al., 2006). Then the flow extraction phase translates the realtime packet stream into several flow streams. In the grouping phase, the flow stream for every host H is partitioned into several time windows of constant size T and a group is created for each H that contains all of the nodes that have communicated with this host. Using the P2P identification phase, the system detects whether H is involved in a P2P communication by checking the number of nodes in each group, which represents a P2P host when it has enough members. In the botnet detection phase, the system detects P2P botnets by analyzing their network behavior based on the RCS.

Filtering Phase
The goal of this phase is to filter out network flows that are unlikely to be generated by P2P network activities (He et al., 2014). It consists of two stages. The first stage filters out only the TCP and UDP packets discarding data from other protocols because these protocols are mainly used as transport layer protocols to communicate and transfer data (Karagiannis et al., 2004;Perényi et al., 2006). The filter module keeps only TCP and UDP flows,because P2P application use them to exchange data. In addition, in the filter module eliminates flows that follow a successful DNS resolution, considering the data flow of non-P2P applications. Most non-P2P applications typically need to resolve domain names before beginning flows. By contrast, members of P2P applications frequently join and leave the network and often contact one another directly by looking up IP addresses from a routing table without need to DNS requests. P2P members communicate directly by using IP addresses in the overlay network (Aberer and Hauswirth, 2002). Concluding these stages, a large portion of non-P2P network streams can be filtered, while retaining P2P network flows (see algorithm 1).

P2P Detection Phase
In this phase, all P2P types are detected before identifying P2P bots. The stream of packets represents a set of IP packets exchanged between two nodes. It is uniquely identified by the five-tuple set that contains the following information: protocol, source IP address, destination IP address, source port number and destination port number. These packets are generated by various P2P network activities, such as continuation of communication between network members, peer discovery, content request and data transmission.
All the nodes sending packets to a specific node Pj and all nodes receiving packets from that node as a group g(j) are considered. The peer Pj is considered as the master peer in g(j).
For every time windowi, a set of groups are captured and stored in vector g i (j)=<T l , P i , R p , S p >, where T i is the timestamp associated with the packet that belongs to a specific time window, P j denotes the master node, which is the source or the destination for the packets within this time window, R p represents the distinct addresses of the set of source nodes for the packets received by the master node P j and S p is the distinct addresses of the set of destination nodes for the packets sent by P j . g i as a proposed group composed of sender and receiver nodes that communicate with the master node. In this approach, the real-time packet stream generated by every host H can be translated into a set of groups G(H) = {g i (j)}(see Algorithm 1).
P2P botnets communicate with each other without a C&C server. That is, P2P bots have a network behavior that is similar to those of benign P2P applications.
The system detects all P2P hosts by identifying the groups that present P2P network behaviors. To detect the group in real-time, the flow stream for every H is divided into time windows of constant size T according to the timestamp Ti. For every time window, a set of groups is extracted by H. These groups are denoted as G(H) = {g i (j)}. At the end of each time window, the size of each group inG(H) is calculated. For each group g i (j), only distinct members are considered and the size of group j are denoted by δ j =δ i (j) and δ(H) = {δ i (j)}. Groups with δ j smaller than the threshold θ δ are discarded and the remaining groups are considered as P2P groups that may represent a botnet or benign network (see Fig. 2). Thus, for each H, a set of groups G(H) can be extracted from a segment of the flow stream at the end of the time window, that is, G(H) = g i = {g 1 ,…,g m }. An H is considered as a P2P host when it generates at least one group (see lines 3-12 in Algorithm 1 in Fig. 3).

P2P Botnet Detection Phase
The goal of this phase is to identify malicious P2P or benign P2P networks (groups) resulting from the previous phase. Both types of networks share similar network behavior patterns. However, little differences exist between the two types because their goals in using the P2P protocol vary. The members of the Botnet groups must periodically recommunicate with the botmaster. That is, the group is constructed for a reasonably long time. The reasons for this situation are the following: First, P2P bots are likely to experience less peer churn than benign P2P members (Stutzbach and Rejaie, 2006). Second, most P2P bots store a list of known peers for bootstrapping itself into the botnet (Holz et al., 2008;Obeidat, 2016) and determining the number of peers communicating with them.
By contrast, benign P2P systems communicate with the master node, such as file-sharing systems and IPTV platforms, which are extremely dynamic because of the availability of the desired files and their short lifetimes (Aberer and Hauswirth, 2002). These features are expected of P2P bots that build groups containing bots that tend to terminate communication with the same botmasters. The system can provide an early decision at the end of the second time window according to the behavior of P2P traffic. The communication between members and botmasters are repeated. The members and botmasters then produce a positive rate of change in the number of group members. After computing the RCS for every detected P2P host, H is labelled as a P2P bot when the RCS is greater than or equal to a threshold θ RCS .
This method is simple yet successful and does not require additional tools for detection, such as machine learning. The Proposed system provides a real-time bot detection mechanism that works well in high-traffic networks and its efficiency is due to the constant filtering of the data flow through all stages apart from the fact that it does not require data storage for more than two consecutive time windows.

Dataset Collection
The experiment of the proposed system used a dataset of non-P2P traffic, dataset of P2P traffic generated by a variety of popular P2P applications and dataset of traffic from three famous P2P botnets. Table 1 summarizes the details of all datasets with respect to the duration of data capture, number of hosts involved and the size of the data collected.

Dataset of Non-P2P Traffic
Non-P2P traffic dataset collection involves the following processes: monitoring of the traffic crossing the campus network over the period of 1 day and collecting all packets from hosts not running P2P applications. The stream of packets contain a large number of general traffic from a variety of applications, such as web-browsing and email.

Dataset of P2P Traffic
The P2P traffic dataset is collected in a fully controlled network. Three of the common P2P applications are selected, namely, BitTorrent, eMule and Ares. An experimental local network is built in the campus such that it consists of four hosts capturing the network traffic generated by these hosts into the dataset.

Dataset of P2P Botnet Traffic
The dataset of P2P botnet traffic is obtained from a third party (Rahbarinia et al., 2013). This dataset includes a five-hour trace of Waledac, which contains three bots; a 24-hour trace of Zeus, which contains one bot; and a 6.15-hour trace of Neris, which also contains one bot. Table 1 summarizes these traffic datasets.
The three dataset types were merged together into a single dataset to construct a strong experimental dataset. The proposed system is tested using different lengths of time windows and the performance is discussed for every case in the subsequent sections.

Evaluation of P2P Host Detection
Flows from the P2P network can lead to relatively large groups, while unrelated flows can form smaller groups. So, the threshold value has a very important role in detecting P2P hosts. To achieve a high TPR while keeping the FPR low, the value of should be selected carefully. Separately assigning different values to, ranging from 0 to 10. To determine the best value of threshold, the P2P host detection is applied for these values. The results with respect to the TPR and FPR are explained in Fig. 4. In this phase, the hosts within the experimental dataset are classified into two categories, namely, the positive category, which represents a P2P host (either benign or malicious) and the negative category, which represent a non-P2P host. As shown in Fig. 4

Evaluation of P2P Botnet Detection
To evaluate the effectiveness of the differences between benign P2P hosts and bots, different values of θ RCS ranging from -4 to 4 are investigated and the results are shown in Fig. 5. In this phase, the positive category consists of one Neris bot, one Zeus bot and three Waledac bots, whereas the negative category consists of four hosts that only run benign P2P applications. As seen from the curves, the system has a high TPR value when θ RCS is small. However, the FPR values for BitTorrent, Emule and Ares are extremely low at the same θ RCS values. When θ RCS =0, the average TPR is 97.0% for botnet networks and average FPR is 3.0% for benign networks at T of 3 min.
The curves in Fig. 6 represent the accuracy in detecting P2P hosts running benign P2P and botnet applications for different time windows where T= 1 to 5 min. The accuracy of detection is proportional to T reaching 100% for some applications and starts giving the best results for all applications at T=3.

Comparing the New Method and other Similar Works
The experimental results show that the proposed method in this paper has satisfied good results that are better than many other methods found in the same field. The accuracy of the detection methos Fig. 7. A comparison between the new method and related work for the accuracy of detection of P2P bots Figure 7 shows a comparison between the new method and some previous works in terms of bot setection accuracy (Saad et al., 2011;Kheir and Wolley, 2013;Dillon, 2014;He et al., 2014).

Conclusion
In this paper, a novel system to detect P2P bots within a monitored network through traffic analysis is proposed. It first detects all hosts engaged in P2P communication based on the size of the groups which present P2P applications then identifies P2P bots among the detected P2P hosts based on the rate of change in the size of the groups. The strength of the system lies in the following features: The system is not affected by encrypted traffic because it does not rely on payload data. Second, it is simple such that it does not involve the use of complicated statistical features or sophisticated algorithms. Third, the system does not undergo any training phase and thus detect bots in real time. Fourth, it can detect bots during the C&C or attack phase. Finally, its results are superior to those of other similar work.
The evaluation results demonstrated that the proposed system can detect P2P hosts with an average TPR of 99% and average FPR of 1.45%. While, P2P bots can be identified with an average TPR in the range of 62-99% and an average FPR in the range of 18-0.003% for different values of θ RCS .

Future Work
The current approach has several limitations, which we intend to resolve in our future work. Given that the results obtained are based on the availability of existing malicious data, the experiments must be developed to include more types of P2P applications to produce more realistic results. Strengthening the bot detection model based on other factors is possible with respect to the rate of change between members of the groups and RCS of the data transferred between these members. These concerns may be addressed by developing a hybrid botnet detection system that utilizes two factors in addition to the current factor used in the bot detection model.