An Adaptive Assessment and Prediction Mechanism in Network Security Situation Awareness

: Network intrusion attempts have reached an alarming level. Cisco’s 2014 Security Report indicated that 50,000 network intrusions were detected and 80 million suspicious web requests were blocked daily. Hence, Intrusion Prevention System (IPS) had been chosen as a defence mechanism in many organizations. However, the University of South Wales reported that seven big-brand IPS had failed to detect and block 34-49% of attacks in web-based applications. The accuracy of IPS can be improved if the network situation is also considered in preventing intrusion attempts. Knowledge about current and incoming network security situation is required before any precaution can be taken. Situation assessment and prediction are two main phases of Network Security Situation Awareness. This paper presents a network security situation assessment and prediction mechanism that proposes an Entropy-based situation assessment scheme to assess current network security status with the aid of the Analytical Hierarchy Process and the introduction of an adaptive situation prediction mechanism based on Grey Verhulst and Kalman Filtering to predict the incoming security situation. The effectiveness of the mechanism is evaluated using National Advanced IPv6 Center (NAv6) 2015 dataset. The findings demonstrated that Entropy-based Network Security Situation Assessment (E-NESSAS) assessed more comprehensively network security situation by using Entropy concept. Meanwhile, Adaptive Grey Verhulst-Kalman Network Security Situation Prediction (AGVK-NESSIP) provided high predictive accuracy with accuracy of 82.77%. The results clearly revealed that the proposed mechanism could assess current security situation systematically by E-NESSAS and was able to predict the situation more accurately by AGVK-NESSIP regardless of the time intervals and behaviour of the data sequence.


Introduction
The Internet infiltrates our lives with offering convenient services and information sharing. It raises the number of Internet users worldwide to reach 3.17 billion which is almost 40% of the world population in 2015 (ITU, 2015). Unfortunately, the immense popularity of the Internet and prevalent use of online applications has made the Internet a breeding ground for malware and cyber criminals. In 2014, Symantec had encountered a 40% increase in phishing attacks compared to previous year, 2013 (Symantec, 2015). Meanwhile, Arbor Network had also revealed that Distributed Denial of Service (DDoS) attack was the most frequently observed threat in an enterprise with an average of 21 attacks in a month. The situation became worse when more than 33% of organizations had their intrusion prevention system devices experience failure during the attack (Anstee, 2015). Based on a survey conducted on United Kingdom (UK)'s businesses discovered that 90% and 74% of large and small organizations respectively had a security breach in the year 2014 and it caused losses of £1.46 million-£3.14 million average in the year (PricewaterhouseCoopers, 2015). This phenomenon brings serious challenges and problems to network security.
Due to the rising number of threats, detection alone is no longer able to provide an organization a reliable network. Prevention before an incident occurs should be in place. As preventing an incident requires careful analysis and planning, network security communities are constantly on the alert to monitor the current and incoming security situation in their networks before any precautions could be taken. Endsley (1988a) has introduced a concept of situation awareness to the world which covers the capabilities of security situation assessment and prediction in his 3hierarchical framework.

Concept of Network Security Situation Awareness
Situation awareness is defined as the observation of changing critical factors in a complex global network within a time and space interval, the understanding of those factors mean according to the operator's goals and the projection of their status in next interval (Endsley, 1988b). It consists of three stages which are event detection, current situation assessment and future situation prediction (Endsley, 1995). Event Detection is a basic process which mainly to identify the abnormal and malicious activity in the network and translates them into logical format. Current Situation Assessment is a process to evaluate the security situation of the entire network by using the information obtained from the detected alerts in the event detection then Future Situation Prediction uses the current and historical network security situation status to forecast the future network security tendency according to the current and historical network security situation status. In 1999, these stages have been adapted in cyberspace called as Network Security Situation Awareness (NSSA) (Bass and Gruber, 1999).
The governments, enterprises and other stakeholders are seeking appropriate strategies with this kind of capability to manage their information and control system. For instances, the Australia Government established a Cyber Security Operations Centre (CSOC) within the Department of Defense to provide a 24/7 cyber situation awareness capability to facilitate operational responses to cyber security events of national importance (AAGD, 2009). In United States (US), President Barack Obama sealed a strategy plan to share situational awareness of network vulnerabilities and risks among the public and private sector networks and to work with other countries (between government and industries) in order to expand the international network in building a greater global situation awareness and incident response (WH, 2011). The United Kingdom (UK) Cyber Security Strategy clearly stated that they will continue to improve their detection and analysis of sophisticated cyber threats especially in the UK's critical national infrastructure as well as to pool knowledge and situational awareness when appropriate with their partners across all businesses to build a genuinely national response (CO, 2011). One of the main responsibilities of The German National Cyber Response Centre is to alert the crisis management staff whenever the cyber security situation reaches the level of an imminent or already occurred crisis (FMI, 2011). In Malaysia, the National Cyber Security Policy addressed the need to develop effective cyber security incident reporting mechanisms capable of disseminating vulnerability advisories and threat warnings in a timely manner so as to strengthen the National Computer Emergency Response Teams (CERTs) in monitoring the situation of critical national information infrastructure (MOSTI, 2012). From the effort of aforementioned countries in their strategic planning, it obviously reflected a concerted concern that NSSA is very much in demand at the top level of cyber security strategic plan.

Proposed Mechanism
The proposed mechanism called Network Security Situation Assessment and Prediction (NESSAP) mechanism which aimed to assess the entire network security situation by considering the tangible and intangible criteria and predict the incoming network security situation on next time-interval by using historical and current situation. In NESSAP, it comprises four main modules: (1) Data Preparation, (2) Data Normalization, (3) Entropy-based Network Security Situation Assessment (E-NESSAS) and (4) Adaptive Grey Verhulst-Kalman Network Security Situation Prediction (AGVK-NESSIP). Each module contains several components. The suspicious alerts are collected from Intrusion Detection Systems, as example, Snort in this research and flow into NESSAP mechanism. Figure 1 depicts the overall modules and flows in proposed mechanism.

Data Preparation Module
Data Preparation Module is the first module in NESSAP mechanism. The purpose of this module is to prepare appropriate data in proper format for the mechanism. The generated alerts from different alert detection engines may have different format. It is difficult for the mechanism to identify all possible formats for the alerts. Therefore, before the generated alerts have been used in the mechanism, they are required to be standardized in a specific format. Alert Procurement is the first component in Data Preparation Module. It used to collect the alerts from multiple alert detection engines and save them into a text file. Meanwhile, Alert Formatting is a component to extract the features in each alert from the text file (output of Alert Procurement) and save them in a Comma Separated Value (CSV) type file. The features are loaded in a table which consists of number of rows and columns. There is a column called Alert_Score has been added into the table. The value of it is based on the priority value which able to represent the severity of this particular alert.

Data Normalization Module
Data Normalization module is responsible to categorize the alerts in specific groups based on their features and eliminate the redundant alerts which might lengthen the processing time. In this module, Alert Fusion is the component to gather the alerts in several groups based on the timestamp and destination address. In this context, user is required to define the time-interval which can be in minute basis to segment the alerts from a bunch of alerts for this particular time period. After that, the alerts within this timeframe will be classified based on its destination address. The alerts which have same destination address will be grouped under the same cluster. The fused alerts will be flowed to the next component, Alert Filtering. The process flow of Alert Fusion is depicted in Fig. 2.
After receiving the fused alerts from previous component, Alert Filtering is used to filter out the redundant alerts in the group. In this stage, fields such as IP destination, rules and priority in each alert are compared regarding its similarity. The similar alerts which have same features except the time are grouped. A counter is responsible to calculate the number of alerts in each group before these redundant alerts to be eliminated. The process flow of this component is demonstrated in Fig. 3. The alert types with its frequency will be fed into next module, Network Security Situation Assessment Module.   Entropy-based Network Security Situation Assessment module is one of the core methodologies of the proposed mechanism. Its main function is to assess the current security situation in entire network and provide this useful information to the security administrator. Instead of assessing the security situation of network in overall, threat of every single asset has been evaluated in the first step. Asset Threat Assessment is responsible to evaluate the threat of every asset in the network. It is a prerequisite process for next component to assess the whole network security situation. In a network, different asset has different function. Thus, it brings them a different threat value based on their importance in a particular network. As example, a printer may have different threat value as a server in the same attack. With the aid of Analytic Hierarchy Process, a weight with range from 0 to 1.0 has been determined and assigned to the assets after considering their implication of particular attack which relatively measured as its loss on confidentiality, integrity, availability, damage and response cost as well as likelihood of occurrence. This weight assignment represents the importance value of particular asset in the network. It can be done when any new network asset has been added into the network. Asset Threat Assessment has four main process as described in Fig. 4. The well-format and non-redundant alert types from previous module are ready to be used for assessment. Based on the IP destination address in an alert, the importance value of asset involved has been identified by referring to the list of network assets and their weight.
where, AS k is k th type of Alert Score (0.0≤ AS n ≤ 1.0) and k is the number of Alert Type. After that, a table called Table of  The risk state (range from 0 to 1.0) has been divided into 10 segments which symbolize different risk level of the alert type as depicted in Table 1.
Once the risk level of alert has been pinpointed, Risk Index (RI) of the alert also can be determined by using Table of Risk Index as presented in Table 2. The range of RI is from 0.1 to 1.0.
High (H) 0.6 ≤ RS < 0.7 Higher (HR) 0.7 ≤ RS < 0.8 Much Higher (MH) 0.8 ≤ RS < 0.9 Highest (HT) 0.9 ≤ RS < 1.0 Extreme (E) After obtaining the risk index of every type of alerts, Asset Threat, AT for each is also calculated by using Equation 2. Adapted the concept of grade calculation for security situational index (Xiaorong et al., 2012), in this research, different severity of alert is corrected by 10 k AS where aggravate the seriousness of an attack that causes severe consequence on the asset: where, AT k is k th type of Asset Threat. Security Situation Assessment acts as a center of assessment module to consider the situation of all the assets in the network in order to determine the security situation for the whole network. With the Asset Threat, AT in all detected alerts from previous component, Security Situation Assessment applied the concept of information entropy to measure the uncertainty degree of the network assets. As the nature of network alerts from environments that are characterized as unknown, high entropy, nonstationary and noisy (Tan, 2013), the concept of information entropy is useful in measurement of the uncertainty in a random variable. In general, the more uncertain or random the event is, the more information it will contain. In this research, the greater the entropy, the more serious the security situation of this asset is. Figure 5 demonstrates the processes in Security Situation Assessment.
By using the Asset Threat, AT, Information Entropy, E of each asset is calculated as Equation 3 and 4. For = 1: and for >1: where: Where: n = n th asset E n = Entropy of n th asset AT k = Asset Threat of k th alert type fk = Frequency of k th alert type in particular asset F = Total of frequency of all alert types in particular asset After calculating the Information Entropy of all the assets, the value of Network Security Situation Assessment (NESSAS) for each time interval can be evaluated. This value is the sum of security situation assessment for all the assets, which can be determined by multiplying Information Entropy with the importance value of asset in the network (AW). The formula to calculate the value of NESSAS is as follows: where: Where: T i = i th time-interval AW n = Importance value of ℎ asset ASSE n = n th Asset Security Situation Entropy Once the network security situation assessment has been computed, a sequence of situational values from E-NESSAS module is built in order based on the time interval.

Adaptive Grey Verhulst-Kalman Network Security Situation Prediction Module
Adaptive Grey Verhulst-Kalman Network Security Situation Prediction module is also a core methodology in the proposed mechanism. Its purpose is to predict the incoming network security situation and then taking some preemptive actions before an incident occurs. The module is constructed by two main processes in different components which are Preliminary Network Security Situation Prediction and Residual Prediction. To increase the precision of prediction, the value of forecasted network security situation in this module is a combination of preliminary situation prediction and its residual prediction in a particular time interval.
GM(1,1) and Grey Verhulst are theories to deal with indeterminate and incomplete system with their superiority in small sample. Unfortunately, they have similar problem in overshoots which caused by the nonmonotonic time series data (Yao et al., 2013). The background value, z (1) (t) of both theories can be written as Equation 8: In fact, the background value is a crucial factor that influences the adoption of Grey theories and their forecasting result. The value of developing coefficient, and the precision of the model will be affected by different background values (Yeh et al., 2009). In Grey Verhulst, the Grey differential equation is written as Equation 11: where background value, z (1) (t) can be obtained through Equation 8. From the Equation 9, background value has direct influences on the precision of the Grey Verhulst. Its value is determined by α which range 0≤α≤1. In traditional Grey theories, α is always set as 0.5 to equalize the importance of each data (Yao et al., 2003;Wen et al., 2000;El-Fouly et al., 2007). In this context, the ignorance of data characteristic has produced more prediction errors (Yeh et al., 2009;Lin et al., 2009).
In order to improve the performance of Grey theories especially in Grey Verhulst, the error term resulted from the background value calculation needs to be eliminated. In other words, finding a suitable background value for the model is an essential subject to improve the prediction accuracy. The most suitable background value should be located in between x (1) (t−1) and x (1) (t) (Li and Lin, 2013). Figure 6 illustrates the possible area of background value located. Due to the developing coefficient will direct affect the background value, thus the newer data should be emphasized by assigning a larger value of a (Yeh et al., 2009). In fact, setting the value of a is a process to search the optimal solutions within the value space. The time series dataset should be regarded as several different populations (Lin et al., 2010). Hence, the value of a should be adaptable at each timescale with different adjustable background values as Fig. 7.
The possible error which might degrade the precision of Grey Verhulst can be identified prior to its elimination. The whitening equation of Grey Verhulst model is: By integrating both side of Equation 10: where: Thus, Equation 11 can be written as: By comparing the Equation 9 and 13, the background value can be determined as Equation 14: From the Equation 14, the error is exist if there has an inequality equation as follows: where: In other words, to eliminate the error, the background value must be equal to the integration of x (1) (t) from two consecutive time interval t-1 to t: Equation 17 is the area underneath a curve which can be represented as in Fig. 8. Since the curve function is unknown, the background value, z (1) (t) is calculated by the combination methods of Trapezoidal Rule and Simpson's Rule. These rules are used to determine the area underneath a graph. Trapezoidal rule is based on approximating the integrand by a first order polynomial and then integrating the polynomial in the interval of integration while Simpson's rule is an extension of Trapezoidal rule where the integrand is approximated by a second order polynomial. Simpson's Rule is able to approximate more accurately the area under a curve but it limited to even number of region divisions. Therefore, the combination of Trapezoidal Rule, which can be used in any number of region divisions and Simpson's Rule are chosen to find the area under a curve in this research.
From Figure 11, the area underneath the curve from time interval t-3 to t can be determined as below: Rearrange the Equation 18: By applying the Trapezoidal rule and Simpson's rule, the Equation 19 can be further simplified as follows: Hence, the background value, z (1) (t) is: where, t = 3, 4,……,n. Preliminary Network Security Situation Prediction is the first part in this module to forecast the incoming security situation according to the previous and current situation assessment. This component adaptively modified Grey Verhulst algorithm which is more suitable to be used in predicting intrusion attack situation with its S-Shape behavior. The sequence of Network Security Situation Assessment, output from the Security Situation Assessment has been used as input in this prediction component. The process flow of Preliminary Network Security Situation Prediction is shown in Fig. 9.
First, a sequence of situational values, X (0) is retrieved from Security Situation Assessment: With this sequence, a new sequence of accumulated data is built by applying the 1-Accumulated Generating Operation (1-AGO) to it. The accumulated data sequence, X (1) is as Equation 23: where: for X (0) (i)≥0 and t = 1, 2,……, n. Second, a sequence of adaptive background value, Z (1) (t) is generated by considering the consecutive (before and after) neighbors of x (1) : where: The value of z (1) (t) is substituted into Grey Verhulst model below: where a is development coefficient which its size reflects the growth rate of the sequence X (0) and b is the role of vector which is grey input in Grey Verhulst model. After that, the equations are rearranged into matrix form Y =

Bâ with the parameter matrix
the matrixes of B and Y as below: where: where, x (0) (1) = x (1) (1). Finally, to obtain the preliminary network security situation prediction value, ( ) ( ) 0 1 g x t + , the Inverse Accumulating Generation Operation (IAGO) has been applied. As can be determined by using the formula below: and: where, t = 2, 3, ……, n. For Residual Prediction, it is second part in the Network Security Situation Prediction Module. It mainly focuses on forecasting the next prediction deviation based on the previous prediction residual. This component intent to ensure the predicted value closer to its actual value in a particular time-interval. In this research, Kalman filtering has been applied to achieve this intention. In general, Kalman filtering is an optimal estimator which can provide an efficient recursive computation means to infer the state of a process in order to minimize the mean of the squared error (Welch and Bishop, 2004). It has some significant features such as less parameters, simple calculation and convenient in real time processing (Lin et al., 2014). Thus, it has been chosen to forecast the incoming residual of adaptive grey Verhulst prediction in the research. The process flow of residual prediction is depicted in Fig. 10.
Firstly, the sequences of actual value and adapted grey Verhulst predicted value of network security situation are used to build a sequence of prediction error, The more meaningful variables to be initialized, the faster convergence will be achieved. Then the process will be started from time update equations. To project the state, 1| t t r + and error covariance, P t+1|t ahead, the equations below are used respectively. The model is assumed constant, therefore F t+1 = 1 for any t ≥0. Control variables are not being used, so B = 0 and u = 0: After completing the time update equations, the first task during the measurement update is to compute the Kalman gain, K t+1 as Equation 44. Due to the same scale of state estimate, r has been used in the measurement, therefore in this research, H = 1: With the Kalman gain, the next step is to measure the process to obtain measurement variable, y and then to generate a posteriori state estimate, 1| 1 t t r + + : where: the deviation from real value on previous network security situation prediction. The final step is to update the posteriori error variance estimate through Equation 47: After completing each time and measurement update pair, the process is repeated with the previous posteriori state estimate to project the new priori estimates.
In order to acquire the predicted value for incoming network security situation, preliminary network security situation prediction value, ( ) ( ) 0 g x t which computed by using adaptive grey Verhulst model and residual prediction value 1| 1 t t r + + which obtained from Kalman Filtering model have been combined. Figure 11 presents the process flow for acquiring the final network security situation prediction in the proposed mechanism.
Thus, the calculation of final predicted network security situation, ( ) ( ) 0 1 p x t + as Equation 48: x t x t r + + + = + + Then, the result obtained will sent back to the predicted network security situation sequence for updating purpose. Lastly, all the processes in the proposed mechanism will be repeated and the sequences are keep updating timely.

Case Study and Results
A new NAv6 2015 Intrusion Detection Dataset has also been generated and used for testing purpose as the model takes into account the importance of network assets in assessing the security situation. A testbed which simulates NAv6 Centre's network environment was designed and implemented in order to generate the test set as the input for NESSAP mechanism. Figure 12 illustrates the testbed for NAv6 2015 dataset generation.
The testbed was created by using VMware vSphere. There are three components in this testbed topology which are attacker machines, intrusion detection system and victim servers. Attack machines were designed to launch the probing and malicious payload attacks which can be detected for evaluating the performance of the proposed mechanism. Meanwhile, intrusion detection system was created to detect the attack packets from outsiders (attack machines) in a particular time period and victim machines which represent the servers in NAv6 Centre such as web server, database server, mail server and Domain Name System (DNS) server were connected to the outsiders (attack machines) through intrusion detection system and they act as attack targets in this testbed.
To run the testbed, two Ubuntu-based Linux attacker hosts were created and BackTrack 5 R3 was installed on them. The attackers typically launch the attacks on victim servers which represented the assets in the network. Meanwhile, Snort was deployed to capture the suspicious packets from the attackers. The log file generated by Snort was stored and used as input for evaluating the performance of NESSAP mechanism.

Asset Weight for NAv6 2015 Dataset
A new dataset had been generated for overall evaluation of the mechanism, i.e. NAv6 2015 dataset. This dataset was captured under the simulation of NAv6's network environment. The dataset consists of 861 intrusion alerts generated by Snort by analyzing probing attacks and malicious payload attacks. The dataset was generated by capturing network activity laced with attacks over a period of 36 h and it involved four main servers in NAv6, web server, database server, main server and DNS server.
The weight assignment of NAv6 2015 dataset was based on the concept of Analytic Hierarchy Process (AHP). A set of AHP table forms for each type of attack had been filled up by the network administrator from NAv6 Centre based on his knowledge and experience as well as business policy of the Centre. An average priority which represents the importance of the servers for each attack types such as probing attack, denial of service attack, remote to user attack and user to root attack have been calculated. Then by averaging all average priority values of each server for every type of attack, the asset weight which represents the importance of servers in this dataset is calculated as illustrated in Table 3.

Assessment Result for NAv6 2015 Dataset
In order to examine the flexibility of the proposed E-NESSAS module in assessing security situation in various length of time interval, NAv6 2015 dataset had been divided based on 2 h time interval and it has 18 time frames in total. The NAv6 2015 dataset was then channeled into the E-NESSAS prototype to test the capability of proposed module in assessing the security situation. The situational value obtained from the experiment which represents the tendency of security situation with tangible and intangible criteria consideration on each asset in real environment is shown in Table 4.
The result showed that the situational value of network security situation in NAv6 Centre during the attacks in this dataset was between 0.07 and 0.26. The network situation started with low situational value and grew up steadily throughout the period from 11:01 to 17:00. During the increment, the servers, in this experiment were vulnerable to attacks by the attackers outside the network through the probing attacks. The graph showed that there was a slightly drop of situational value at time frame 17:01-19:00 and reverted to reach 0.2574 in the following hours. The changes were matched with the attack launched in the experiment. The attacker stopped the probing attack that time-frame and started the malicious payload attack at the last 4 hours in the experiment. By observing the changes of situational values which illustrate the trend movement of these values are matched with the attacks launched by attackers in particular time period. Therefore, it proved that the proposed E-NESSAS module can assess the security situation in any time interval with the aid of AHP in determining the importance of assets.

Prediction Result for NAv6 2015 Dataset
The experiment was conducted to evaluate the predictive accuracy performance of proposed grey prediction model. It used a set of 2 h time interval situational values which was generated by E-NESSAS model as input to channel into the developed prototype. The first 11 situational values were used to predict the last 7 situational values in NAv6 2015 dataset.

Prediction Results
In this section, the performance of AGV-NESSIP and AGVK-NESSIP models were evaluated. The purpose of this evaluation is to determine the appropriateness of considering the previous prediction error in the situation prediction. The results obtained are presented in Table 5.     The results showed that the MAPE of proposed AGVK-NESSIP and AGV-NESSIP models are 17.23 and 31.36% respectively. In other words, the prediction accuracy of AGVK-NESSIP model is almost 14% better than AGV-NESSIP model. As for AGVK-NESSIP model, it has a lower RPE than AGV-NESSIP model. It remains the RPE below 20% is at all the time frames except at 1901-2100 and 2101-2300. Due to this, AGVK-NESSIP model has smaller RMSD which is only 0.04 while AGV-NESSIP model has only 0.08 in this experiment. Hence, it showed that the proposed AGVK-NESSIP model has significant predictive accuracy compared with AGV-NESSIP model. Although there is a up and down in network situation between 1501 to 2101, but the result shows that AGVK-NESSIP model is still outperforms compared to AGV-NESSIP model in this situation. Therefore, it can be concluded the accuracy of prediction for the proposed prediction model can improve by adding residual prediction in the situation prediction.

Comparison with other Grey Prediction Models
For the purpose in claiming that the proposed grey prediction model has superior performance in forecasting incoming network security situation, a comparison test has been conducted among existing grey prediction models. Table 6 presents the predicted situational values generated by prediction models at every time interval.
The results showed that only traditional GM(1,1) and proposed AGVK-NESSIP models have achieved predictive accuracy above 80% while traditional grey Verhulst and AGV-NESSIP models only managed to achieve around 68 and 65% respectively. Although both traditional GM(1,1) and AGVK-NESSIP models have promising prediction results, but AGVK-NESSIP model has performed slightly better in this case with 0.84% lower MAPE and 0.02 smaller RMSD. In this experiment, EDGV Verhulst model has demonstrated the weakest performance due to its highest MAPE and RMSD compared with other prediction models. In this context, the assumption made in EDGF Verhulst model is considered inappropriate for the dataset. In other words, the performance of EDGF Verhulst is unconvincing and limited to certain network situations only. Meanwhile, compared with other prediction models, AGVK-NESSIP model behaves closer to the real value especially at 1701-1900. Hence, the results proved that the performance of AGVK-NESSIP model in the aspect of predictive accuracy surpassed other grey prediction models in forecasting incoming network security situation.