DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH
DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH
ABSTRACT
Attacks on computers and data networks have become a regular and sophisticated issue. Intrusion detection has shifted its attention from hosts and operating systems to networks and has become a way to provide a sense of security to these networks. Intrusion detection aims to detect misuse and unauthorised use of computer systems by internal and external elements. Typically, intrusion detection systems allow statistical anomalies and rule-based misuse models to detect intrusions, as the behaviour of the intruding element is considered to be different from the authorised user behaviour. Machine learning techniques provide a promising result in improving the detection accuracy of intrusion detection systems (IDS). A variety of machine learning techniques have been designed and integrated with IDSs. But most of the intrusion detection systems still have a poor intrusion detection rate and a high false-positive rate. This thesis focused on the ensemble method, which involves the integration of predictions by multiple individual classifiers. Ensemble methods enable individuals to compensate for the weaknesses of individual classifiers and use their combined knowledge to enhance their performance. Different ensemble methods in the field are analysed, taking into consideration different types of ensembles and various approaches for integrating the predictions of individual classifiers into an ensemble classifier. This research has attempted to build a predictive ensemble ML model for intrusion detection using a new standard dataset from the Canadian Institute for Cyber Security intrusion detection system (CIC-IDS2017) for performance evaluation. Simulation outcomes prove that the proposed ensemble model outperforms current IDS systems, attaining accuracy of up to 99%. The performance of this algorithm is measured using accuracy, precision, false positive, F1 score, and recall which found promising results for deployment on real network infrastructure.
Keywords: Cyber Security, Intrusion Detection, Machine learning algorithms, ensemble model, CIC-IDS2017Datase.
CHAPTER 1: INTRODUCTION
1.1 Background of the study
With the constant growth of the Internet, cyber-attacks are increasing in numbers and diversity; ransomware is on the rise like never before; and zero-day exploits have become so critical that they are gaining media coverage. Antiviruses and firewalls are no longer sufficient to ensure the protection of a company network, which should be based on combined layers of security. One of the most important layers, designed to protect its target against any potential attack through continuous monitoring of the system, is provided by an Intrusion Detection System (IDS).
Cybersecurity is a growing problem in modern times because of rapid growth and technological advancement. The internet provides all the knowledge that has been accumulated by man, and with the advent of mobile computing at every person’s fingertips, cyber-attacks and cybercrimes have become all too popular. A report from the anti-phishing working group has shown that about 227,000 malware detections occur daily, which is linked to over 20 million new malwares daily [1]. There has been a straightforward method for dealing with malware in the past, but over the past two decades, there has been an evolution in cyber-attacks and how exploits are carried out. As such, cyber security techniques are also undergoing an evolution into more intelligent approaches.
Network and internet communication is rapid and uses electronic devices like computers, laptops, mobiles, etc. to transfer, process, store, and retrieve information. The security issue is very important. Nowadays, cyberattacks are highly increasing all over the world. Africa lost $3.5b to cyber security attacks in 2020 [2]. In our country, Ethiopia, many people are connected to the internet and other networks; even the majority of society has not gotten access yet. In the last few years’ cyberattacks have been sharply increasing all over the world.
An activity known as a cyberattack or network intrusion aims to jeopardise a computer network's regular operation. We must create an intrusion detection mechanism, which is a way to lessen or report these incursions, to protect against cyberattacks. Yet, with typical IDS, monitoring and detecting intrusions at very fast network speeds and during an uptick in distributed denial of service (DDoS) attacks becomes challenging. There have been numerous attempts in recent years to design effective intrusion detection systems (IDS) (IDS) (IDS) to address these difficulties. IDS is a program that keeps an eye out for unusual activities that could jeopardise the network's confidentiality, integrity, and availability (CIA) qualities. It includes monitoring the unwanted utilisation of network resources, keeping them available for legitimate users, and, in some cases, preventing the loss of information/data to the intruder.
The most efficient way to tackle this growing problem involves the use of machine learning algorithms to detect attacks before they attack a legitimate system. A lot of network logs (IDS dataset) already exist from past attacks; this log file can be fed into the algorithm to train it to recognise attacks and send alerts to the system administrator or intrusion prevention system (IPS) [4].
To detect these intrusions, various intrusion detection systems (IDSs) are implemented in many organisations' networks. These systems are classified into host-based IDS, network-based (NIDS), and hybrid IDS. HIDS monitors the system and looks for malicious activities, and NIDS examines the traffic payload in the network for suspicious events. [5] Based on detection methods, IDS is characterised into two types, namely signature-based IDS and anomaly-based IDS [6].
The lack of a comprehensive network-based data set that can depict contemporary network traffic scenarios, a wide range of low-footprint intrusions, and deeply structured information about network traffic is one of the primary research difficulties in this subject [6]. A decade ago, the benchmark data sets KDD98, KDDCUP99, and NSLKDD were created to assess network intrusion detection systems research efforts. However, multiple recent studies have revealed that these data sets do not completely capture network traffic and contemporary low-footprint assaults in the current network threat environment [7]. Countering the unavailaIntrusion Detection Systemk data set challenges, this paper will examine a Canadian Institute for Cyber security intrusion detection system (CIC-IDS2017) dataset This data set has a hybrid of the real modern normal and the contemporary synthesised attack activities of the network traffic [7]. Existing and novel methods are utilised to generate the features of the CIC-IDS2017 data set. This data set is available for research purposes and can be accessed freely.
The goal of this thesis is to suggest strategies for improving the detection quality of intrusion detection systems (IDS) utilising machine learning approaches for implementation on real-world networks. This research has attempted to build an ensemble ML model for intrusion detection using a new standard dataset from the Canadian Institute for Cyber Security Intrusion Detection System (CIC-IDS2017) for performance evaluation.
1.2 Statement of the Problem
Machine learning techniques provide a promising solution for improving intrusion detection systems (IDS) [8]. Machine learning techniques are classified into supervised and unsupervised learning techniques. Supervised learning needs a training dataset with labelled instances for normal as well as anomaly classes, whereas, in unsupervised learning, the algorithm directly learns patterns from the data without any human intervention. Developing an IDS model with better accuracy and a low false positive detection system has become an important solution to detect existing and emerging attacks. Using ensemble model machine learning algorithms for intrusion detection can improve its overall performance as the weaknesses of one algorithm might be complemented by the second one. The security system of an enterprise network should be improved in line with technological advancements to enhance network security defences. This research has attempted to build ensemble machine-learning algorithms for intrusion detection using a new benchmark dataset from the Canadian Institute for Cyber Security Intrusion Detection System (CIC-IDS2017) for performance evaluation. Applying a new dataset to meet the current significant advances in internet traffic diversity and emerging attack types is mandatory.
The ensemble model machine learning algorithm intrusion detection method is capable of detecting attacks with high accuracy [4]. Ensemble model machine learning algorithms NIDS attempts to overcome the shortcomings of accuracy and false positive rates NIDSs [9] Based on the literature review, many intrusion detection-related papers specific to machine learning-based IDSs have been developed. However, the existing IDSs still have their shortcomings. Some of the shortcomings are the low detection rate, high training time, low processing speed, and relatively high false alarm rate (FAR).
To address these issues, several attempts have been made in recent years to develop effective IDS. However, there is still room for development in these systems. The suggested study effort centred on constructing an intrusion detection model with a higher detection rate, reduced training time, and enhanced performance by parallelizing training and selecting the optimal parameters for models that can increase model performance.
To strengthen network security defences, enterprise networks' security systems should be developed in step with technological innovation. The goal of this study was to develop a predictive ensemble ML model for intrusion detection utilising the new benchmark dataset CIC-IDS2017 from the Canadian Institute for Cyber Security. To address the present major improvements in internet traffic diversity and emerging attack types, a new dataset must be used. High rates of false-positive alarms and low detection rates for zero-day assaults are the two main drawbacks of current intrusion detection systems. To overcome these problems, we need intrusion detection techniques that can learn and effectively detect intrusions. Ensemble methods based on machine learning techniques have been proposed by different researchers. These methods take advantage of the single detection methods and leverage their weaknesses [10].
The detection rate, false alarm rate, complexity, and evaluation of both known and new threats were generally neglected in earlier attempts at intrusion detection systems. The goal of this study is to close the gap between the aforementioned issues. The ensemble approach refers to the fusion of various machine-learning algorithms. According to a review of the literature, machine learning's ensemble method lowers the rate of false positives. Basic machine learning classifiers can be combined using four major techniques: bagging, boosting, randomization, and stacking. The implementation of an intrusion detection system using the bagging-based ensemble approach is suggested in this research. In this study, the ensemble model and benchmark dataset of the Canadian Institute for Cyber Security Intrusion Detection System (CIC-IDS2017) datasets are used for model evaluation. This ensemble model combines the output of several classifiers and produces a single composite classification.
The suggested ensemble machine learning model consists of two distinct algorithms. The first is a random forest, while the second is the bagging method found on the WEAK tool. The proposed model is trained using supervised data. This distinction can actually serve to improve overall detection rates since the shortcomings of one approach may be addressed by the other. So, the goal of this research will be to create a model that is more capable of recognising diverse cyberattacks and has the potential to reduce the percentage of false positives by boosting the accuracy of detecting newer attacks.
1.3 Motivation
The motivations of this research work are the application of intrusion detection systeaboutnt organisations, the availability of open sources, and the weaknesses of currently available network security tools with regard to detecting intrusion. Even though intrusion detection systems are applicable in different organisations and have been used for more than a decade, there are still many issues with IDS. Including false positives and low detection capacities.
1.4 Research question
There are several problems associated with IDS. In this research, we will address the following questions:
- How can we minimise intrusion?
- Which network intrusion detection dataset is better for simulation?
- Which detection technique is the best to use for detection rate enhancement?
- How can we detect unknown attacks and minimise the false alarm rate?
1.5 Objectives
1.5.1 General Objective
The general objective of this study is to build an ML model for the network intrusion detection system (NIDS) using an ensemble approach that will enhance the computer network security system.
1.5.2 Specific Objectives
The specific objectives of this research study are:
- To study different types of intrusion detection systems for classification.
- To conduct training and testing of the predictive models using the new CIC-IDS2017 benchmark dataset.
- To design a suitable machine learning log classification and attack prediction model.
- To extract the most prominent features and apply classification techniques.
- To compare the accuracy rates of different classifiers.
- To apply pre-processing techniques to the CIC-IDS2017 benchmark dataset
- To interpret and analyze the results of the selected model
1.6 Significance of the study
- Improve the intrusion detection system (IDS). System administrators and IPS use the result as input, which helps the prevention mechanism be proactive rather than reactive. The technique by which an attack occurs and responses are applied indicates a reactive approach, whereas making prevention before an attack occurs and building a model is a proactive approach.
- Reduces cyber security risk impact; DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH Knowing the behaviors of users are very important to prevent asset damage early by applying IPS based on specified features.
- Reduce the staff cost and misconfiguration of using the SIEM system.
- Reduce cyberattacks and their impacts, such as financial, political, and social.
1.7 Scope
In this thesis, we will design an ensemble intrusion detection system. It focuses on identifying possible cyber incidents and reporting them to the security administrators, or IPS. This system is designed to increase detection rates and reduce false positive rates and applies to any organization's network. The data used for this thesis was obtained from a publicly available, state-of-the art IDS dataset.
One of the limitations of this research work is that the dataset is from the Canadian Institute for Cyber Security organisation and cannot directly implement the trained model in a specific organization's network. This is due to the network infrastructure and configuration of one organisation being different from the others.
1.8 Organisation of the Thesis Report
The following is an overview of the structure of this thesis. The first chapter gives an introduction to this research, giving a statement of the problem, thesis objectives, motivation, and scope of this work. This is followed by a second chapter that introduces conceptual information on intrusion detection and related works in the field of machine learning-based intrusion detection systems using different detection techniques. It also discusses how intrusion detection systems are classified. The third chapter introduces the research methods, algorithms, and datasets to be used in this paper. The fourth chapter introduces the research experiment and explores the study done, including evaluation setup, The fifth chapter includes a performance analysis of the selected algorithms. and chapter six introduces concluding remarks and presents ideas for improvements and recommendations for future research.
DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH, GET MORE PROJECT TOPICS AND MATERIALS
Attached Files
DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH.docx
₦5,000.00
DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH
DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH
ABSTRACT
Attacks on computers and data networks have become a regular and sophisticated issue. Intrusion detection has shifted its attention from hosts and operating systems to networks and has become a way to provide a sense of security to these networks. Intrusion detection aims to detect misuse and unauthorised use of computer systems by internal and external elements. Typically, intrusion detection systems allow statistical anomalies and rule-based misuse models to detect intrusions, as the behaviour of the intruding element is considered to be different from the authorised user behaviour. Machine learning techniques provide a promising result in improving the detection accuracy of intrusion detection systems (IDS). A variety of machine learning techniques have been designed and integrated with IDSs. But most of the intrusion detection systems still have a poor intrusion detection rate and a high false-positive rate. This thesis focused on the ensemble method, which involves the integration of predictions by multiple individual classifiers. Ensemble methods enable individuals to compensate for the weaknesses of individual classifiers and use their combined knowledge to enhance their performance. Different ensemble methods in the field are analysed, taking into consideration different types of ensembles and various approaches for integrating the predictions of individual classifiers into an ensemble classifier. This research has attempted to build a predictive ensemble ML model for intrusion detection using a new standard dataset from the Canadian Institute for Cyber Security intrusion detection system (CIC-IDS2017) for performance evaluation. Simulation outcomes prove that the proposed ensemble model outperforms current IDS systems, attaining accuracy of up to 99%. The performance of this algorithm is measured using accuracy, precision, false positive, F1 score, and recall which found promising results for deployment on real network infrastructure.
Keywords: Cyber Security, Intrusion Detection, Machine learning algorithms, ensemble model, CIC-IDS2017Datase.
CHAPTER 1: INTRODUCTION
1.1 Background of the study
With the constant growth of the Internet, cyber-attacks are increasing in numbers and diversity; ransomware is on the rise like never before; and zero-day exploits have become so critical that they are gaining media coverage. Antiviruses and firewalls are no longer sufficient to ensure the protection of a company network, which should be based on combined layers of security. One of the most important layers, designed to protect its target against any potential attack through continuous monitoring of the system, is provided by an Intrusion Detection System (IDS).
Cybersecurity is a growing problem in modern times because of rapid growth and technological advancement. The internet provides all the knowledge that has been accumulated by man, and with the advent of mobile computing at every person’s fingertips, cyber-attacks and cybercrimes have become all too popular. A report from the anti-phishing working group has shown that about 227,000 malware detections occur daily, which is linked to over 20 million new malwares daily [1]. There has been a straightforward method for dealing with malware in the past, but over the past two decades, there has been an evolution in cyber-attacks and how exploits are carried out. As such, cyber security techniques are also undergoing an evolution into more intelligent approaches.
Network and internet communication is rapid and uses electronic devices like computers, laptops, mobiles, etc. to transfer, process, store, and retrieve information. The security issue is very important. Nowadays, cyberattacks are highly increasing all over the world. Africa lost $3.5b to cyber security attacks in 2020 [2]. In our country, Ethiopia, many people are connected to the internet and other networks; even the majority of society has not gotten access yet. In the last few years’ cyberattacks have been sharply increasing all over the world.
An activity known as a cyberattack or network intrusion aims to jeopardise a computer network's regular operation. We must create an intrusion detection mechanism, which is a way to lessen or report these incursions, to protect against cyberattacks. Yet, with typical IDS, monitoring and detecting intrusions at very fast network speeds and during an uptick in distributed denial of service (DDoS) attacks becomes challenging. There have been numerous attempts in recent years to design effective intrusion detection systems (IDS) (IDS) (IDS) to address these difficulties. IDS is a program that keeps an eye out for unusual activities that could jeopardise the network's confidentiality, integrity, and availability (CIA) qualities. It includes monitoring the unwanted utilisation of network resources, keeping them available for legitimate users, and, in some cases, preventing the loss of information/data to the intruder.
The most efficient way to tackle this growing problem involves the use of machine learning algorithms to detect attacks before they attack a legitimate system. A lot of network logs (IDS dataset) already exist from past attacks; this log file can be fed into the algorithm to train it to recognise attacks and send alerts to the system administrator or intrusion prevention system (IPS) [4].
To detect these intrusions, various intrusion detection systems (IDSs) are implemented in many organisations' networks. These systems are classified into host-based IDS, network-based (NIDS), and hybrid IDS. HIDS monitors the system and looks for malicious activities, and NIDS examines the traffic payload in the network for suspicious events. [5] Based on detection methods, IDS is characterised into two types, namely signature-based IDS and anomaly-based IDS [6].
The lack of a comprehensive network-based data set that can depict contemporary network traffic scenarios, a wide range of low-footprint intrusions, and deeply structured information about network traffic is one of the primary research difficulties in this subject [6]. A decade ago, the benchmark data sets KDD98, KDDCUP99, and NSLKDD were created to assess network intrusion detection systems research efforts. However, multiple recent studies have revealed that these data sets do not completely capture network traffic and contemporary low-footprint assaults in the current network threat environment [7]. Countering the unavailaIntrusion Detection Systemk data set challenges, this paper will examine a Canadian Institute for Cyber security intrusion detection system (CIC-IDS2017) dataset This data set has a hybrid of the real modern normal and the contemporary synthesised attack activities of the network traffic [7]. Existing and novel methods are utilised to generate the features of the CIC-IDS2017 data set. This data set is available for research purposes and can be accessed freely.
The goal of this thesis is to suggest strategies for improving the detection quality of intrusion detection systems (IDS) utilising machine learning approaches for implementation on real-world networks. This research has attempted to build an ensemble ML model for intrusion detection using a new standard dataset from the Canadian Institute for Cyber Security Intrusion Detection System (CIC-IDS2017) for performance evaluation.
1.2 Statement of the Problem
Machine learning techniques provide a promising solution for improving intrusion detection systems (IDS) [8]. Machine learning techniques are classified into supervised and unsupervised learning techniques. Supervised learning needs a training dataset with labelled instances for normal as well as anomaly classes, whereas, in unsupervised learning, the algorithm directly learns patterns from the data without any human intervention. Developing an IDS model with better accuracy and a low false positive detection system has become an important solution to detect existing and emerging attacks. Using ensemble model machine learning algorithms for intrusion detection can improve its overall performance as the weaknesses of one algorithm might be complemented by the second one. The security system of an enterprise network should be improved in line with technological advancements to enhance network security defences. This research has attempted to build ensemble machine-learning algorithms for intrusion detection using a new benchmark dataset from the Canadian Institute for Cyber Security Intrusion Detection System (CIC-IDS2017) for performance evaluation. Applying a new dataset to meet the current significant advances in internet traffic diversity and emerging attack types is mandatory.
The ensemble model machine learning algorithm intrusion detection method is capable of detecting attacks with high accuracy [4]. Ensemble model machine learning algorithms NIDS attempts to overcome the shortcomings of accuracy and false positive rates NIDSs [9] Based on the literature review, many intrusion detection-related papers specific to machine learning-based IDSs have been developed. However, the existing IDSs still have their shortcomings. Some of the shortcomings are the low detection rate, high training time, low processing speed, and relatively high false alarm rate (FAR).
To address these issues, several attempts have been made in recent years to develop effective IDS. However, there is still room for development in these systems. The suggested study effort centred on constructing an intrusion detection model with a higher detection rate, reduced training time, and enhanced performance by parallelizing training and selecting the optimal parameters for models that can increase model performance.
To strengthen network security defences, enterprise networks' security systems should be developed in step with technological innovation. The goal of this study was to develop a predictive ensemble ML model for intrusion detection utilising the new benchmark dataset CIC-IDS2017 from the Canadian Institute for Cyber Security. To address the present major improvements in internet traffic diversity and emerging attack types, a new dataset must be used. High rates of false-positive alarms and low detection rates for zero-day assaults are the two main drawbacks of current intrusion detection systems. To overcome these problems, we need intrusion detection techniques that can learn and effectively detect intrusions. Ensemble methods based on machine learning techniques have been proposed by different researchers. These methods take advantage of the single detection methods and leverage their weaknesses [10].
The detection rate, false alarm rate, complexity, and evaluation of both known and new threats were generally neglected in earlier attempts at intrusion detection systems. The goal of this study is to close the gap between the aforementioned issues. The ensemble approach refers to the fusion of various machine-learning algorithms. According to a review of the literature, machine learning's ensemble method lowers the rate of false positives. Basic machine learning classifiers can be combined using four major techniques: bagging, boosting, randomization, and stacking. The implementation of an intrusion detection system using the bagging-based ensemble approach is suggested in this research. In this study, the ensemble model and benchmark dataset of the Canadian Institute for Cyber Security Intrusion Detection System (CIC-IDS2017) datasets are used for model evaluation. This ensemble model combines the output of several classifiers and produces a single composite classification.
The suggested ensemble machine learning model consists of two distinct algorithms. The first is a random forest, while the second is the bagging method found on the WEAK tool. The proposed model is trained using supervised data. This distinction can actually serve to improve overall detection rates since the shortcomings of one approach may be addressed by the other. So, the goal of this research will be to create a model that is more capable of recognising diverse cyberattacks and has the potential to reduce the percentage of false positives by boosting the accuracy of detecting newer attacks.
1.3 Motivation
The motivations of this research work are the application of intrusion detection systeaboutnt organisations, the availability of open sources, and the weaknesses of currently available network security tools with regard to detecting intrusion. Even though intrusion detection systems are applicable in different organisations and have been used for more than a decade, there are still many issues with IDS. Including false positives and low detection capacities.
1.4 Research question
There are several problems associated with IDS. In this research, we will address the following questions:
1.5 Objectives
1.5.1 General Objective
The general objective of this study is to build an ML model for the network intrusion detection system (NIDS) using an ensemble approach that will enhance the computer network security system.
1.5.2 Specific Objectives
The specific objectives of this research study are:
1.6 Significance of the study
1.7 Scope
In this thesis, we will design an ensemble intrusion detection system. It focuses on identifying possible cyber incidents and reporting them to the security administrators, or IPS. This system is designed to increase detection rates and reduce false positive rates and applies to any organization's network. The data used for this thesis was obtained from a publicly available, state-of-the art IDS dataset.
One of the limitations of this research work is that the dataset is from the Canadian Institute for Cyber Security organisation and cannot directly implement the trained model in a specific organization's network. This is due to the network infrastructure and configuration of one organisation being different from the others.
1.8 Organisation of the Thesis Report
The following is an overview of the structure of this thesis. The first chapter gives an introduction to this research, giving a statement of the problem, thesis objectives, motivation, and scope of this work. This is followed by a second chapter that introduces conceptual information on intrusion detection and related works in the field of machine learning-based intrusion detection systems using different detection techniques. It also discusses how intrusion detection systems are classified. The third chapter introduces the research methods, algorithms, and datasets to be used in this paper. The fourth chapter introduces the research experiment and explores the study done, including evaluation setup, The fifth chapter includes a performance analysis of the selected algorithms. and chapter six introduces concluding remarks and presents ideas for improvements and recommendations for future research.
DESIGN AND IMPLEMENTATION OF A PREDICTIVE MODEL OF NETWORK INTRUSION DETECTION SYSTEMS USING MACHINE LEARNING APPROACH, GET MORE PROJECT TOPICS AND MATERIALS
Attached Files