INPUT SPLITS DESIGN TECHNIQUES FOR NETWORK INTRUSION DETECTION ON HADOOP CLUSTER

Vladimir M. Ćirić, Dušan Cvetković, Nadja Gavrilović, Natalija Stojanović, Ivan Milentijević

DOI Number
https://doi.org/10.2298/FUEE2102239C
First page
239
Last page
257

Abstract


Intrusion detection system (IDS) is one of the most important components being used to monitor network for possible cyber-attacks. However, the amount of data that should be inspected imposes a great challenge to IDSs. With recent emerge of various
big data technologies, there are ways for overcoming the problem of the increased amount of data. Nevertheless, some of this technologies inherit data distribution techniques that can be a problem when splitting a sensitive data such as network data frames across a cluster nodes. The goal of this paper is design and implementation of Hadoop based IDS. In this paper we propose different input split techniques suitable for network data distribution across cloud nodes and test the performances of their Apache Hadoop implementation. Four different data split techniques will be proposed and analysed. The techniques will be described in detail. The system will be evaluated on Apache Hadoop cluster with 17 slave nodes. We will show that processing speed can differ for more than 30% depending on chosen input split design strategy. Additionally, we’ll show that malicious level of network traffic can slow down the processing time, in our case, for nearly 20%. The scalability of the system will al so be discussed.

Keywords

Network Intrusion Detection, Cloud Computing, Apache Hadoop

Full Text:

PDF

References


L. A. Maglaras, K.-H. Kim, H. Janicke, M. A. Ferrag, S. Rallis, P. Fragkou, A. Maglaras, and T. J. Cruz, “Cyber security of critical infrastructures,” Ict Express, vol. 4, no. 1, pp. 42–45, 2018.

M. A. Ferrag, L. Maglaras, S. Moschoyiannis, and H. Janicke, “Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study,” Journal of Information Security and Applications, vol. 50, pp. 1–19,

J. Svoboda, I. Ghafir, V. Prenosil et al., “Network monitoring approaches: An overview,” Int J Adv Comput Netw Secur, vol. 5, no. 2, pp. 88–93, 2015.

I. Ghafir, V. Prenosil, J. Svoboda, and M. Hammoudeh, “A survey on network security monitoring systems,” in 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW). IEEE, 2016, pp. 77–82.

B. Schneier, “Managed security monitoring: Network security for the 21st century,” Computers & Security, vol. 20, no. 6, pp. 491–503, 2001.

G. Kumar, K. Kumar, and M. Sachdeva, “The use of artificial intelligence based techniques for intrusion detection: a review,” Artificial Intelligence Review, vol. 34, no. 4, pp. 369–387, 2010.

M. Aldwairi and D. Alansari, “Exscind: Fast pattern matching for intrusion detection using exclusion and inclusion filters,” in 2011 7th International Conference on Next Generation Web Services Practices. IEEE, 2011, pp. 24–30.

D. Xu, H. Zhang, and Y. Fan, “The gpu-based high-performance patternmatching algorithm for intrusion detection,” Journal of computational information

systems, pp. 3791–3800, 2013.

M. Kharbutli, M. Aldwairi, and A. Mughrabi, “Function and data parallelization of wu-manber pattern matching for intrusion detection systems.” Netw. Protoc. Algorithms, vol. 4, no. 3, pp. 46–61, 2012.

X. Su, Z. Ji, and X. Lian, “A parallel ac algorithm based on spmd for intrusion detection system,” in Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering. Atlantis Press, 2013.

M. Aldwairi, A. M. Abu-Dalo, and M. Jarrah, “Pattern matching of signaturebased ids using myers algorithm under mapreduce framework,” EURASIP Journal on Information Security, vol. 2017, no. 1, pp. 1–11, 2017.

V. Ciric, D. Cvetkovic, and I. Milentijevic, “Design and implementation of network intrusion detection system on the apache hadoop platform,” in Proceedings on 5th International Conference on Electrical, Electronic, and Computer Engineering (IcETRAN 2018), Palic, Serbia, 2018, pp. 102–1105.

C. Lam, Hadoop in action. Manning Publications Co., 2010.

M. Y. Eltabakh, Y. Tian, F. ¨Ozcan, R. Gemulla, A. Krettek, and J. McPherson, “Cohadoop: flexible data placement and its exploitation in hadoop,” Proceedings of the VLDB Endowment, vol. 4, no. 9, pp. 575–585, 2011.

A. Sayar et al., “Hadoop optimization for massive image processing: case study face detection,” International Journal of Computers Communications & Control, vol. 9, no. 6, pp. 664–671, 2014.

J. Cheon and T.-Y. Choe, “Distributed processing of snort alert log using hadoop,” International Journal of Engineering and Technology, vol. 5, no. 3, pp. 2685–2690, 2013.

P. Prathibha and E. Dileesh, “Design of a hybrid intrusion detection system using snort and hadoop,” International journal of computer applications, vol. 73, no. 10, 2013.

K. Kato and V. Klyuev, “Development of a network intrusion detection system using apache hadoop and spark,” in 2017 IEEE Conference on Dependable and Secure Computing. IEEE, 2017, pp. 416–423.

C. F. Endorf, E. Schultz, and J. Mellander, Intrusion detection & prevention. McGraw-Hill Osborne Media, 2004.

H.-D. J. Jeong, W. Hyun, J. Lim, and I. You, “Anomaly teletraffic intrusion detection systems on hadoop-based platforms: A survey of some problems and solutions,” in 2012 15th International Conference on Network-Based Information

Systems. IEEE, 2012, pp. 766–770.

A. Khraisat, I. Gondal, P. Vamplew, and J. Kamruzzaman, “Survey of intrusion detection systems: techniques, datasets and challenges,” Cybersecurity, vol. 2, no. 1, p. 20, 2019.

U. ofWashington, “Cpu performance,” ttps://boinc.bakerlab.org/rosetta/cpu list.php, accessed: 2020-10-22.


Refbacks

  • There are currently no refbacks.


ISSN: 0353-3670 (Print)

ISSN: 2217-5997 (Online)

COBISS.SR-ID 12826626