The Hadoop Security in Big Data: A Technological Viewpoint and Analysis

Yusuf Perwej¹

Section:Research Paper, Product Type: Journal-Paper
Vol.7 , Issue.3 , pp.10-14, Jun-2019

CrossRef-DOI:   https://doi.org/10.26438/ijsrcse/v7i3.1014

Online published on Jun 30, 2019

Copyright © Yusuf Perwej . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
 

View this paper at   Google Scholar | DPI Digital Library

XML View     PDF Download

How to Cite this Paper

1197 Views    549 Downloads    108 Downloads

Abstract :
At present, the world is going to become more digital. As every person using the internet today, a huge amount of data gets generated day-to-day. The data are very essential with respect to carrying out their day-to-day activities, and also helping the business management to achieve their aims and make the best decisions on the basis of the information extracted from them. The big data phenomenon is a direct consequence of the digitization and ‘datafication’ of nearly every activity in public, private, and commercial life. Despite that, big data originated new matter related not only to the volume or the diversity of the data, but also to data security. There is need to endow security of such data. The Apache Hadoop platform is used to handle, store, manage, and distribute big data across many server nodes. Here are different tools, which research on the top of Apache Hadoop stack to provide security for data. In order to obtain a full perspective of the problem, we decided to carry out examine with the objective of existing security methods for Apache Hadoop security in big data.

Key-Words / Index Term :
Big Data; Hadoop; Kerberos Protocol; Hadoop Metrics; Amazon Web Services; Security Protocols; Hadoop Security

References :
[1] Yusuf Perwej, “An Experiential Study of the Big Data,” International Transaction of Electrical and Computer Engineers System (ITECES), USA, ISSN (Print): 2373-1273 ISSN (Online): 2373-1281, Vol. 4, No. 1, page 14-25, March 2017, DOI:10.12691/iteces-4-1-3
[2] V Mayer-Schonberger, K Cukier, Big data: a revolution that will transform how we live work and think, Boston:Houghton Mifflin Harcourt, 2013
[3] Yusuf Perwej, Mahmoud Ahmed AbouGhaly, Bedine Kerim and Hani Ali Mahmoud Harb,“An Extended Review on Internet of Things (IoT) and its Promising Applications”, Communications on Applied Electronics (CAE), ISSN : 2394-4714, Foundation of Computer Science FCS, New York, USA, Volume 9, Number 26, Pages 8– 22, February 2019, DOI: 10.5120/cae2019652812
[4] Yusuf Perwej, Majzoob K. Omer, Osama E. Sheta, Hani Ali M. Harb, Mohmed S. Adrees, “The Future of Internet of Things (IoT) and Its Empowering Technology” , International Journal of Engineering Science and Computing (IJESC), ISSN: 2321- 3361, Volume 9, Issue No.3, Pages 20192– 20203, March 2019
[5] Gartner says 4.9 Billion Connected ‘Things’ Will Be in Use in 2015,” Gartner Inc., 2014
[6] Nikhat Akhtar, Firoj Parwej, Dr. Yusuf Perwej, “A Perusal Of Big Data Classification And Hadoop Technology,” International Transaction of Electrical and Computer Engineers System (ITECES), USA, ISSN (Print): 2373-1273 ISSN (Online): 2373-1281, Vol. 4, No. 1, page 26-38, May 2017,
DOI: 10.12691/iteces-4-1-4
[7] Khadija Aziz, Dounia Zaidouni, Mostafa Bellafkih, “Real-time data analysis using Spark and Hadoop”, 4th International Conference on Optimization and Applications (ICOA), IEEE, Mohammedia, Morocco , April 2018
[8] Yusuf Perwej, Md. Husamuddin, Fokrul Alom Mazarbhuiya ,“An Extensive Investigate the MapReduce Technology”, International Journal of Computer Sciences and Engineering (IJCSE), E-ISSN : 2347-2693, Volume-5, Issue-10, Page no. 218-225, Oct-2017, DOI : 10.26438/ijcse/v5i10.218225
[9] Johnson Anumol, P.H. Havinash, Vince. Paul, Mr. Sankaranarayanan, "Big Data Processing Using Hadoop MapReduce Programming Model", International Journal of Computer Science and Information Technologies, vol. 6, no. 1, pp. 127-132, 2015
[10] Tim Hegeman, Yong Guo, Mihai Capota, Bogdan Ghit, "Big Data in the Cloud: Enabling the Fourth Paradigm by Matching SMEs with Data Centers", 2nd ISO/IEC JTC 1 Study Group on Big Data, Amsterdam, 2014
[11] Youssef Gahi, Mouhcine Guennoun, Hussein T. Mouftah ,” Big Data Analytics: Security and privacy challenges”, IEEE Symposium on Computers and Communication (ISCC), Messina, Italy, June 2016
[12] Firoj Parwej, Nikhat Akhtar, Yusuf Perwej, “A Close-Up View About Spark in Big Data Jurisdiction”, International Journal of Engineering Research and Application (IJERA), ISSN: 2248-9622, Vol. 8, Issue 1, (Part -I1), pp.26-41 January 2018, DOI : 10.9790/9622-0801022641
[13] Yusuf Perwej, “The Ambient Scrutinize of Scheduling Algorithms in Big Data Territory”, International Journal of Advanced Research (IJAR), ISSN 2320-5407, Volume 6, Issue 3, PP 241-258, March 2018, DOI : 10.21474/IJAR01/6672
[14] A.A. Cardenas, P.K. Manadhata, S.P. Rajan, "Big Data Analytics for Security", IEEE Security & Privacy, vol. 11, no. 6, pp. 74-76, 2013
[15] Min Lei, Yixian Yang, Xinxin Niu, Yu Yang, Jie Hao,” An overview of general theory of security”, China Communications, Volume: 14 , Issue: 7,PP 1 – 10, IEEE, July 2017
[16] F. Greitzer, A. Moore, D. Cappelli, D. Andrews, L. Carroll, T. Hull, "Combating the insider cyber threat", IEEE Security Privacy, vol. 6, no. 1, pp. 61-64, Jan./Feb. 2008
[17] M. Clarkson and F. Schneider. Hyper properties. Journal of Computer Security, 18(6):1157-1210, 2010
[18] A. Datta, J. Franklin, D. Garg, L. Jia, and D. Kaynar. On adversary models and compositional security. Security Privacy, IEEE, 9(3):26-32, 2011
[19] Carsten Rudolph , Andreas Fuchs, ”Redefining Security Engineering”, 5th International Conference on New Technologies, Mobility and Security (NTMS), IEEE, Istanbul, Turkey, May 2012
[20] A. Yasinsac ; J. Childs ,” Analyzing Internet security protocols ”, Proceedings Sixth IEEE International Symposium on High Assurance Systems Engineering. Special Topic: Impact of Networking, Boco Raton, FL, USA, USA, Oct. 2001
[21] Yusuf Perwej, Firoj Parwej, Mumdouh Mirghani Mohamed Hassan, Nikhat Akhtar, “The Internet-of-Things (IoT) Security: A Technological Perspective and Review”, International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 5, Issue 1, Pages 462-482, February 2019, DOI: 10.32628/CSEIT195193
[22] Perrig Adrian, Szewczyk Robert, Wen Victor, Culler David, J.D. Tygar, M. Luk, G. Mezzour, A. Perrigo, V. GLigor, "SPINS: Security protocols for sensor networks", Seventh Annual ACM International Conference on Mobile Computing and Networks(MobiCom 2001) July 2001, 2007
[23] R.M. Needham and M. D. Schroeder,” Using encryption for authentication in large networks of computers”, Comm. ACM, Vol.21, No.12, pp. 993-999, 1978
[24] M. Sirbu, J. Chuang, "Distributed authentication in Kerberos using public key cryptography", IEEE Symposium On Network and Distributed System Security (NDSS`97), pp. 134-141, 1997
[25] A. Harbitter, D. Menasce, "Perofrmance of public-key-enabled Kerberos authentication in large networks" in Proceedings of 2001 IEEE Symposium on Security and Privacy, IEEE Computer Society Press, 2001
[26] Lin H., Seh S., Tzeng W., Lin B.P., “Toward Data Confidentiality via Integrating Hybrid Encryption Schemes and Hadoop Distributed FileSystem”, 26th IEEE International Conference on Advanced Information Networking and Applications in 2012
[27] Yusuf Perwej, Bedine Kerim, Mohmed Sirelkhtem Adrees, Osama E. Sheta, “An Empirical Exploration of the Yarn in Big Data”, International Journal of Applied Information Systems (IJAIS), ISSN : 2249-0868 , Foundation of Computer Science FCS, New York, USA Volume 12 , No.9, page 19-29 , December 2017, DOI: 10.5120/ijais2017451730
[28] Yusuf Perwej, Md. Husamuddin, Majzoob K.Omer, Bedine Kerim, “A Comprehend TheApache Flink In Big Data Environments”, IOSR Journal of Computer Engineering (IOSR-JCE), e-ISSN: 2278-0661,p-ISSN: 2278-8727, USA, Volume 20, Issue 1, Ver. IV, PP 48-58, Feb. 2018, DOI : 10.9790/0661-2001044858
[29] Yusuf Perwej, Firoj Parwej, Mumdouh Mirghani Mohamed Hassan, Nikhat Akhtar, “The Internet-of-Things (IoT) Security: A Technological Perspective and Review”, International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT), ISSN : 2456-3307, Volume 5, Issue 1, Pages 462-482, February 2019, DOI: 10.32628/CSEIT195193
[30] Huang Jing, LI Renfa, C. Tang Zhuo, "The Research of the Data Security for Cloud Disk Based on the Hadoop Framework", International Conference on Intelligent Control and Information Processing, June 9–11, 2013
[31] Chao YANG, Weiwei LIN, Mingqi LIU, "A Novel Triple Encryption Scheme for Hadoop-based Cloud Data Security", International Conference on Emerging Intelligent Data and Web Technologies, 2013
[32] Karim Abouelmehdi, Abderrahim Beni-Hssane, Hayat Khaloufi, Mostafa Saadi ,” Big data emerging issues: Hadoop security and privacy ”, 5th International Conference on Multimedia Computing and Systems (ICMCS), IEEE, Marrakech, Morocco, Oct. 2016
[33] J. Whitworth, S. Suthaharan, "Security problems and challenges in machine learning-based Hybrid Big Data processing network systems", ACM SIGMETRICS Performance Evaluation Review, vol. 41, no. 4, pp. 82-85, March 2014
[34] Kai Zheng , Weihua Jiang,” A token authentication solution for hadoop based on kerberos pre-authentication”, International Conference on Data Science and Advanced Analytics (DSAA), IEEE, Shanghai, China, Nov. 2014
[35] B. Saraladevi, N. Pazhaniraja, P. Victer Paul, M.S. SaleemBasha, P. Dhavachelvan, "Big Data and Hadoop-a Study in Security Perspective", Procedia Computer Science, vol. 50, pp. 596-601, 2015, ISSN 1877-0509
[36] J. Xie, S. Yin, et al. "Improving MapReduce performance through data placement in heterogeneous Hadoop clusters", In 2010 IEEE International Symposium on Parallel & Distributed
[37] GitHub, RJ97/Kuber: A Framework for Large Scale Encryption in Hadoop Environment, Mar. 2017
[38] Apache Hadoop 2.7.3 , Transparent Encryption in HDFS, Feb. 2017
[39] Simon Heron, "Advanced Encryption Standard CAES)", Network Security, vol. 2009, no. 12, pp. 8-12, December 2009
[40] P. Mehrotra, J. Djomehri, S. Heistand, R. Hood, H. Jin, A. Lazanoff, S. Saini, R. Biswas, "Performance Evaluation of Amazon EC2 for NASA HPC Applications", Proceedings of the 3rd Workshop on Scientific Cloud Computing, 2012
[41] Charles Schmitt, "Security and Privacy in the Era of Big Data" in RENCI (Renaissance Computing Institute), NCDS, White Paper, 2013
[42] Shuyu Li, Tao Zhang, Jerry Gao, Younghee Park, "A Sticky Policy Framework for Big Data Security", 2015 IEEE First International Conference on Big Data Computing Services and Application, pp. 71, 2015, ISBN 978-1-4799-8128-1/15
[43] J. Kohl, C. Neuman, "The Kerberos Network Authentication Service (V5)", Rfc, pp. 1510, September 1993
[44] S. M. Bellovin, M. Merritt, "Limitations of the kerberos authenication system", Computer Commun. Rev., vol. 20, no. 5, pp. 119-132, Oct. 1990
[45] J. T. Kohl, B. C. Neuman, T. Y. T`so, The evolution of the Kerberos authentication system. Distributed Open Systems, IEEE Computer Society Press, pp. 78-94, 1994
[46] C. Neuman, T. Yu, S. Hartman, K. Raeburn, "The Kerberos network authentication service (V5)", Network Working Group. Request for Comments: 4120, 2005
[47] F. Butler, I. Cervesato, A. D. Jaggard, A. Scedrov, "A formal analysis of some properties of Kerberos 5 using MSR", University of Pennsylvania Department of Computer & Information Science Philadelphia USA Technical Report MS-CIS-04-04, April 2004
[48] Qin Li, Fan Yang, Huibiao Zhu, Longfei Zhu, "Formal modeling and analyzing Kerberos protocol", IEEE World Congress on Computer Science and Information Engineering (CSIE) 2009
[49] William Stallings, "Cryptography and network security principles and practices" in , Pearson Prentice Hall, pp. 401-419, 2006
[50] A. Boldyreva, V. Kumar, "Provable-security analysis of authenticated encryption in Kerberos", IEEE Symposium on Security and Privacy (SP`07), May 2007
[51] S. Sakane, N. Okabey, K. Kamadaz, H. Esakix, "Applying Kerberos to the communication environment for information appliances", Symposium on Applications and the Internet Workshops (IEEE SAINT-w`03), 2003
[52] Joey Pinto , Pooja Jain , Tapan Kumar ,” Hadoop cluster monitoring and fault analysis in real time ”, International Conference on Recent Advances and Innovations in Engineering (ICRAIE), IEEE, Jaipur, India, Dec. 2016
[53] Kadirvel Selvi, Jeffrey Ho, José Ab Fortes, "Fault management in Map-Reduce through early detection of anomalous nodes", Proceedings of the 10th International Conference on Autonomic Computing (ICAC 13), pp. 235-245, 2013
[54] Hu Peng, Wei Dai, "Enhancing fault tolerance based on Hadoop cluster", International Journal of Database Theory and Application 7, no. 1, pp. 37-48, 2014
[55] Jianxi Yang , Chaoxiao Shen , Yaping Chi , Ping Xu , Wei Sun ,” An extensible Hadoop framework for monitoring performance metrics and events of OpenStack cloud”, IEEE 3rd International Conference on Big Data Analysis (ICBDA), Shanghai, China, March 2018
[56] Bao Rong, Chang Hsiu, Fen Tsai, Zih-Yao Lin, Chi-Ming Chen, "Access Security on Cloud Computing Implemented in Hadoop System", 2011 Fifth International Conference on Genetic and Evolutionary Computing IEEE, pp. 77-80, September 2011
[57] A. D. Birrell, D. J. Nelson, "Implementing Remote Procedure Calls", ACM Transactions on Computer Systems, vol. 2, no. 1, pp. 39-59, Feb. 1984
[58] J. Whitworth, S. Suthaharan, "Security problems and challenges in machine learning-based Hybrid Big Data processing network systems", ACM SIGMETRICS Performance Evaluation Review, vol. 41, no. 4, pp. 82-85, March 2014
[59] C. Yang, W. Lin, M. Liu, "A Novel Triple Encryption Scheme for Hadoop-Based Cloud Data Security", Emerging Intelligent Data and Web Technologies (EIDWT) 2013 Fourth International Conference, pp. 437-442, 2013
[60] J. Daemen, V. Rijmen, The design of Rijndael: AES-the advanced encryption standard, Springer Science & Business Media, 2002
[61] M. Hou, Q. Xu, "Secure and efficient two-party authenticated key agreement protocol from certificate less public key encryption scheme", INC IMS and IDC 2009. NCM`09. Fifth International Joint Conference on. IEEE, pp. 894-897, 2009
[62] Xin Zhou, Xiaofei Tang, "Research and Implementation of RSA Algorithm for Encryption and Decryption", the 6th International Forum on Strategic Technology, pp. 1118-1121, 2011
[63] https://pdfs.semanticscholar.org/a140/3588bbcb75452243bb8f3246dea5d49df4b1.pdf
[64] S. Gueron, "A Memory Encryption Engine Suitable for General Purpose Processors", Cryptology ePrint Archive report 2016/204, 2016
[65] Victor Costan, Srinivas Devadas, Intel sgx explained. Cryptology ePrint Archive Report 2016/086, 2016
[66] Robinson Glen, Narin Attila, Elleman Chris, "Amazon Web Services- Using AWS for Disaster Recovery", White Papers, October 2014
[67] Hamoud Alshammari , Jeongkyu Lee , Hassan Bajwa ,” Evaluate H2Hadoop and Amazon EMR performances by processing MR jobs in text data sets”, IEEE Long Island Systems, Applications and Technology Conference (LISAT), Farmingdale, NY, USA, April 2016
[68] C. Mora et al., "Top ten big data security and privacy challenges", Cloud Security Alliance, 2012
[69] A. Cuzzocrea, "Privacy and security of big data: Current challenges and future research perspectives", Proceedings of the First International Workshop on Privacy and Secuirty of Big Data PSBD `14, pp. 45-47, 2014
[70] M. Jensen, "Challenges of Privacy Protection in Big Data Analytics", Proceedings of the International Congress on Big Data, pp. 235-238, 2013