Computing Performance Optimization Through Parallelization: Techniques and Evaluation
Article Sidebar
Page Numbers:
418-439
Digital Object Identifier:
10.58578/ijemt.v2i3.4210
Viewed: 740 times
Downloaded: 238 times
LYAS Publisher cordially invites qualified professionals to serve as Editors or Reviewers. Your expertise will make an important contribution to maintaining and strengthening the academic quality of our publications. Interested applicants are kindly requested to complete the application form at the following link: Editors & Reviewers
Please feel free to contact us if you need any further information about the submission process or if you have any additional questions.
Authors:
Taiwo Abdulahi Akintayo1, Neibo Augustine Olobo2, Aregbesola Taobat Atinuke3, Idayat Olaide AbdulKareem4 
Copyright :
Authors retain copyright and grant the journal right of first publication.
Main Article Content
Abstract
Parallelization has become a cornerstone technique for optimizing computing performance, especially in addressing the growing complexity and scale of modern computational tasks. By leveraging concurrent processing capabilities of multi-core processors, GPUs, and distributed systems, parallel computing enables the efficient execution of large-scale problems that would otherwise be computationally prohibitive. This paper explores various parallelization techniques, including data parallelism, task parallelism, pipeline parallelism, and the use of GPUs for massive parallel computations. We also examine the key performance evaluation metrics such as speedup, efficiency, Amdahl’s Law, scalability, and load balancing that are critical in assessing the effectiveness of parallelization strategies. Through case studies in scientific simulations, machine learning, and big data analytics, we demonstrate how these techniques can be applied to real-world problems, offering significant improvements in execution time and resource utilization. The paper concludes by discussing the trade-offs involved in parallel computing and suggesting future avenues for optimizing parallelization methods in the context of evolving hardware and software technologies.
Citation Metrics:
Downloads
Article Details
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
References
Baker, M., Salathé, M., & Knight, J. (2019). Parallel computing and climate modeling: Accelerating predictions for global warming. Environmental Modeling and Software, 112, 133-145. https://doi.org/10.1016/j.envsoft.2018.11.004
Brecht, T., Smith, L., & Zhao, Y. (2020). GPU acceleration in high-performance computing: A review. Journal of Computational Science, 44(5), 329-345. https://doi.org/10.1016/j.jocs.2020.03.002
Brown, T., & Mitchell, D. (2021). Optimizing load balancing for parallel applications. Parallel Computing and Performance, 45(3), 120-133. https://doi.org/10.1016/j.pcp.2021.02.001
Davis, J., Kumar, R., & Allen, S. (2018). A survey of parallel computing models for large-scale simulations. International Journal of Supercomputing, 35(2), 202-219. https://doi.org/10.1007/jssr-2018-0150
Dean, J., & Ghemawat, S. (2004). MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1), 107-113. https://doi.org/10.1145/1327452.1327492
Gustafson, J. L. (1988). Reevaluating Amdahl’s law. Communications of the ACM, 31(5), 532-533. https://doi.org/10.1145/42411.42412
Hennessy, J. L., & Patterson, D. A. (2019). Computer architecture: A quantitative approach (6th ed.). Elsevier
Huang, G., Smith, W. M., & Zhao, X. (2020). Speedup and efficiency in parallel computing: A review. Computing Performance Journal, 29(1), 45-61. https://doi.org/10.1007/cpj.2020.04.03
Huang, X., Wang, Y., & Li, W. (2020). GPU-accelerated deep learning for computational biology: A survey. Computational Biology and Chemistry, 84, 107-119. https://doi.org/10.1016/j.compbiolchem.2020.107319
Jones, A., & Taylor, M. (2020). Parallel computing techniques and their impact on modern computational problems. IEEE Transactions on Parallel and Distributed Systems, 32(10), 2028-2043. https://doi.org/10.1109/TPDS.2020.2964174
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097-1105. https://doi.org/10.1145/3065386
Lee, M., & Kim, S. (2019). Load balancing in parallel computing systems. Journal of Parallel Algorithms, 18(3), 215-228. https://doi.org/10.1109/JPA.2019.2889754
Li, X., He, K., & Zhang, M. (2021). Deep learning acceleration using GPUs: Techniques and tools. International Journal of High-Performance Computing, 36(1), 1-19. https://doi.org/10.1109/IJHPC.2021.3041857
Patel, S., Agarwal, R., & Mehta, P. (2018). Reducing synchronization overhead in parallel computing systems. Parallel Computing and Performance, 43(4), 223-237. https://doi.org/10.1016/j.pcp.2018.04.002
Plimpton, S. (1995). Fast parallel algorithms for short-range molecular dynamics. Journal of Computational Physics, 117(1), 1-19. https://doi.org/10.1006/jcph.1995.1039
Ranganathan, P., & Hennessy, J. L. (2007). Task parallelism and load balancing in multi-core systems. IEEE Transactions on Parallel and Distributed Systems, 18(9), 1445-1457. https://doi.org/10.1109/TPDS.2007.205018
Shvachko, K., Kuang, H., Radia, S., & Chansler, R. (2010). The Hadoop Distributed File System. Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies, 1-10. https://doi.org/10.1109/MSST.2010.5496972
Xu, W., Zhang, L., & Song, Z. (2021). Parallel simulation of fluid dynamics on a high-performance computing cluster. Journal of Computational Physics, 429, 109750. https://doi.org/10.1016/j.jcp.2021.109750
Yuan, H., Wang, S., & Zhang, X. (2019). GPU-accelerated simulations of large-scale physical systems. Computational Physics Communications, 243, 23-34. https://doi.org/10.1016/j.cpc.2019.04.022
Zaharia, M., Chowdhury, M., Das, T., & Franklin, M. J. (2010). Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, 15, 15-28. https://doi.org/10.5555/1855712.1855723
Zhou, J., Liu, H., & Lin, Q. (2019). Advances in data parallelism: Algorithms and frameworks for efficient big data analytics. IEEE Access, 7, 128654-128670. https://doi.org/10.1109/ACCESS.2019.2936173