رایانش با کارایی بالا: الزامات، نیازمندیهای نسلهای آتی و محورهای تحقیقاتی
محورهای موضوعی :احسان آریانیان 1 * , محمدمهدي اثني عشري 2 , فاطمه احسانی بشلی 3 , شقایق سادات حسینی بیان 4 , مسعود ده یادگاری 5 , بهنام صمدی 6
1 - عضو هیئت علمی پژوهشگاه ارتباطات و فناوری اطلاعات
2 - دانشکده مهندسی کامپیوتر، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران
3 - دانشکده مهندسی کامپیوتر، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران
4 - دانشکده مهندسی کامپیوتر، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران
5 - عضو هیئت علمی پژوهشگاه ارتباطات و فناوری اطلاعات
6 - دانشکده مهندسی کامپیوتر، دانشگاه صنعتی خواجه نصیرالدین طوسی، تهران
کلید واژه: رایانش با کارایی بالا, محاسبات Exascale, آینده پژوهی, معماریهای سختافزاری, معماریهای نرمافزاری,
چکیده مقاله :
حرکت فعلی جهان در جهت هرچه توانمندتر کردن سامانههای رایانش با کارایی بالا، نشاندهنده نیاز روزافزون به این فناوری است. بدیهی است که هرچه این نیاز افزایش یابد، این سامانهها نیز لازم است که توانمندتر شوند تا بتوانند فعالیتهای بیشتر و سنگینتری را اجرا نمایند. در یک نگاه کلاننگر، نسلهای آتی رایانش با کارایی بالا در دو دسته کلی قرار میگیرند؛ نسلهای رایانشی مبتنی بر فناوریهای نوظهور نظیر نورومورفیک و کوآنتوم و نسلهای رایانشی مرسوم که به سمت Exascale در حال حرکت هستند. با توجه به اینکه در آینده نزدیک، احتمال عملیاتی شدن کامل و استفاده در مقیاس وسیع از فناوریهای نوظهور پایین است، در این مقاله، تمرکز بر نسلهای رایانشی مرسوم قرار گرفته و سعی شده است الزامات و نیازمندیهای آنها از جنبههای سختافزاری و نرمافزاری مورد بررسی قرار گیرند. همچنین، فناوریهای هوش مصنوعی و رایانش ابری به عنوان موتورهای محرکه رایانش با کارایی بالا در جهان مورد بررسی قرار گرفتهاند تا تاثیر متقابل آنها بر رایانش با کارایی بالا مشخص گردد. در نهایت، موضوعات و محورهای تحقیقاتی در سطح جهان که مورد توجه هستند بررسی و تدوین شده است.
Nowadays, increasing the processing power of supercomputers is a worldwide race. This race, signifies the importance of supercomputers in the current era. They are engines of improving technology in almost all scientific areas, such as computational biology, earth sciences, cosmology, fluid dynamics, and plasma modeling, to name a few. Next generation of supercomputers can be divided into two broad categories: 1) emerging technologies such as neuromorphic and quantum computing and 2) Exascala computing. Emerging technologies will be the future of supercomputing, however, not in a very recent future. Therefore, in this paper, we have focused on Exascale computing, and have tried to provide a comprehensive overview of the main requirements for this technology to be achieved and become available. Requirements have been overviewed from different aspects; hardware, software, artificial intelligence, and cloud computing. In addition, we have attempted to provide a complete taxonomy of hot research topics within this area.
"What is High-Performance Computing?," ed. https://www.netapp.com/data-storage/high-performance-computing/what-is-hpc/, 2021.
[2] "High Performance Computing with Accelerators," ed. https://www.seminarsonly.com/computer%20science/High-Performance-Computing-with-Accelerators.php, 2021.
[3] "HPC IN 2020: COMPUTE ENGINE DIVERSITY GETS REAL," ed. https://www.nextplatform.com/2020/01/13/hpc-in-2020-compute-engine-diversity-gets-real/, 2020.
[4] "HPC PROCESSORS – GREATER PERFORMANCE OVER TRADITIONAL CPUS," ed. https://www.aspsys.com/solutions/hpc-processors/, 2020.
[5] "What is GPU Computing?," ed. https://www.e4company.com/en/2021/02/what-is-gpu-computing/, 2021.
[6] "What Is a GPU?," ed. https://www.intel.com/content/www/us/en/products/docs/processors/what-is-a-gpu.html, 2020.
[7] "AMD Announces World’s Fastest HPC Accelerator for Scientific Research¹," ed. https://www.amd.com/en/press-releases/2020-11-16-amd-announces-world-s-fastest-hpc-accelerator-for-scientific-research, 2020.
[8] "Re-Imagining Codesign," ed. https://www.orau.gov/ASCR-CoDesign/, 2021.
[9] "GRAPHCORE IPU-M-2000," ed. https://www.boston.co.uk/products/ipu-m2000.aspx, 2021.
[10] "An AI acceleration solution for production environments," ed. https://www.hpcwire.com/2021/02/22/massively-scalable-ai-that-seamlessly-works-with-your-existing-infrastructures-power-availability-and-budget/, 2021.
[11] L. Armasu, "Google's Big Chip Unveil For Machine Learning: Tensor Processing Unit With 10x Better Efficiency (Updated)," Tom's Hardware, 2016.
[12] Y. E. Wang, G.-Y. Wei, and D. Brooks, "Benchmarking tpu, gpu, and cpu platforms for deep learning," arXiv preprint arXiv:1907.10701, 2019.
[13] P. Teich, "Tearing apart Google’s TPU 3.0 AI coprocessor," Retrieved June, vol. 12, p. 2018, 2018.
[14] N. P. Jouppi et al., "A domain-specific supercomputer for training deep neural networks," Communications of the ACM, vol. 63, no. 7, pp. 67-78, 2020.
[15] P. G. A. Tekin.Tuncer Durak. Piechurski. Kaliszan. Aylin Sungur. Robertsén, "State-of-the-Art and Trends for Computing and Interconnect Network Solutions for HPC and AI," 2020.
[16] "2020 Roadmap," ed. https://ethernetalliance.org/technology/2020-roadmap/, 2020.
[17] "Terabit Ethernet: The New Hot Trend in Data Centers," ed. https://www.lanner-america.com/blog/terabit-ethernet-new-hot-trend-data-centers/, 2019.
[18] "High-Performance Computing Storage," ed: https://dzone.com/articles/high-performance-computing-storage-hybrid-cloud-pa, 2020.
[19] "Introduction to Lustre," ed. https://wiki.lustre.org/Introduction_to_Lustre, 2017.
[20] D. P. C. Prof. Dr. Theo Ungerer, "Eurolab4HPC Long-Term Vision on High-Performance Computing (2nd Edition)," 2020.
[21] "Introduction to Parallel Computing Tutorial " Livermore Computing Center. https://hpc.llnl.gov/training/tutorials/introduction-parallel-computing-tutorial (accessed 2021).
[22] "MVAPICH: MPI over InfiniBand, Omni-Path, Ethernet/iWARP, and RoCE." Network-Based Computing Laboratory. https://mvapich.cse.ohio-state.edu/ (accessed.
[23] "OpenMP 5.0 API Reference Guide," 2019.
[24] "KOKKOS package." Sandia Corporation. https://lammps.sandia.gov/doc/Speed_kokkos.html (accessed 2021).
[25] C. Kessler and J. Keller, "Models for parallel computing: Review and perspectives," Mitteilungen-Gesellschaft für Informatik eV, Parallel-Algorithmen und Rechnerstrukturen, vol. 24, pp. 13-29, 2007.
[26] "The OmpSs Programming Model." 2021 Programming Models Group BSC. https://pm.bsc.es/ompss (accessed April, 2021).
[27] V. K. Pallipuram, M. Bhuiyan, and M. C. Smith, "A comparative study of GPU programming models and architectures using neural networks," The Journal of Supercomputing, vol. 61, no. 3, pp. 673-718, 2012.
[28] "The OpenCL Specification," 2020. [Online]. Available: https://www.khronos.org/registry/OpenCL/specs/3.0-unified/pdf/OpenCL_API.pdf
[29] "rCUDA v20.07 alpha User’s Guide," 2020.
[30] T. K. S. W. Group, "SYCL 2020 Specification (revision 3)," 2020.
[31] "SkePU Autotunable Multi-Backend Skeleton Programming Framework for Multicore CPU and Multi-GPU Systems." https://skepu.github.io/ (accessed 2021).
[32] J. Szuppe, "Boost. Compute: A parallel computing library for C++ based on OpenCL," in Proceedings of the 4th International Workshop on OpenCL, 2016, pp. 1-39.
[33] "WHAT IS HDF5?" https://support.hdfgroup.org/HDF5/whatishdf5.html (accessed 2021).
[34] "HDF5, the new ICM+ data format." University of Cambridge. https://icmplus.neurosurg.cam.ac.uk/home/icm-features/hdf5-new-icm-data-format/ (accessed 2021).
[35] "PnetCDF." Center Of Ultra-Scale Computing and Information Security. http://cucis.ece.northwestern.edu/projects/PnetCDF/ (accessed 2021).
[36] T. Grass et al., "MUSA: a multi-level simulation approach for next-generation HPC machines," in SC'16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016: IEEE, pp. 526-537.
[37] M. Radulovic, R. Sánchez Verdejo, P. Carpenter, P. Radojković, B. Jacob, and E. Ayguadé, "PROFET: modeling system performance and energy without simulating the CPU," Proceedings of the ACM on Measurement and Analysis of Computing Systems, vol. 3, no. 2, pp. 1-33, 2019.
[38] "Extra-P." https://www.scalasca.org/scalasca/software/extra-p/download.html (accessed 2021).
[39] "Dimemas: predict parallel performance using a single cpu machine." BSC. https://tools.bsc.es/dimemas (accessed 2021).
[40] "Paraver: a flexible performance analysis tool." BSC. https://tools.bsc.es/paraver (accessed 2021).
[41] R. Farber, "AI-HPC is Happening Now," 5082598570, 2017.
[42] M. Savonin. "How to Leverage High Performance Computing (HPC) for AI." (accessed.
[43] V. Lee, "Progress in HPC + AI Convergence High performance Computing ( HPC )."
[44] S. Conway, A. Norton, B. Sorensen, and E. Joseph, "Technology Spotlight The Future of HPC Cloud Computing Sponsored by Google," 2019.
[45] T. Vopham, J. E. Hart, F. Laden, and Y. Y. Chiang, "Emerging trends in geospatial artificial intelligence (geoAI): Potential applications for environmental epidemiology," Environmental Health: A Global Access Science Source, vol. 17, pp. 1-6, 2018, doi: 10.1186/s12940-018-0386-x.
[46] Y. Fan, Z. Lan, T. Childers, P. Rich, W. Allcock, and M. E. Papka, "Deep Reinforcement Agent for Scheduling in HPC," arXiv preprint arXiv:2102.06243, 2021.
[47] J. Bang et al., "HPC Workload Characterization Using Feature Selection and Clustering," presented at the Proceedings of the 3rd International Workshop on Systems and Network Telemetry and Analytics, 2020.
[48] Y. Liu et al., "GPTune: Multitask learning for autotuning exascale applications," presented at the Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP, 2021.
[49] G. Batra, Z. Jacobson, S. Madhav, A. Queirolo, and N. Santhanam, "Artificial-intelligence hardware: New opportunities for semiconductor companies," McKinsey & Company, New York, NY, USA, Tech. Rep, 2018.
[50] A. Rodriguez. "Lowering Numerical Precision to Increase Deep Learning Performance." https://www.intel.com/content/www/us/en/artificial-intelligence/posts/lowering-numerical-precision-increase-deep-learning-performance.html (accessed.
[51] C. Y. A. C. Zhang Kai. "The Development Trends of Six Major Container Technologies in 2021." https://www.alibabacloud.com/blog/the-development-trends-of-six-major-containertechnologies-in-2021_597227 (accessed.
[52] E. Hsu, "Cloud Adoption is Driving HPC Toward Digital R&D," 2021. [Online]. Available: http://www.bigcompute.org/blog/state-of-cloud-hpc
[53] "Google Cloud Announces ‘Pre-Tuned’ HPC VM Images." https://www.hpcwire.com/off-the-wire/google-cloud-announces-pre-tuned-hpc-vmimages (accessed.