Course Syllabus

Course Information：

Year-Semester Course Number Course Name Phase Credits Hours Required/Elective Instructor Class Student NO. Withdraw Note

Spring Semester
Year 2025 342306 Parallel Computing 1 3.0 3 ★ Yuan-Sen Yang GCDPE 8 0 土木208

Year-Semester	Course Number	Course Name	Phase	Credits	Hours	Required/Elective	Instructor	Class	Student NO.	Withdraw	Note
Spring Semester Year 2025	342306	Parallel Computing	1	3.0	3	★	Yuan-Sen Yang	GCDPE	8	0	土木208

Course Plan：

Instructor Yuan-Sen Yang　Office Hours

Email ysyang@ntut.edu.tw

Last Updated 2025-01-10 15:01:41

Course Objective This course covers the basic concepts, techniques, and technologies involved in parallel computing as well as programming exercises. Emphasis will be placed on understanding the importance of parallel computing in modern scientific computing and artificial intelligence (AI) applications. The course will introduce students to various parallel programming models and tools, including C/C++ programming, multi-processing in Python, OpenMP (in C) for multi-threading, and GPU programming using CUDA and OpenCL. Web-based programming using JavaScript will be included for user interface of some hands-on exercises. No prior programming experience is required.

Course Schedule W1-2: Introduction to Parallel Computing Overview of Parallel Computing: History, applications, and relevance in modern computing. Importance in Scientific Computing and AI: How parallelism drives advancements in simulation, data analysis, and machine learning. Basic Concepts: Types of parallelism (data parallelism, task parallelism), and understanding the computational model. Performance Metrics: Speedup, efficiency, and Amdahl's Law. Exercise: Parallel procedural design, diagram representation, speed-up/performance analysis of summing N numbers Exercise: Parallel procedural design, diagram representation, speed-up/performance analysis of vector addition Exercise: Parallel procedural design, diagram representation, speed-up/performance analysis of matrix multiplication W3-4: Parallel Computing in C Programming Introduction to C for Parallel Programming: Basics of C, memory management, and multi-core programming. Shared Memory and Multi-threading: Using threads and synchronization in C. Hands-on Exercises: Writing and testing parallel programs in C. W5-6: Web-based Parallel Programming with JavaScript JavaScript and Asynchronous Programming: Event loop, Promises, and asynchronous functions. Web Workers: Introduction to parallelism in web browsers using JavaScript. Hands-on Exercises: Implementing parallelism in web applications using JavaScript. W7-8: Multi-processing in Python Python for Parallel Computing: Introduction to the Python multiprocessing module. Process-based Parallelism: Differences between threading and multiprocessing in Python. Hands-on Exercises: Implementing parallel algorithms using Python for large-scale data processing. W10-11: Module 5: Multi-threading with OpenMP Introduction to OpenMP: Basics of multi-threading in shared memory systems. Parallel Directives: Using OpenMP directives to parallelize loops and sections of code. Optimization Techniques: Strategies to improve parallel performance. Hands-on Exercises: Writing parallel programs with OpenMP in C/C++. W12-13: Introduction to GPU Programming The Basics of GPU Computing: Understanding GPU architecture, SIMD (Single Instruction, Multiple Data) model. CUDA Programming: Introduction to CUDA, writing and optimizing kernel functions. OpenCL Overview: General-purpose GPU programming with OpenCL. Hands-on Exercises: Simple GPU-accelerated programs using CUDA and OpenCL. Module 14-16: Advanced Topics and Future Directions Scalable Parallel Systems: Clusters, grids, and cloud computing. Parallel Algorithms: Techniques and frameworks for designing efficient parallel algorithms. Emerging Technologies in AI: The role of parallel computing in deep learning and big data analytics. Research Trends: Future developments in parallel computing and hardware. W17-18: Final Project Presentations

Evaluation and grading policy Midterm (programming) 30% +/- 5% Proposal presentation 15% +/- 5% Final presentation and report 40% +/- 5% Misc. 15% +/- 5%

Materials All right reserved. Do not reproduce without permission.

Foreign-language textbooks:Yes
Grama, A. (Ed.). (2003). Introduction to parallel computing. Pearson Education. https://hpc.llnl.gov/documentation/tutorials/introduction-parallel-computing-tutorial

The access to curricular consultation 8:00-9:00 AM. Monday to Thursday.
8:00-10:00 AM. Friday.
Reservation through email is required (email:ysyang@ntut.edu.tw)

The course corresponds to the SDGs SDG11：永續城市與社區（Sustainable Cities and Communities）

Does the course incorporate AI

Note
The lecture is regularly video recorded and is available at the NTUT iStudy (北科 i 學園). In case that the lecture has to be carried out online, it will be at https://meet.google.com/yiu-jwqa-ckm

Instructor	Yuan-Sen Yang　Office Hours
Email	ysyang@ntut.edu.tw
Last Updated	2025-01-10 15:01:41
Course Objective	This course covers the basic concepts, techniques, and technologies involved in parallel computing as well as programming exercises. Emphasis will be placed on understanding the importance of parallel computing in modern scientific computing and artificial intelligence (AI) applications. The course will introduce students to various parallel programming models and tools, including C/C++ programming, multi-processing in Python, OpenMP (in C) for multi-threading, and GPU programming using CUDA and OpenCL. Web-based programming using JavaScript will be included for user interface of some hands-on exercises. No prior programming experience is required.
Course Schedule	W1-2: Introduction to Parallel Computing Overview of Parallel Computing: History, applications, and relevance in modern computing. Importance in Scientific Computing and AI: How parallelism drives advancements in simulation, data analysis, and machine learning. Basic Concepts: Types of parallelism (data parallelism, task parallelism), and understanding the computational model. Performance Metrics: Speedup, efficiency, and Amdahl's Law. Exercise: Parallel procedural design, diagram representation, speed-up/performance analysis of summing N numbers Exercise: Parallel procedural design, diagram representation, speed-up/performance analysis of vector addition Exercise: Parallel procedural design, diagram representation, speed-up/performance analysis of matrix multiplication W3-4: Parallel Computing in C Programming Introduction to C for Parallel Programming: Basics of C, memory management, and multi-core programming. Shared Memory and Multi-threading: Using threads and synchronization in C. Hands-on Exercises: Writing and testing parallel programs in C. W5-6: Web-based Parallel Programming with JavaScript JavaScript and Asynchronous Programming: Event loop, Promises, and asynchronous functions. Web Workers: Introduction to parallelism in web browsers using JavaScript. Hands-on Exercises: Implementing parallelism in web applications using JavaScript. W7-8: Multi-processing in Python Python for Parallel Computing: Introduction to the Python multiprocessing module. Process-based Parallelism: Differences between threading and multiprocessing in Python. Hands-on Exercises: Implementing parallel algorithms using Python for large-scale data processing. W10-11: Module 5: Multi-threading with OpenMP Introduction to OpenMP: Basics of multi-threading in shared memory systems. Parallel Directives: Using OpenMP directives to parallelize loops and sections of code. Optimization Techniques: Strategies to improve parallel performance. Hands-on Exercises: Writing parallel programs with OpenMP in C/C++. W12-13: Introduction to GPU Programming The Basics of GPU Computing: Understanding GPU architecture, SIMD (Single Instruction, Multiple Data) model. CUDA Programming: Introduction to CUDA, writing and optimizing kernel functions. OpenCL Overview: General-purpose GPU programming with OpenCL. Hands-on Exercises: Simple GPU-accelerated programs using CUDA and OpenCL. Module 14-16: Advanced Topics and Future Directions Scalable Parallel Systems: Clusters, grids, and cloud computing. Parallel Algorithms: Techniques and frameworks for designing efficient parallel algorithms. Emerging Technologies in AI: The role of parallel computing in deep learning and big data analytics. Research Trends: Future developments in parallel computing and hardware. W17-18: Final Project Presentations
Evaluation and grading policy	Midterm (programming) 30% +/- 5% Proposal presentation 15% +/- 5% Final presentation and report 40% +/- 5% Misc. 15% +/- 5%
Materials	All right reserved. Do not reproduce without permission. Foreign-language textbooks:Yes Grama, A. (Ed.). (2003). Introduction to parallel computing. Pearson Education. https://hpc.llnl.gov/documentation/tutorials/introduction-parallel-computing-tutorial
The access to curricular consultation	8:00-9:00 AM. Monday to Thursday. 8:00-10:00 AM. Friday. Reservation through email is required (email:ysyang@ntut.edu.tw)
The course corresponds to the SDGs	SDG11：永續城市與社區（Sustainable Cities and Communities）
Does the course incorporate AI
Note	The lecture is regularly video recorded and is available at the NTUT iStudy (北科 i 學園). In case that the lecture has to be carried out online, it will be at https://meet.google.com/yiu-jwqa-ckm