University of Heidelberg

Introduction to GPU Accelerated Computing

Winter Semester 23/24 Block Course with Practical Exercises

NEWS: March 14: "Passed" remarks have been communicated during the last days to the exam office (for those who passed ...) . Participants belonging to faculty of mathematics and informatics can request an email certificate by me (which I will send to your exam office). Those who belong to other faculties will soon get a document ("Schein") by email.

The course will be held in the CIP pool of Kirchhoff Institute for Physics, Im Neuenheimer Feld (INF) 227, room KIP 1.401, hybrid (online zoom participation possible); hands-on exercises will be offered in person in KIP 1.401 in the afternoons (except Friday, limited number of places) and fully online.

Informations: Link to registration:
Block Course, February 12-16, 2024;
Begin: Monday, Feb. 12, 10:15; End: Friday Feb. 16, 13:00
Schedule: Morning Lecture, Mon-Fri 10:15-13:00;
Afternoon Hands-on Exercise: Mon - Thu 14:15-17:00; Fri: none
Lunch Break: daily 13:00-14:15

You may use your personal laptop or desktop or the computer terminals in the CIP pool rooms; the online tutorial can only be done using a laptop. In any case in the course we have to connect to our GPU supercomputer (kepler cluster of ARI/ZAH). Please prepare to use ssh, do you know how to do this? Under Linux it is quite easy, but under Windows you may have to install a putty client (see also and ). All numerical experiments will be done on the kepler cluster.

Files: Lecture pdf files for download:
Start Instructions for Lecture and Tutorials (Updated Feb. 13, for Group 3)
Links to presentation slides of morning lectures:
Feb. 12 ; Feb. 13 ; Feb. 14 ; Feb. 16 ;
Nbody Lecture (Feb. 15) ; Special informations to run NBODY exercise (Feb. 15) ;
Description of CUDA histograms (8_histo) from Jason Sanders Book (Feb. 16, password protected)

Links to zoom recordings of morning lectures:
Feb. 12 (~230 MB) (password protected see rocket chat, same as Jason Sanders lecture) ; Feb. 13 (~200 MB) ; Feb. 14 (~180 MB) (recording of about 20-30 minutes missing in the middle, explanation of Jason Sanders p. 41-63, sorry) ; Feb. 15 (~275 MB) ; Feb. 15 (~190 MB) ;

Links to Whiteboard Sketches: Monday, Feb. 12 ; Tuesday, Feb. 13 ; Wednesday, Feb. 14 ; Friday, Feb. 16 ;

During the first days the lecture will closely follow a lecture of Jason Sanders:
XX Lecture Slides Jason Sanders complete
XX : The file is protected; access data can be found in the Rocket Chat and will be given also in the lecture.
In CUDA by Example (Book by Jason Sanders) the lecture material of the first days is explained in depth. The course will go beyond the book in the last days. For successful participation in the course it is NOT necessary to buy or borrow the book. All materials needed will be distributed in the course; the book provides useful additional material only.
Wenmei Hwu Lecture 1: Computational Thinking is a more general lecture on parallel computing and computational thinking; it goes much beyond what is covered in our course:

Summary: We will learn the basic technique to use GPU (graphical processing units, graphics cards) for numerical accelerated computing at the example of CUDA - an extension of C. The course is inspired by Jason Sanders Lecture, but not identical to it.
Lecturer: Prof. Rainer Spurzem; ZAH/ARI, Mönchhofstr. 12-14, 69120 Heidelberg
Tutors: Group: Helena Ren (in person group)
Group: Vahid Amiri (online group)
In person exercises will take place in: CIP Pool Im Neuenheimer Feld INF227 KIP 1.401
Group: Lorena Acuña (hybrid, in person AND online)
Online exercises take place here: URL link to zoom Room ; zoom Meeting ID: 675 9750 7858 ; Passcode: will be shown in the uebungen webserver (see registration link above) and in the rocket chat
Time and Place: Lecture: Monday - Friday, February 12-16, 2024: 10:15 - 13:00;
Exercises: Monday - Thursday, February 12-16, 2024: 14:15 - 17:00; (no exercises Fri afternoon!)
Requirements: Basic knowledge of a higher programming language such as C, C++, Fortran or similar

Topic: This is a five day block course with lectures in the morning and practical hands-on exercises in the afternoon. We will learn the basic technique to use GPU (graphical processing units, graphics cards) for numerical accelerated computing at the example of CUDA - an extension of the C programming language, which is used for the NVIDIA GPU accelerated supercomputer to be used in our course. More general approaches for other systems will be discussed. Concepts of parallel programming are introduced. GPU accelerated parallel computing is a technique, which is now widely used in computational physics and astrophysics. Many supercomputers of EuroHPC Petascale systems use GPU.
Topics: Parallel Computing, GPU Hardware, Elements of CUDA Language, Data Transfer, Vector and Matrix Operations, Simple Application for N-Body Problem.
Pass: To pass the course (it is NOT graded): proof of daily hands-on exercises under your account, homework assignment submitted; you may do everything in a team of up to two students.
Literature: CUDA by Example, of Jason Sanders and Edward Kandrot . Our course is inspired by this book and lectures of the authors, will cover much less details, but go for some topics beyond the content of the book; also we will learn an example astrophysical application.

(Responsible for contents: Rainer Spurzem )
Contact: D. Möricke
to top of page