Join the Colossal-AI Team for two Tutorials at ISC High Performance 2023

Hello to all HPC and AI enthusiasts out there! We are thrilled to announce that our Colossal-AI team will be presenting two tutorials at the prestigious ISC High Performance 2023, an international conference and exhibition fostering growth and innovation in the global HPC community. This year, the ISC High Performance will take place from May 21 – 25 in Hamburg, Germany.At the conference, we will be presenting two distinct tutorials on Sunday, May 21, 2023.

The first tutorial, "Colossal-AI: Scaling AI Models in Big Model Era", will take place from 9:00 AM to 1:00 PM (Europe/Berlin). It focuses on the rapidly expanding domain of large Transformer-based models. We'll address key questions, such as how to improve training and inference efficiency to reduce costs, how to accommodate larger AI models even with limited resources, and how to make these large models and applications more accessible. Practical examples and hands-on activities will be based on our open-source solution, Colossal-AI, which you can find at https://github.com/hpcaitech/ColossalAI.

In the afternoon, from 2:00 PM to 6:00 PM, we will be presenting the second tutorial titled "Large-Scale Deep Learning Optimization Techniques". This tutorial addresses the challenges associated with the recent surge of extremely large models that, despite their impressive performance, come with exorbitant training costs due to large communication overhead and the sheer volume of computations they execute. We aim to provide a clear overview of the optimizations for large-scale deep learning in terms of model accuracy and efficiency. The tutorial will cover a range of topics including gradient descent optimization, large batch training optimization, the generalization gap in large-batch training, second-order optimization, and the state-of-the-art strategies in addressing communication overhead and reducing memory footprints.

If you're unable to attend the presentation, we will make the slides available for download on our website, and we will share the link to the download page on our social media channels. You could try the tutorials by visiting our Colossal-AI GitHub repository to access the latest version. Feel free to reach out on our Colossal-AI developers Slack channel if you have any technical questions.

ISC High Performance 2023 is a unique platform where a global HPC community of technology providers and users, now including machine learning, data analytics, and quantum computing practitioners, gather to exchange visions, ideas, and knowledge. We hope to see you there and together, let's dare to imagine a better tomorrow. With over 3,000 international attendees expected, it's an opportunity you won't want to miss!

If you would like to connect with our team, we invite you to arrange an appointment through our website at https://www.hpc-ai.tech/contact.