Knowledge Base Resources

• HPC University Resources

A comprehensive list of training resources from the HPC University. HPCU is a virtual organization whose primary goal is to provide a cohesive, persistent, and sustainable on-line environment to share educational and training materials for a continuum of high performance computing environments that span desktop computing capabilities to the highest-end of computing facilities offered by HPC centers.

• Workshop Tutorial

This comprehensive workshop is designed to guide participants through the world of cryptography, from foundational concepts to advanced implementations. Starting with the basics of encryption, decryption, and hashing, the workshop discusses real-world applications like SSL, blockchain, and digital signatures. Interactive Python-based coding examples, such as symmetric and asymmetric encryption, will provide hands-on experience. Participants will also learn to identify cryptographic vulnerabilities and perform attacks like length extension. Finally, the workshop also explores future trends such as quantum cryptography and zero-knowledge proofs, providing participants with the knowledge to apply cryptography in securing modern digital systems. Ideal for beginners and intermediate learners alike, this workshop is a step-by-step journey into mastering cryptographic principles and practices.

• Plumed
• WESTPA

Tools and plugins to enhance molecular dynamics sampling

• Using Linux Commands in a Python Script

Learn how to use Linux commands in a python script. Specifically, learn how to use the subprocess and os modules in python to run shell commands (which run Linux commands) in a python script that is run on a cluster.

• MatPlotLib Docs

Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python. It makes analyzing and presenting your data extremely easy and works with Python which many people already know.

• Roadmaps in Cornell Virtual Workshop
• Search for topics

Cornell Virtual Workshop is a comprehensive training resource for high performance computing topics. The Cornell University Center for Advanced Computing (CAC) is a leader in the development and deployment of Web-based training programs. Our Cornell Virtual Workshop learning platform is designed to enhance the computational science skills of researchers, accelerate the adoption of new and emerging technologies, and broaden the participation of underrepresented groups in science and engineering. Over 350,000 unique visitors have accessed Cornell Virtual Workshop training on programming languages, parallel computing, code improvement, and data analysis. The platform supports learning communities around the world, with code examples from national systems such as Frontera, Stampede2, and Jetstream2.

• NCSA HPC Training Moodle Site

Self-paced tutorials on high-end computing topics such as parallel computing, multi-core performance, and performance tools. Other related topics include 'Cybersecurity for End Users' and 'Developing Webinar Training.' Some of the tutorials also offer digital badges. Many of these tutorials were previously offered on CI-Tutor. A list of open access training courses are provided below. Parallel Computing on High-Performance Systems Profiling Python Applications Using an HPC Cluster for Scientific Applications Debugging Serial and Parallel Codes Introduction to MPI Introduction to OpenMP Introduction to Visualization Introduction to Performance Tools Multilevel Parallel Programming Introduction to Multi-core Performance Using the Lustre File System

• https://mc-stan.org/users/documentation/
• https://vasishth.github.io/bayescogsci/book/ch-introstan.html
• https://pystan.readthedocs.io/en/latest/

The Stan language is used to specify a (Bayesian) statistical model with an imperative program calculating the log probability density function. Here are some useful links to start your exploration of this statistical programming language, and a Python interface to Stan.

• Spatial Data Science in the Cloud (Alpine HPC) using Python Webpage

Spatial Data Science is a growing field across a wide range of industries and disciplines. The open-source programming language Python has many libraries that support spatial analysis, but what do you do when your computer is unable to tackle the massive file sizes of high-resolution data and the computing power required in your analysis? There materials have been prepared to teach you spatial data science and how to execute your analysis using a high-performance computer (HPC).

• Automated Machine Learning: Methods, Systems, Challenges

The authoritative book on automated machine learning, which allows practitioners without ML expertise to develop and deploy state-of-the-art machine learning approaches. Describes the background of techniques used in detail, along with tools that are available for free.

• Using MATLAB with Other Programming Languages

MATLAB is a really useful tool for data analysis among other computational work. This tutorial takes you through using MATLAB with other programming languages including C, C++, Fortran, Java, and Python.

• GPU Acceleration in Python

This tutorial explains how to use Python for GPU acceleration with libraries like CuPy, PyOpenCL, and PyCUDA. It shows how these libraries can speed up tasks like array operations and matrix multiplication by using the GPU. Examples include replacing NumPy with CuPy for large datasets and using PyOpenCL or PyCUDA for more control with custom GPU kernels. It focuses on practical steps to integrate GPU acceleration into Python programs.

• Python Tools for Data Science

Python has become a very popular programming language and software ecosystem for work in Data Science, integrating support for data access, data processing, modeling, machine learning, and visualization. In this webinar, we will describe some of the key Python packages that have been developed to support that work, and highlight some of their capabilities. This webinar will also serve as an introduction and overview of topics addressed in two Cornell Virtual Workshop tutorials, available at https://cvw.cac.cornell.edu/pydatasci1 and https://cvw.cac.cornell.edu/pydatasci2

• https://scipy-lectures.org/advanced/mathematical_optimization/

Mathematical optimization deals with the problem of finding numerically minimums or maximums of a functions. This tutorial provides the Python solutions for the optimization problems with examples.

• Learn Regular Expressions with simple, interactive exercises
• An online tool to learn, build, & test Regular Expressions
• An Online tool that lets you enter your own text and regular expressions to see what matches

Regular expressions (sometimes referred to as RegEx) is an incredibly powerful tool that is used to define string patterns for "find" or "find and replace" operations on strings, or for input validation. Regular Expressions are used in search engines, in search and replace dialogs of word processors and text editors, and text-processing Linux utilities such as sed and awk. They are supported in many programming languages, including Python, R, Perl, Java, and others.

• HPCwire

HPCwire is a prominent news and information source for the HPC community. Their website offers articles, analysis, and reports on HPC technologies, applications, and industry trends.

• marimo | a next generation python notebook

Introduction seminar for new reactive python notebook from marimo ambassador.

• Dask Tutorial Github Page
• Video Recording of Tutorial - Part 1
• Video Recording of Tutorial - Part 2

A tutorial on the effective use of Dask on HPC resources. The four-hour tutorial will be split into two sections, with early topics focused on novice Dask users and later topics focused on intermediate usage on HPC and associated best practices. The knowledge areas covered include (but are not limited to): Beginner section High-level collections including dask.array and dask.dataframe Distributed Dask clusters using HPC job schedulers Earth Science data analysis using Dask with Xarray Using the Dask dashboard to understand your computation Intermediate section Optimizing the number of workers and memory allocation Choosing appropriate chunk shapes and sizes for Dask collections Querying resource usage and debugging errors

• Overview
• Cookbook

The Biopython Tutorial and Cookbook website is a dedicated online resource for users in the field of computational biology and bioinformatics. It provides a collection of tutorials and practical examples focused on using the Biopython library. The website offers a series of tutorials that cover various aspects of Biopython, catering to users with different levels of expertise. It also includes code snippets and examples, and common solutions to common challenges in computational biology.

5 Days of recordings of Python data analysis and visualization training.

• Arkansas High Performance Computing Center

This link is a documentary website to use AHPCC.

• Handwritten Digits PyTorch

This tutorial is essentially the "hello world" of image recognition and feed-forward neural network (using PyTorch). Using the MNIST database (filled within images of handwritten digits), the tutorial will instruct how to build a feed-forward neural network that can recognize handwritten digits. A solid understanding of feed-forward and back-propagation is recommended.

• https://lectures.scientific-python.org/

Comprehensive tutorials and lecture notes covering various aspects of scientific computing using Python and Scipy.

• Powerpoint - Cursor and Termius benefits for HPC

These slides provide an introduction on how Termius and Cursor, two new and freemium apps that use AI to perform more efficient work, can be used for faster HPC research.

• Free Online Courses on Kaggle

Kaggle Learn is an accessible, hands-on platform offering free, beginner-friendly courses in data science and machine learning. Designed for learners ranging from novices to aspiring professionals, it provides interactive tutorials that focus on practical skills you can apply immediately. Key Features: Structured Micro-Courses: Each course is concise, typically taking 2–5 hours to complete, and includes interactive coding exercises. Real-World Tools: Courses cover essential tools and libraries like Python, Pandas, Matplotlib, SQL, and TensorFlow. Practice-Oriented Learning: The platform emphasizes learning by doing, allowing you to work directly with code and datasets. No Setup Required: Courses run in your browser using Kaggle Notebooks, eliminating the need for local installations. Community Support: Engage with a global community of learners and experts through discussions and shared notebooks.