Skip to content

Example Use Cases and Workflows

There are many different ways to use Jetstream2 for your course or workshop. In this section we will cover a few example workflows that you might end up using in your own setup. We encourage you to take these examples as building blocks or starting points in developing your own course content, which may end up looking very different from what we have laid out here.

Creating a Standard Image

No matter your use case, there is one essential feature that can save you time - images. An image saves the state of an instance, including all software and configuration, and can be used as a starting point when launching new instances. By using images, you will only have to install software and configure your instances once, no matter how many students you have.

Here is an example workflow using images. Say your course goal is for students to use machine learning techniques to determine trends in large datasets.

  1. Before the beginning of the course, you create a sample instance. Your students will need a GPU, so you launch a g3.small.
  2. Installing software is not the focus of the course, so you install all the software and packages your students will need for their work. For example, you create a virtual python environment and install PyTorch (as well as all other necessary packages).
  3. For consistency, you want every student to compute against the same dataset. However, this dataset is several hundred gigabytes, so instead of loading this dataset onto each students’ instance, you create a Manila share to be mounted on every students’ instance.
    • You then mount this Manila share onto your sample instance following these instructions.
  4. You add a folder to the ‘exouser’ home directory with a few example python scripts to help students get started.
  5. Once you are done installing software, mounting Manila shares, and adding any additional content your students may need, you create a snapshot image of your instance.
  6. Now that you have a snapshot image, you or your students can use this image as a source when creating new instances. The new instances will be identical to your sample instance at the time of snapshotting, except with a new randomly generated exouser passphrases. This is extremely useful for saving your students time on configuring their instances and saving your instructors headaches in troubleshooting.

Workshop with Unknown Attendees

Perhaps you are running a workshop but you do not have a list of attendees or have a way of communicating with them ahead of time. Requesting ACCESS accounts from attendees is impossible without spending a significant amount of time, so you instead decide to create instances ahead of the workshop and give them out to the attendees. This comes with some challenges though.

As an example, let’s suppose you are running a science gateways workshop where each attendee learns how to deploy a simple web portal. You expect no more than 70 attendees.

  1. Using the Creating a Standard Image steps above, you create a standard image with some software and packages preinstalled. This could be Python/Django, Javascript/Node, or some other other web development tools.
  2. Following the Deploying Multiple VMs for workshops guide, you use the CACAO interface to launch 72 m3.small instances (one for each attendee, one for yourself, and one as a backup) from the image you created in step 1. Since you don’t know attendee names, you use usernames student01 to student70.
  3. During the workshop, you randomly assign numbers 1 -> 70 to your attendees and hand out their corresponding credentials, which were randomly generated during the process above.

Multi-Instance per Student

Maybe you are teaching a course focused on deployment of a slurm-based cluster. If you don’t have funding to purchase hardware for this purpose, you will likely need to teach the content using virtual slurm clusters running on the cloud instead of on bare-metal clusters.

  1. In this scenario, your students will regularly be building and destroying instances, so on the first day of class you ask each student to create an ACCESS account and send it to you.
  2. You then add each of your students to your allocation. Once added, they will be able to login to your allocation and begin creating resources.
  3. In class you go over how to create and manage resources on Jetstream2 and train your students how to be courteous stewards of their SU utilization.
  4. From this point, your students should be equipped to start creating their own instances and eventually their own Virtual Clusters on Jetstream2.

JupyterHub

JupyterHub is an extremely powerful and user-friendly interface that lets multiple users develop and run scripts, all in their own web browser. This versatility and ease-of-use comes with a a few caveats though. The service has to run somewhere, and setting it up can be challenging. The good news is that Jetstream2 is an ideal place to run your JupyterHub, and our community has created several guides for creating your own JupyterHub, regardless of whether you plan to use it for a course or workshop. Here are some points to consider before starting this process:

  1. JupyterHub has authentication tools built in, including its own user management system, as well as integration with popular OAuth services. This means there is no need for your users to obtain ACCESS accounts if they don’t need access beyond JupyterHub.
  2. You may hear the terms “multi-VM” or “auto-scaling” thrown around when referring to JupyterHub. While extremely powerful, these additions are not needed for the majority of use cases. Unless you have hundreds of users, or expect users to run demanding scripts (such as those requiring GPUs or many parallel CPU cores), you will most likley be fine using a single large instance for your JupyterHub. The Littlest JupyterHub is designed precisely for this purpose.
  3. If you require a more advanced multi-instance JupyterHub setup, we recommend using the Zero-to-JupyterHub Template on Cacao.
  4. If the above options do not provide you with enough flexibility, there is also the option of launching your own JupyterHub. One of our community members has written a wonderful guide about Deploying JupyterHub on Jetstream2 on top of Kubernetes. If you need GPUs for your cluster, we also have documentation on how to Deploy Kubernetes and JupyterHub with GPU support.