How to Migrate¶
In order to migrate from Jetstream1 (JS1) to Jetstream2 (JS2) you’ll need to understand:
Just as in JS1, you’ll need an allocation of compute (
storage resources, awarded by NSF and currently managed by XSEDE.
The allocation process is described here: Allocation Overivew
You can take one of two approaches:
New allocations vs renewals
By XSEDE policy, a PI may only have one startup allocation. If you have a valid allocation, consider renewing your startup, education, or champion allocation rather than trying to get a new allocation. Research allocations may be extended within reason but extensions are no substitute for the renewal process in general.
Supplement Allocation ¶
If you already have a JS1 allocation, then we encourage you to request a supplement at a STARTUP amount of resources for the
storage resources on JS2. This will enable you to create the Code, Performance, & Scaling estimates you’ll need for a successful project renewal on JS2.
If your startup or champion allocation is set to expire in June or July and is a NOT a research allocation AND is below the normal startup limits on JS2, you may also choose to renew it instead. Please ONLY look at extending allocations if you have a research allocation that won’t be able to be reviewed at XRAC prior to Jetstream1 going offline.
Education allocations may also be renewed without restriction as long as there is proper justification.
New Allocation ¶
Of course, if your research will be changing, you can always apply for a new allocation of resources on JS2. We recommend you still start with a Startup level in order to create the Code, Performance, & Scaling estimates you’ll need later for a Research allocation.
How to move your data ¶
If you’ve already been using JS1, you’ll likely wish to retain your old VMs and data, found within both VMs and in external volumes. Below, we provide instructions for moving your VMs. As time permits, we anticipate the development of tools to help facilitate and partially automate migration. Volumes on JS1 do not persist on JS2 because they are completely separate systems and do not share storage.
Data volumes and VM transfers
As we note below, it is highly preferable to recreate resources on Jetstream2. Volume data, especially larger volumes, may be difficult to transfer easily. The Jetstream team can help with volume transfers and will discuss potentially helping with VM snapshots/transfers, though we highly encourage recreating VMs on Jetstream2. Please use the contact form and include your allocation number, resource names and UUIDs that you wish to discuss migrating.
There are essentially three approaches to accomplish data retention:
Recreate your work (STAFF RECOMMENDED) In order to get you going the fastest, take advantage of all the new features of JS2, and avoid any legacy configuration differences, it’s often advisable to simply create new VMs and bring in fresh software and data.
Network configurations and any instance management tools and scripts you’ve used previously will likely require updating to current values appropriate for JS2.
Copying data from JS1 to JS2, particularly from within the same regional provider, will generally have good performance relative to a transfers across the internet.
Transfer your work You can create snapshots of your existing JS1 VMs and request the Help Desk Support team copy these snapshots as well as data volumes to Jetstream2.
Please be aware that configurational differences between JS1 and JS2 generally prevent straight forward re-deployment of a JS1 VM on JS2. It may be more advisable to transfer the VM and mount the snapshot as a external volume on a new JS2 VM.
Web shell and desktop will not work nor will the software repository if you are transferring an Atmosphere instance. If you want these features, you will need to launch a new instance under Exosphere on JS2 and rebuild your VM there.
Do you want the value added features of Exosphere and Jetstream2 ?
Or do you want your exact instance/image from Atmosphere on JS1.
You cannot easily have both with an Atmosphere-migrated instance.
How to preserve Jetstream1 VMs and data ¶
- Identify if your VM or volume used the Atmosphere or API/CLI interface
- STAFF RECOMMENDED but not required:: Contact staff and request a staff ssh-key to add to your VM to more easily allow staff to prepare the image for transfer.
- We recommand using a running instance for the migration. NOTE: It is strongly recommended that users upgrade the operating system before imaging. For CentOS based systems, it’s
sudo yum update
For Ubuntu based systems, do
sudo apt-get updateand then
sudo apt-get upgrade. VMs that are too far out of date may be in violation of Jetstream/Jetstream2 policies.
- For the instance you wish to migrate, do the following:
- Go to the instance that you wish to migrate in Atmosphere
- Click on COPY for the Alias (UUID) of the image
- Submit a ticket with your TG-XXXXXXXX allocation number, USERNAME, VM PROJECT FOLDER, and VM UUID to firstname.lastname@example.org to ask staff to copy your instance to Jetstream2.
- Click on the PROJECTS tab
- Click on the Project Folder
- Scroll down to VOLUMES and click on the desired volume
- Click on COPY to grab the UUID of the volume.
- Submit a ticket with your TG-XXXXXXXX allocation number, USERNAME, and volume UUID to email@example.com to ask staff to copy your volume to Jetstream2.
- STAFF RECOMMENDED but not required: Contact staff and request a staff ssh-key to add to your VM to more easily allow staff to prepare the image for transfer.
- Follow the instructions here to create a snapshot of your instance
- Make note of the VM UUID
- Submit a ticket with• your XSEDE::USERNAME, & PROJECT CODE (TG-#####),• VM name, & UUID/Alias• any related Volume name, size, & UUID/Aliasto firstname.lastname@example.org to ask staff to copy your image to Jetstream2.
- Use the command:
openstack volume list
- Note the Volume UUID
- Submit a ticket with• your XSEDE::USERNAME, & PROJECT CODE (TG-#####),• Volume name, size, & UUID/Alias• any related VM name, & UUID/Aliasto email@example.com to ask staff to copy your image to Jetstream2.
- Use the command:
Where to find help documentation ¶
JS1:Documentation for JS1 will temporarily remain during the summer of 2022 at https://wiki.jetstream-cloud.org
This documentation will likely NOT be updated.
JS2:The latest JS2 documentation will be maintained at docs.jetsteam-cloud.org
NEWS: Announcements will be made at the XSEDE User News : https://www.xsede.org/news/user-news site To ensure that you are receiving all updates to your inbox, login to the page to manage your XSEDE User News subscriptions. All Jetstream users should be added when they are placed on an allocation, but you may wish to verify so that you don’t miss any important migration updates.
Updates will all be made Jetstream2 System Status and Information and shared via XSEDE User News : https://www.xsede.org/news/user-news site .