g1-g2-ostack-cloud-migration

e-INFRA CZ G1 to G2 OpenStack cloud workload migration.

This is automation CI pipeline is core of the G1 to G2 migration process. More details of the progress can be found here.

G1 e-INFRA CZ / MetaCentrum OpenStack cloud is available here. G2 Brno e-INFRA CZ OpenStack is available here.

Migration checklist

• Define date and time of the specific project workload migration
• Decide what resources has to be migrated.
• Decide whether migrate workload at once or step by step.
• Are there any data in object store?
• Are there any DNS records on FIPs?
• Do you need to migrate non running VMs?
• Do you need to migrate not-attached openstack volumes? (wording)
• Do you request block of FIPs or random is ok for you?
• To agree on how user gives final migration signoff handshake.

How to launch G1 to G2 migration

1. Go to the pipenines page

2. Launch your own pipeline by clicking button Run Pipeline

3. Click on the next page just click again button Run Pipeline without providing any configuration variables

4. Click on displayed ostack-einfra_cz-project-migration job

5. Provide below listed migration input arguments as pipeline variables and click Run Job

6. Pipeline should finish within 10 - 120 minutes based on number of cloud entities

Input arguments are:

• (required) PROJECT_NAME
  • Common project name in G1 and in G2 OpenStack clouds.
• (required) VALIDATION_SERVER_ID
  • id of any existing keypair in the G1 cloud
• (optional) MIGRATE_EXPLICIT_SERVER_NAMES
  • migrate just part of the source G1 project. You may specify explicitly server names delimited by space or comma. This option can be used for explicit migration of inactive VM servers as those are by default not migrated.
• (optional) MIGRATE_EXPLICIT_VOLUME_NAMES
  • all volumes connected to migrated VM servers are migrated. Unused i.e. not attached volumes are by default skipped. This option allows migration of not-attached volumes.

What are the cloud migration steps?

1. Cloud team or cloud user initiates cloud workload migration
2. All necessary information are gathered (see above checklist)
3. Migration pipeline is configured and launched a. Connects to source (G1) and destination (G2) clouds as privileged user[s] and tests project exists in both clouds b. Privileged users are reconnecting under scope of source and destination project c. Server entities are listed in source and destination cloud d. In main loop migrator iterates over source servers and
   • receives all necessary information
   • stops the source virtualized server
   • transfers all attached volumes from source (G1) to destination (G2) cloud using low-level ceph RBD operations
   • starts back the source virtualized server
   • starts up the destination server in destination cloud from the cloned volumes (so using mapped flavor, network, volumes, keypair, security-groups)
   • associate FIP address if source VM has one e. Configured extra volumes are transfered to destination cloud
4. Cloud user validates workload is properly migrated
5. Source cloud resources are made inactive for 2-4 weeks by cloud team
6. Source cloud resources are archived

Migration limitations, additional notes

• Servers in SHUTOFF are not migrated by default, see MIGRATE_EXPLICIT_SERVER_NAMES option description above.
• Project volumes which are not attached are not migrated by default, see MIGRATE_EXPLICIT_VOLUME_NAMES option description above.
• FIP addresses in source and destination cloud will differ
• cloud resources in destination cloud have prefixed names migrated-*
• Retry of the workload migration is possible when resources are removed in destination cloud.
• As of writing (2024-03-05) ephemeral VM servers cannot be migrated.