Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • fix/migration_root_eph_plus_ceph
  • freznicek-test-branch
  • v1.10.1
  • v1.9.13
  • v1.9.11
  • v1.9.10
  • v1.9.9
  • v1.9.8
  • v1.9.7
  • v1.9.6
  • v1.9.5
  • v1.9.4
  • v1.9.3
  • v1.9.2
  • v1.9.1
  • v1.9.0
  • v1.8.1
  • v1.7.2
  • v1.7.1
  • v1.7.0
  • v1.6.1
  • v1.5.2
23 results
Compare History
František Řezníček's avatar
fix: Handle situation when source server has no security-groups linked
František Řezníček authored 
This patch fixes following problem:
2024-07-02 12:29:49,117 project-migrator INFO F.9 Destination OpenStack server keypair exists (vscht)
Traceback (most recent call last):
  File "/home/freznicek/prjs/muni/sources/gitlab.ics.muni.cz/g1-g2-ostack-cloud-migration.git/ci/./project-migrator.py", line 389, in <module>
    sys.exit(main(ARGS))
             ^^^^^^^^^^
  File "/home/freznicek/prjs/muni/sources/gitlab.ics.muni.cz/g1-g2-ostack-cloud-migration.git/ci/./project-migrator.py", line 190, in main
    olib.get_or_create_dst_server_security_groups(args,
  File "/home/freznicek/prjs/muni/sources/gitlab.ics.muni.cz/g1-g2-ostack-cloud-migration.git/ci/olib.py", line 452, in get_or_create_dst_server_security_groups
    for i_src_server_security_group_name in {i_sg['name'] for i_sg in src_server.security_groups}:
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable
[freznicek@lenovo-t14 ci 1]$
2eda43f4
Name Last commit Last update
ci
images
.gitlab-ci.yml
CHANGELOG.md
README.md
README.md

g1-g2-ostack-cloud-migration

e-INFRA CZ G1 to G2 OpenStack cloud workload migration.

This is automation CI pipeline is core of the G1 to G2 migration process. More details of the progress can be found here.

G1 e-INFRA CZ / MetaCentrum OpenStack cloud is available here. G2 Brno e-INFRA CZ OpenStack is available here.

Migration checklist

  • Describe to user migration steps and get acknowledge that source servers may be stopped. Are there SLAs?
  • Define date and time of the specific project workload migration
  • Define who is contack person (AAI group manager) and/or someone else
  • (group project only) Define expiration of the project
  • Decide what resources has to be migrated.
  • Decide whether migrate workload at once or step by step.
  • Are there any data in object store?
  • Are there any DNS records on FIPs?
  • Do you need to migrate non running VMs?
  • Do you need to migrate not-attached openstack volumes? (wording)
  • Do you request block of FIPs or random is ok for you?
  • To agree on how user gives final migration signoff handshake.

How to launch G1 to G2 migration

  1. Go to the pipenines page

  2. Launch your own pipeline by clicking button Run Pipeline

  3. Click on the next page just click again button Run Pipeline without providing any configuration variables

  4. Click on displayed ostack-einfra_cz-project-migration job

  5. Provide below listed migration input arguments as pipeline variables and click Run Job

  6. Pipeline should finish within 10 - 120 minutes based on number of cloud entities

Input arguments are:

  • (required) PROJECT_NAME
    • Common project name in G1 and in G2 OpenStack clouds.
  • (required) VALIDATION_SERVER_ID
    • id of any existing server in the G1 cloud
  • (optional) MIGRATE_EXPLICIT_SERVER_NAMES
    • migrate just part of the source G1 project. You may specify explicitly server names delimited by space or comma. This option can be used for explicit migration of inactive VM servers as those are by default not migrated.
  • (optional) MIGRATE_EXPLICIT_VOLUME_NAMES
    • all volumes connected to migrated VM servers are migrated. Unused i.e. not attached volumes are by default skipped. This option allows migration of not-attached volumes.
  • (optional) PROJECT_MIGRATOR_EXTRA_ARGS
    • the expert mode arguments

What are the cloud migration steps?

  1. Cloud team or cloud user initiates cloud workload migration
  2. All necessary information are gathered (see above checklist)
  3. Migration pipeline is configured and launched a. Connects to source (G1) and destination (G2) clouds as privileged user[s] and tests project exists in both clouds b. Privileged users are reconnecting under scope of source and destination project c. Server entities are listed in source and destination cloud d. In main loop migrator iterates over source servers and
    • receives all necessary information
    • stops the source virtualized server
    • transfers all attached volumes from source (G1) to destination (G2) cloud using low-level ceph RBD operations
    • starts back the source virtualized server
    • starts up the destination server in destination cloud from the cloned volumes (so using mapped flavor, network, volumes, keypair, security-groups)
    • associate FIP address if source VM has one e. Configured extra volumes are transfered to destination cloud
  4. Cloud user validates workload is properly migrated
  5. Source cloud resources are made inactive for 2-4 weeks by cloud team
  6. Source cloud resources are archived

Migration limitations, additional notes

  • Servers in SHUTOFF are not migrated by default, see MIGRATE_EXPLICIT_SERVER_NAMES option description above.
  • Project volumes which are not attached are not migrated by default, see MIGRATE_EXPLICIT_VOLUME_NAMES option description above.
  • FIP addresses in source and destination cloud will differ
  • cloud resources in destination cloud have prefixed names migrated-*
  • Retry of the workload migration is possible when resources are removed in destination cloud.
  • As of writing (2024-03-05) ephemeral VM servers cannot be migrated.

Tools needed for manual execution