Skip to content
Snippets Groups Projects
Commit 24394c0f authored by Adrián Rošinec's avatar Adrián Rošinec
Browse files

Merge branch 'feature/upd_cloud_doc' into 'master' 

add site specific docs, replace Metacentrum cloud with e-INFRA CZ cloud

See merge request !30
parents 95752943 5eb4f18c
Branches
No related tags found
1 merge request!30add site specific docs, replace Metacentrum cloud with e-INFRA CZ cloud
Pipeline #248906 failed
Stage: prepare
Stage: test
Stage: build_img
Stage: build
Stage: deploy
Stage: after_test
Hide whitespace changes
Inline Side-by-side
Showing
with 356 additions and 353 deletions
topics/compute/concepts/docs/index.md
+ 1
− 1
View file @ 24394c0f
......@@ -46,7 +46,7 @@ e-INFRA CZ provides a wide range of computational services for the scientific co
## How to Choose Between Computing Services?
- [Read computing service overview](./concepts/)
- Read computing service overview
- Check comparision between computing services _(TODO)_
## See How Different Scientific Use-Cases Are Implemented.
......
topics/compute/openstack/docs/additional-information/contribute.md deleted 100644 → 0
+ 0
− 64
View file @ 95752943
---
title: Contribute to Documentation
search:
exclude: false
---
# Contribute to Documentation
We use the OpenSource [MkDocs Material](https://github.com/squidfunk/mkdocs-material) project to generate the documentation.
## Requirements
Install [MkDocs Material](https://squidfunk.github.io/mkdocs-material/getting-started/).
## Work-Flow Overview
1. Fork & clone repository
2. Create a branch
3. Commit your changes
4. Push to the branch
5. Create a Merge Request with the content of your branch
### Fork Repository
See [GitLab @ ICS MU](https://gitlab.ics.muni.cz/cloud/documentation/forks/new) for details. This will create your own clone of our repository where you will be able to make changes. Once you are happy with your changes, use GitLab to submit them to our original repository.
### Clone Repository
``` bash
# after creating your own copy of the repository on GitLab
git clone git@gitlab.ics.muni.cz:${GITLAB_USER}/documentation.git
```
### Create New Branch
``` bash
# in `mkdocs-material`
git checkout -b my_change
```
### Make Changes & Run Local Server
``` bash
# in `mkdocs-material`
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
```
> Edits will be shown live in your browser window, no need to restart the server.
### Commit and Push Changes
``` bash
git commit -am "My updates"
git push origin my_change
```
### Submit Changes
Create a *Merge Request* via [GitLab @ ICS MU](https://gitlab.ics.muni.cz/cloud/documentation/merge_requests/new).
## Theme Documentation
Full theme documentation is available on page [squidfunk.github.io/mkdocs-material/](https://squidfunk.github.io/mkdocs-material/).
topics/compute/openstack/docs/additional-information/custom-images.md
+ 2
− 2
View file @ 24394c0f
......@@ -7,8 +7,8 @@ search:
# Custom Images
We don't support uploading personal images by default. MetaCentrum Cloud images are optimized for running in the cloud and we recommend users
customize them instead of building their own images from scratch. If you need to upload a custom image, please contact user support at cloud@metacentrum.cz for appropriate permissions.
We don't support uploading personal images by default. e-INFRA CZ Cloud images are optimized for running in the cloud and we recommend users
customize them instead of building their own images from scratch. If you need to upload a custom image, please contact user support for appropriate permissions.
## Image Upload
......
topics/compute/openstack/docs/additional-information/faq.md
+ 16
− 148
View file @ 24394c0f
......@@ -9,14 +9,18 @@ hide:
# Frequently Asked Questions
Also you can access site specific FAQ:
- [Brno site FAQ](../technical-reference/brno-site/faq.md)
- [Ostrava site FAQ](../technical-reference/ostrava-site/faq.md)
??? "What to expect from the cloud and cloud computing"
[Migration of Legacy Systems to Cloud Computing](https://www.researchgate.net/publication/280154501_Migration_of_Legacy_Systems_to_Cloud_Computing) article gives an overview of what to expect when joining a cloud with a personal legacy application.
## What are the cloud computing benefits?
The most visible [cloud computing](https://en.wikipedia.org/wiki/Cloud_computing) benefits are:
* cost savings
* online access to the cloud resources for everyone authorized
* cloud project scalability (elasticity)
......@@ -25,72 +29,13 @@ hide:
* encouraged cloud project agility
??? "How do I register?"
Follow instructions for registering in [MetaCentrum Cloud](../additional-information/register.md).
??? "Where do I report a problem?"
First, try searching the documentation for an answer to your problem. If that does not yield results, open a
ticket with [cloud@metacentrum.cz](mailto:cloud@metacentrum.cz). When contacting user support, always
include your *username* (upper right corner of the web interface) and *domain* with
active *project* (upper left corner of the web interface) as well as a description of
your problem and/or an error message if available.
??? "What networks can I use to access my instances?"
Personal projects can allocate floating IPs from *public-cesnet-78-128-250-PERSONAL*. Routing is preset for this address pool.
Group projects can currently allocate floating IPs from networks ending with *GROUP* suffix as well as *private-muni-10-16-116*.
Furthermore, IP addresses allocated from *public-muni-147-251-124-GROUP* and *public-muni-147-251-255-GROUP* are released daily, so we encourage
using only *public-cesnet-78-128-251-GROUP* and *private-muni-10-16-116* for group projects.
??? "Issues with network MTU (Docker, kubernetes, custom network overlays)"
OpenStack compute server instances should use 1442 bytes MTU (maximum transmission unit) instead of the standard 1500 bytes MTU. The instance itself can set up the correct MTU with its counterpart via Path MTU Discovery. Docker needs MTU set up explicitly. Refer documentation for setting up 1442 MTU in [Docker](https://docs.docker.com/v17.09/engine/userguide/networking/default_network/custom-docker0/) or [Kubernetes](https://docs.projectcalico.org/v3.5/usage/configuration/mtu) or change the configuration with the steps below.
## Changes in Docker daemon
```sh
# edit docker configuration
sudo vi /etc/docker/daemon.json
# MTU 1442 or lower
{
"mtu": 1442
}
# then restart docker
sudo systemctl restart docker
```
## MTU detection
You can use following bash function to detect end-to-end maximum packet size without packet fragmentation.
```sh
# detect_mtu <host>
# measure end-to-end MTU
function detect_mtu() {
local endpoint_host="$1"
for i_mtu in `seq 1200 20 1500` `seq 1500 50 9000`; do
if ping -M do -s $(( $i_mtu - 28 )) -c 5 "${endpoint_host}" >/dev/null; then
echo "Packets of size ${i_mtu} work as expected"
else
echo "Packets of size ${i_mtu} are blocked by MTU limit on the path to destination host ${endpoint_host}!"
break
fi
done
}
# execute
detect_mtu www.nic.cz
```
??? "Issues with proxy in private networks"
OpenStack instances can either use public or private networks. If you are using a private network and you need to access the internet for updates etc.,
you can use muni proxy server *proxy.ics.muni.cz*. This server only supports HTTP protocol, not HTTPS. To configure it you must also consider what applications
will be using it because they can have their configuration files, where this information must be set. If so, you must find the particular setting and set up there
mentioned proxy server with port 3128. Most applications use the following setting, which can be done by editing file `/etc/environment` where you need to add a line
`http_proxy="http://proxy.ics.muni.cz:3128/"`. And then you must either restart your machine or use the command `source /etc/environment`.
Follow instructions for [registering](../additional-information/register.md).
??? "How many floating IPs does my group project need?"
One floating IP per project should generally suffice. All OpenStack instances are deployed on top of internal OpenStack networks. These internal networks are not by default accessible from outside of OpenStack, but instances on top of the same internal network can communicate with each other.
To access the internet from an instance, or access an instance from the internet, you could allocate floating public IP per instance. Since there are not many public IP addresses available and assigning public IP to every instance is not a security best practice, both in public and private clouds these two concepts are used:
* **internet access is provided by virtual router** - all new OpenStack projects are created with *group-project-network* internal network connected to a virtual router with public IP as a gateway. Every instance created with *group-project-network* can access the internet through NAT provided by its router by default.
* **accessing the instances:**
+ **I need to access instances by myself** - best practice for accessing your instances is creating one server with floating IP called [jump host](https://en.wikipedia.org/wiki/Jump_server) and then access all other instances through this host. Simple setup:
......@@ -99,91 +44,14 @@ hide:
3. Install [sshuttle](https://github.com/sshuttle/sshuttle) on your client.
4. `sshuttle -r root@jump_host_fip 192.168.0.1/24`. All your traffic to the internal OpenStack network *192.168.0.1/24* is now tunneled through the jump host.
+ **I need to serve content (e.g. web service) to other users** - public and private clouds provide LBaaS (Load-Balancer-as-a-Service) service, which proxies users traffic to instances. MetaCentrum Cloud provides this service in experimental mode.
In case, that these options are not suitable for your use case, you can still request multiple floating IPs.
??? "I can't log into OpenStack, how is that possible?"
The most common reason why you can't log into your OpenStack account is that your membership in Metacentrum has expired. To extend your membership in Metacentrum,
you can visit [https://metavo.metacentrum.cz/en/myaccount/prodlouzeni](https://metavo.metacentrum.cz/en/myaccount/prodlouzeni).
In case, that these options are not suitable for your use case, you can still request multiple floating IPs.
??? "Backups"
All the data is protected against disk failures. We are not responsible for any data loss that may occur. For now, we do not provide any means for offsite backups.
All the data is protected against disk failures. We are not responsible for any data loss that may occur. For now, we do not provide any means for offsite backups.
What can I do?
- Use OpenStack Snapshots for local backup.
- Use backup software like Borg or Restic to create an offsite incremental backup.
- Use backup/data storage services provided by MUNI or CESNET (e. g. [https://it.muni.cz/sluzby/zalohovani-bacula](https://it.muni.cz/sluzby/zalohovani-bacula)).
??? "I can't access my cloud VMs. MetaCentrum OpenStack network security protection"
Access to the MetaCentrum cloud is protected by [CSIRT-MU](https://csirt.muni.cz/?lang=en) and [CSIRT-CESNET](https://csirt.cesnet.cz/en/index) security teams.
Some interactions with allocated cloud resources may cause cloud access blockage. This is caused by the fact, that legal SSH access to a new virtual machine (VM) which is being allocated is very similar to a (SSH) brute-force attack.
A newly created VM will respond to SSH connection attempts in different ways as it moves through the setup stages:
* A) VM is booting and network is being established. At this stage, there is no functional connection point, and connection attempts will timeout.
* B) SSH connection is being set. At the start of its lifetime, a VM runs the cloud-init process, which enables SSH authentication with the user's SSH key. A connection is refused, because it can't verify the user.
* C) Connection is finally successfull. All setup processes are finished.
When a (ssh) brute-force attack is attempted, scenario is very similar. Repeated unsuccessful (unauthorized) connections to the VM are made (resulting in connection reset or timeout). Once the attacker passes the right credentials, gets connected and logged.
Therefore, when security systems discover such suspicious series of unsuccessfull connections followed by successful one, they likely block Your IP address to the allocated cloud VMs.
## Best practices for accessing cloud resources without getting blocked
The key practices helping to avoid source IP address blockage are:
* connect to cloud infrastructure via single public facing jump / bastion node (using [sshuttle](https://github.com/sshuttle/sshuttle#readme) or [ssh ProxyJump](https://www.jeffgeerling.com/blog/2022/using-ansible-playbook-ssh-bastion-jump-host) or eventually [ssh ProxyCommand](https://blog.ruanbekker.com/blog/2020/10/26/use-a-ssh-jump-host-with-ansible/))
* use OpenStack API to watch whether VM is ACTIVE
* relax public IP try-connect loop timing
* configure SSH client to [reuse connection for instance with `-o ControlMaster=auto -o ControlPersist=60s`](https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing)
!!! example
As an example, consider a group of virtual machines, where at least one has access to the internet using an IPv4 or IPv6 public address, and they are connected by an internal network (e.g. 10.0.0.0/24).
To access the first VM with a public address `<public-ip-jump>`:
* Wait for the machine to enter ACTIVE state via Openstack API: `openstack server show <openstack-server-id> -f json | jq -r .status`.
* After VM is in ACTIVE state try to open connection to SSH port with timeout of approx. 5 seconds and period of at least 30 seconds.
To access other VMs on the same cloud internal network (once ssh connection to 1st is established):
* The recommended method is to create an SSH VPN using sshuttle with `sshuttle -r user@<public-ip-jump> 10.0.0.0/24`
* Address all internal virtual servers with their internal address (CIDR 10.0.0.0/24) and use the 1st (jump / bastion) machine with the public address as an SSH proxy.
* Follow the same steps to connect – first wait for ACTIVE state and then try a port connection.
## How to check, whether you are blocked
Run the following bash script from the machine, where you believe you got blocked (A), and also from another one located in another IP network segment (B, for instance VM in other cloud):
```sh
# Test Cloud Accessibility for a linux or Windows WSDL 2 environments
# BASH function requires following tools to be installed:
# ip, host tracepath traceroute ping, curl, ncat, timeout, bash
# Execution example: test_cloud_access 178.128.250.99 22
function test_cloud_access() {
local basion_vm_public_ip="$1"
local basion_vm_public_port="${2:-22}"
local cloud_identity_host=${3:-identity.cloud.muni.cz}
local timeout=60
set -x
cmds=("ip a" "ip -4 r l" "ip -6 r l")
for i_cmd in "${cmds[@]}"; do
${i_cmd}; echo "ecode:$?";
done
for i_cmd in host tracepath traceroute ping ; do
timeout --signal=2 ${timeout} ${i_cmd} "${cloud_identity_host}"
echo "ecode:$?"
done
timeout --signal=2 ${timeout} curl -v "https://${cloud_identity_host}"
echo "ecode:$?"
timeout --signal=2 ${timeout} ncat -z "${basion_vm_public_ip}" "${basion_vm_public_port}"
echo "ecode:$?"
set +x
}
```
## How to report network issue and get unblocked
If You are suspecting, that Your virtual machines are blocked, You should contact support by sending an email to the address cloud@metacentrum.cz. To make things easier and resolve the issue faster, it is important to add the outputs of the bash function `test_cloud_access()` above, ran from both VMs (A and B).
- Use backup/data storage services provided by your local it support or CESNET (e. g. on MU [https://it.muni.cz/sluzby/zalohovani-bacula](https://it.muni.cz/sluzby/zalohovani-bacula)).
topics/compute/openstack/docs/additional-information/ip-allocation-policy.md
+ 0
− 30
View file @ 24394c0f
......@@ -15,30 +15,6 @@ If you decide to attach second interface to your VM, you should verify the inter
Don't forget to setup security groups accordingly.
## IPv4 Networking
### IPv4 Personal Floating IPs
Is currently limited to the common internal networks. You can start your machine in network `147-251-115-pers-proj-net` or `78-128-250-pers-proj-net` and allocate floating IP address from pools `public-muni-147-251-115-PERSONAL` or `public-cesnet-78-128-250-PERSONAL` respectively. All VMs need to be connected to the same network. You cannot use virtual routers with personal projects. We encourage users to also use IPv6 addresses for long term use. Unassigned allocated addresses are released daily.
### IPv4 Group Floating IPs
The situation is rather different for group projects. You cannot use the same approach as for personal projects. You should create a virtual network as described on page [Create networking](../how-to-guides/create-networking.md) instead and select one of the pools with `-GROUP` suffix as show on page [Allocating floating IPs](../how-to-guides/allocating-floating-ips.md). Namely:
- `public-cesnet-78-128-251-GROUP`
- `public-cesnet-195-113-167-GROUP`
- `public-muni-147-251-21-GROUP`
- `public-muni-147-251-124-GROUP`
- `public-muni-147-251-255-GROUP`
!!! warning
Addresses that are unassigned for longer than 3 months can be released.
!!! tip
If you use a MUNI account, you can use `private-muni-10-16-116` and log into the network via MUNI VPN or you can set up Proxy networking, which is described on page [Proxy networking](../additional-information/proxy-networking.md).
### Floating IP Conversion
......@@ -55,9 +31,3 @@ To access the internet from an instance, or access an instance from the internet
- __I need to serve content (e.g. web service) to other users__ - public and private clouds provide LBaaS (Load-Balancer-as-a-Service) service, which proxies users traffic to instances. MetaCentrum Cloud provides this service in experimental mode.
In case, that these options are not suitable for your use case, you can still request multiple floating IPs.
## IPv6 Networking
### IPv6 Shared Network
We have prepared an IPv6 prefix `public-muni-v6-432`, which is available for both personal and group projects. The network is available as an attachable network for VMs with no limits. For more information please refer to page [Attaching interface](../how-to-guides/attaching-interface.md).
topics/compute/openstack/docs/additional-information/register.md
+ 2
− 94
View file @ 24394c0f
---
title: "How to Get Access"
search:
exclude: false
......@@ -7,96 +6,5 @@ search:
# How to Get Access
Access to MetaCentrum Cloud is granted to users with active accounts in
one of the following identity federations:
* __EINFRA CESNET__,
* __LIFESCIENCE AAI__`*`.
* __EGI Check-in__`*`,
* __DEEP AAI__`*`.
Users from the Czech academic community should always use the `EINFRA CESNET`
identity provider, unless instructed otherwise by user support.
Identity providers marked with `*` should only be used by international
communities with explicitly negotiated resource allocations.
MetaCentrum Cloud provides the following ways for allocating resources
* __personal project__,
* __group project__.
## Personal Project
A personal project goal is to gain cloud environment knowledge, resource allocation quotas can not be increased.
!!! caution
A personal project resource allocation quotas can not be increased.
Personal projects are available automatically to all users of the Czech
e-infrastructure for science, development, and education.
To register, follow instructions for
[registration in the MetaCentrum VO](https://metavo.metacentrum.cz/en/application/index.html).
Personal projects are intended as a low-barrier entry
into the infrastructure for testing and exploration of features. Any
serious resource usage requires the use of a group project, see below.
The following already established terms and conditions apply
* [Terms and Conditions for Access to the CESNET e-infrastructure](https://www.cesnet.cz/conditions/?lang=en)
* [MetaCentrum End User Statement and Usage Rules](https://www.metacentrum.cz/en/about/rules/index.html)
* [Appreciation Formula / Acknowlegement in Publications](https://wiki.metacentrum.cz/wiki/Usage_rules/Acknowledgement)
* [Account Validity](https://wiki.metacentrum.cz/wiki/Usage_rules/Account)
* [Annual Report](https://wiki.metacentrum.cz/wiki/MetaCentrum_Annual_Report_%E2%88%92_Author_Instructions)
## Group Project
!!! notice
Preferred way to request new <a href="https://cloud.gitlab-pages.ics.muni.cz/documentation/register/#group-project">GROUP</a> project is through this online application form: <a href="https://projects.cloud.muni.cz/">https://projects.cloud.muni.cz/</a>
Group projects are the primary resource allocation unit for MetaCentrum Cloud.
Any user or a group of users requiring a non-trivial amount of resources must
request a group project using [this form](https://projects.cloud.muni.cz/) and provide the following basic information:
* __name of the project__,
* __purpose of the project__,
* __contact information__,
* __amount and type of requested resources__,
* __impact would the service have on unavailability for 1h, 1d, 1w__,
* __estimated length of the project__,
* __access control information__ _[(info)](#get-access-control-information)_.
## Increase Quotas for Existing Group Project
To request quota increase or access to particular [flavor](../technical-reference/flavors.md), please use [this form](https://projects.cloud.muni.cz/).
## Get Access Control Information
__Access control__ is based on information provided by the selected identity federation
and is presented in the form of a VO name and, optionally, a group name. Every user
with active membership in the specified VO/group will have full access to all resources
of the requested group project. Membership is managed with tools provided by the selected
identity federation, in a self-service manner.
If you already have VO/group, and already using services
provided by the __CESNET e-Infrastructure__ e.g. MetaCentrum, Data Care,
or you represent a project internal to __Masaryk University__,
please visit OpenID Connect User Profile according to your federation:
- [EINFRA CESNET](https://login.cesnet.cz/oidc/manage/user/profile)
- [LIFESCIENCE](https://profile.aai.lifescience-ri.eu/profile)
- EGI (contact EGI support)
and provide us with information that you see on the page. That is going to be __access control information__.
If you don't have VO/group or you know nothing about it, please contact the MUNI Identity Management team
to create a new group within the Unified Login service.
In the request, describe that you need a group for accessing MetaCentrum Cloud and provide the following information:
* Project/group name
* Project members
After creating a group, you will be provided with a graphical interface for managing group members within the Perun system.
And you will be able to make further adjustments to the member list on your own.
- Get access to [Brno site](../technical-reference/brno-site/get-access.md)
- Get access to [Ostrava site](../technical-reference/ostrava-site/get-access.md)
topics/compute/openstack/docs/additional-information/terms-of-service.md
+ 2
− 0
View file @ 24394c0f
......@@ -11,6 +11,8 @@ The following documents and rules describe your rights and responsibilities as a
## Documents
- [Terms for accessing e-INFRA CZ services](https://www.e-infra.cz/cs/file/8c17a18b8d2d21bade1fe2b4b0efe81c/525/20210408_Podminky_e-INFRA-CZ_EN.pdf)
- [e-INFRA_CZ Personal data processing](https://www.e-infra.cz/en/personal-data-processing)
- [Terms and conditions for the access to the CESNET e-infrastructure](https://www.cesnet.cz/conditions/?lang=en) ([CZ](https://www.cesnet.cz/podminky/))
- [NGI MetaCentrum - Rules of Use](https://www.metacentrum.cz/en/about/rules/index.html) ([CZ](https://www.metacentrum.cz/cs/about/rules/index.html))
- [Masaryk University Directive No. 10/2017 Use of information technology](https://is.muni.cz/do/mu/Uredni_deska/Predpisy_MU/Masarykova_univerzita/Smernice_MU/SM10-17/102278820/MU_Directive_No._10_2017_-_Use_of_Information_Technology.pdf) ([CZ](https://is.muni.cz/do/mu/Uredni_deska/Predpisy_MU/Masarykova_univerzita/Smernice_MU/SM10-17/102278820/Smernice_MU_c.10_2017_-_Pouzivani_informacnich_technologii__ucinna_od_15.6.2020_.pdf))
......
topics/compute/openstack/docs/additional-information/virtual-networking.md
+ 1
− 1
View file @ 24394c0f
......@@ -7,7 +7,7 @@ search:
# Virtual Networking
MetaCentrum Cloud offers software-defined networking as one of its services. Users can create their own
e-INFRA CZ Cloud offers software-defined networking as one of its services. Users can create their own
networks and subnets, connect them with routers and set up tiered network topologies.
Prerequisites:
......
topics/compute/openstack/docs/how-to-guides/accessing-instances.md
+ 2
− 0
View file @ 24394c0f
......@@ -46,6 +46,8 @@ connect to the VM via SSH.
| Ubuntu | ubuntu |
| Centos | centos |
In more general the default user is the value of *default_user* property of image.
## Problems
If you have problems connecting to the VM, verify correct configuration of
......
topics/compute/openstack/docs/index.md
+ 21
− 10
View file @ 24394c0f
---
title: Documentation structure
title: Overview
search:
exclude: false
hide:
- toc
---
# Documentation
# About e-INFRA CZ IaaS cloud
e-INFRA CZ Cloud consists of 2 sites, 17 computational clusters containing 277 hypervisors with a sum of 8968 cores, 96 GPU cards, and 178 TB RAM in two geographical locations Brno and Ostrava. Special demand applications can utilize our clusters with local SSDs and GPU cards. OpenStack instances, object storage and image storage can leverage more than 1.5 PTB of highly available data storage provided by the CEPH storage system.
This is the MetaCentrum Openstack Cloud documentation. The documentation is structured into four logical parts, namely Getting started, How-to guides, Technical reference and Additional information. In each section you can find information based on specific perspective.
More than 400 users are using the MetaCentrum Cloud platform and more than 130k VMs were started last year.
**Getting started**
[Read more about e-INFRA CZ Infrastructure as a Service][readmore]
This section focuses on tutorials and aims to show step by step how to how to use the infrastructure.
# How to read the documentation
OpenStack Cloud documentation is structured into four logical parts, namely Getting started, How-to guides, Technical reference and Additional information. In each section you can find information based on specific perspective.
**How-to guides**
**Getting started**
This section focuses on tutorials and aims to show step by step how to how to use the infrastructure.
**How-to guides**
Guides aim to show how to solve specific problems.
**Technical reference**
**Technical reference**
This section contains mainly operational information about the e-INFRA CZ Openstack Cloud (definition of flavors, networking policy, OpenStack internals) that could be necessary when deploying an infrastructure in Brno or Ostrava site.
This section contains mainly operational information about the MetaCentrum Openstack Cloud that could be necessary when deploying an infrastructure.
* [Technical details of **Brno** cloud site][tech-brno]
* [Technical details of **Ostrava** cloud site][tech-ost]
**Additional information**
**Additional information**
This section desribes further information and problems that users can approach when using e-INFRA CZ Openstack Cloud.
This section desribes further information and problems that users can approach when using MetaCentrum Openstack Cloud.
[readmore]: https://www.cerit-sc.cz/infrastructure-services/data-processing/cloud-service
[tech-brno]: ./technical-reference/brno-site/
[tech-ost]: ./technical-reference/ostrava-site/
\ No newline at end of file
topics/compute/openstack/docs/technical-reference/brno-site/faq.md 0 → 100644
+ 145
− 0
View file @ 24394c0f
---
title: "Frequently Asked Questions"
search:
exclude: false
hide:
- toc
---
# Frequently Asked Questions
??? "Where do I report a problem?"
First, try searching the documentation for an answer to your problem. If that does not yield results, open a
ticket with [cloud@metacentrum.cz](mailto:cloud@metacentrum.cz). When contacting user support, always
include your *username* (upper right corner of the web interface) and *domain* with
active *project* (upper left corner of the web interface) as well as a description of
your problem and/or an error message if available.
??? "What networks can I use to access my instances?"
Personal projects can allocate floating IPs from *public-cesnet-78-128-250-PERSONAL*. Routing is preset for this address pool.
Group projects can currently allocate floating IPs from networks ending with *GROUP* suffix as well as *private-muni-10-16-116*.
Furthermore, IP addresses allocated from *public-muni-147-251-124-GROUP* and *public-muni-147-251-255-GROUP* are released daily, so we encourage
using only *public-cesnet-78-128-251-GROUP* and *private-muni-10-16-116* for group projects.
??? "Issues with network MTU (Docker, kubernetes, custom network overlays)"
OpenStack compute server instances should use 1442 bytes MTU (maximum transmission unit) instead of the standard 1500 bytes MTU. The instance itself can set up the correct MTU with its counterpart via Path MTU Discovery. Docker needs MTU set up explicitly. Refer documentation for setting up 1442 MTU in [Docker](https://docs.docker.com/v17.09/engine/userguide/networking/default_network/custom-docker0/) or [Kubernetes](https://docs.projectcalico.org/v3.5/usage/configuration/mtu) or change the configuration with the steps below.
## Changes in Docker daemon
```sh
# edit docker configuration
sudo vi /etc/docker/daemon.json
# MTU 1442 or lower
{
"mtu": 1442
}
# then restart docker
sudo systemctl restart docker
```
## MTU detection
You can use following bash function to detect end-to-end maximum packet size without packet fragmentation.
```sh
# detect_mtu <host>
# measure end-to-end MTU
function detect_mtu() {
local endpoint_host="$1"
for i_mtu in `seq 1200 20 1500` `seq 1500 50 9000`; do
if ping -M do -s $(( $i_mtu - 28 )) -c 5 "${endpoint_host}" >/dev/null; then
echo "Packets of size ${i_mtu} work as expected"
else
echo "Packets of size ${i_mtu} are blocked by MTU limit on the path to destination host ${endpoint_host}!"
break
fi
done
}
# execute
detect_mtu www.nic.cz
```
??? "Issues with proxy in private networks"
OpenStack instances can either use public or private networks. If you are using a private network and you need to access the internet for updates etc.,
you can use muni proxy server *proxy.ics.muni.cz*. This server only supports HTTP protocol, not HTTPS. To configure it you must also consider what applications
will be using it because they can have their configuration files, where this information must be set. If so, you must find the particular setting and set up there
mentioned proxy server with port 3128. Most applications use the following setting, which can be done by editing file `/etc/environment` where you need to add a line
`http_proxy="http://proxy.ics.muni.cz:3128/"`. And then you must either restart your machine or use the command `source /etc/environment`.
??? "I can't log into OpenStack, how is that possible?"
The most common reason why you can't log into your OpenStack account is that your membership in Metacentrum has expired. To extend your membership in Metacentrum,
you can visit [https://metavo.metacentrum.cz/en/myaccount/prodlouzeni](https://metavo.metacentrum.cz/en/myaccount/prodlouzeni).
??? "I can't access my cloud VMs. MetaCentrum OpenStack network security protection"
Access to the MetaCentrum cloud is protected by [CSIRT-MU](https://csirt.muni.cz/?lang=en) and [CSIRT-CESNET](https://csirt.cesnet.cz/en/index) security teams.
Some interactions with allocated cloud resources may cause cloud access blockage. This is caused by the fact, that legal SSH access to a new virtual machine (VM) which is being allocated is very similar to a (SSH) brute-force attack.
A newly created VM will respond to SSH connection attempts in different ways as it moves through the setup stages:
* A) VM is booting and network is being established. At this stage, there is no functional connection point, and connection attempts will timeout.
* B) SSH connection is being set. At the start of its lifetime, a VM runs the cloud-init process, which enables SSH authentication with the user's SSH key. A connection is refused, because it can't verify the user. * C) Connection is finally successfull. All setup processes are finished.
When a (ssh) brute-force attack is attempted, scenario is very similar. Repeated unsuccessful (unauthorized) connections to the VM are made (resulting in connection reset or timeout). Once the attacker passes the right credentials, gets connected and logged.
Therefore, when security systems discover such suspicious series of unsuccessfull connections followed by successful one, they likely block Your IP address to the allocated cloud VMs.
## Best practices for accessing cloud resources without getting blocked
The key practices helping to avoid source IP address blockage are:
* connect to cloud infrastructure via single public facing jump / bastion node (using [sshuttle](https://github.com/sshuttle/sshuttle#readme) or [ssh ProxyJump](https://www.jeffgeerling.com/blog/2022/using-ansible-playbook-ssh-bastion-jump-host) or eventually [ssh ProxyCommand](https://blog.ruanbekker.com/blog/2020/10/26/use-a-ssh-jump-host-with-ansible/))
* use OpenStack API to watch whether VM is ACTIVE
* relax public IP try-connect loop timing
* configure SSH client to [reuse connection for instance with `-o ControlMaster=auto -o ControlPersist=60s`](https://en.wikibooks.org/wiki/OpenSSH/Cookbook/Multiplexing)
!!! example
As an example, consider a group of virtual machines, where at least one has access to the internet using an IPv4 or IPv6 public address, and they are connected by an internal network (e.g. 10.0.0.0/24).
To access the first VM with a public address `<public-ip-jump>`:
* Wait for the machine to enter ACTIVE state via Openstack API: `openstack server show <openstack-server-id> -f json | jq -r .status`.
* After VM is in ACTIVE state try to open connection to SSH port with timeout of approx. 5 seconds and period of at least 30 seconds.
To access other VMs on the same cloud internal network (once ssh connection to 1st is established):
* The recommended method is to create an SSH VPN using sshuttle with `sshuttle -r user@<public-ip-jump> 10.0.0.0/24`
* Address all internal virtual servers with their internal address (CIDR 10.0.0.0/24) and use the 1st (jump / bastion) machine with the public address as an SSH proxy.
* Follow the same steps to connect – first wait for ACTIVE state and then try a port connection.
## How to check, whether you are blocked
Run the following bash script from the machine, where you believe you got blocked (A), and also from another one located in another IP network segment (B, for instance VM in other cloud):
```sh
# Test Cloud Accessibility for a linux or Windows WSDL 2 environments
# BASH function requires following tools to be installed:
# ip, host tracepath traceroute ping, curl, ncat, timeout, bash
# Execution example: test_cloud_access 178.128.250.99 22
function test_cloud_access() {
local basion_vm_public_ip="$1"
local basion_vm_public_port="${2:-22}"
local cloud_identity_host=${3:-identity.cloud.muni.cz}
local timeout=60
set -x
cmds=("ip a" "ip -4 r l" "ip -6 r l")
for i_cmd in "${cmds[@]}"; do
${i_cmd}; echo "ecode:$?";
done
for i_cmd in host tracepath traceroute ping ; do
timeout --signal=2 ${timeout} ${i_cmd} "${cloud_identity_host}"
echo "ecode:$?"
done
timeout --signal=2 ${timeout} curl -v "https://${cloud_identity_host}"
echo "ecode:$?"
timeout --signal=2 ${timeout} ncat -z "${basion_vm_public_ip}" "${basion_vm_public_port}"
echo "ecode:$?"
set +x
}
```
## How to report network issue and get unblocked
If You are suspecting, that Your virtual machines are blocked, You should contact support by sending an email to the address cloud@metacentrum.cz. To make things easier and resolve the issue faster, it is important to add the outputs of the bash function `test_cloud_access()` above, ran from both VMs (A and B).
topics/compute/openstack/docs/technical-reference/brno-site/get-access.md 0 → 100644
+ 102
− 0
View file @ 24394c0f
---
title: "How to Get Access"
search:
exclude: false
---
# How to Get Access
Access to MetaCentrum Cloud is granted to users with active accounts in
one of the following identity federations:
* __EINFRA CESNET__,
* __LIFESCIENCE AAI__`*`.
* __EGI Check-in__`*`,
* __DEEP AAI__`*`.
Users from the Czech academic community should always use the `EINFRA CESNET`
identity provider, unless instructed otherwise by user support.
Identity providers marked with `*` should only be used by international
communities with explicitly negotiated resource allocations.
MetaCentrum Cloud provides the following ways for allocating resources
* __personal project__,
* __group project__.
## Personal Project
A personal project goal is to gain cloud environment knowledge, resource allocation quotas can not be increased.
!!! caution
A personal project resource allocation quotas can not be increased.
Personal projects are available automatically to all users of the Czech
e-infrastructure for science, development, and education.
To register, follow instructions for
[registration in the MetaCentrum VO](https://metavo.metacentrum.cz/en/application/index.html).
Personal projects are intended as a low-barrier entry
into the infrastructure for testing and exploration of features. Any
serious resource usage requires the use of a group project, see below.
The following already established terms and conditions apply
* [Terms and Conditions for Access to the CESNET e-infrastructure](https://www.cesnet.cz/conditions/?lang=en)
* [MetaCentrum End User Statement and Usage Rules](https://www.metacentrum.cz/en/about/rules/index.html)
* [Appreciation Formula / Acknowlegement in Publications](https://wiki.metacentrum.cz/wiki/Usage_rules/Acknowledgement)
* [Account Validity](https://wiki.metacentrum.cz/wiki/Usage_rules/Account)
* [Annual Report](https://wiki.metacentrum.cz/wiki/MetaCentrum_Annual_Report_%E2%88%92_Author_Instructions)
## Group Project
!!! notice
Preferred way to request new <a href="https://cloud.gitlab-pages.ics.muni.cz/documentation/register/#group-project">GROUP</a> project is through this online application form: <a href="https://projects.cloud.muni.cz/">https://projects.cloud.muni.cz/</a>
Group projects are the primary resource allocation unit for MetaCentrum Cloud.
Any user or a group of users requiring a non-trivial amount of resources must
request a group project using [this form](https://projects.cloud.muni.cz/) and provide the following basic information:
* __name of the project__,
* __purpose of the project__,
* __contact information__,
* __amount and type of requested resources__,
* __impact would the service have on unavailability for 1h, 1d, 1w__,
* __estimated length of the project__,
* __access control information__ _[(info)](#get-access-control-information)_.
## Increase Quotas for Existing Group Project
To request quota increase or access to particular [flavor](../technical-reference/flavors.md), please use [this form](https://projects.cloud.muni.cz/).
## Get Access Control Information
__Access control__ is based on information provided by the selected identity federation
and is presented in the form of a VO name and, optionally, a group name. Every user
with active membership in the specified VO/group will have full access to all resources
of the requested group project. Membership is managed with tools provided by the selected
identity federation, in a self-service manner.
If you already have VO/group, and already using services
provided by the __CESNET e-Infrastructure__ e.g. MetaCentrum, Data Care,
or you represent a project internal to __Masaryk University__,
please visit OpenID Connect User Profile according to your federation:
- [EINFRA CESNET](https://login.cesnet.cz/oidc/manage/user/profile)
- [LIFESCIENCE](https://profile.aai.lifescience-ri.eu/profile)
- EGI (contact EGI support)
and provide us with information that you see on the page. That is going to be __access control information__.
If you don't have VO/group or you know nothing about it, please contact the MUNI Identity Management team
to create a new group within the Unified Login service.
In the request, describe that you need a group for accessing MetaCentrum Cloud and provide the following information:
* Project/group name
* Project members
After creating a group, you will be provided with a graphical interface for managing group members within the Perun system.
And you will be able to make further adjustments to the member list on your own.
topics/compute/openstack/docs/additional-information/about.md → topics/compute/openstack/docs/technical-reference/brno-site/index.md
+ 12
− 2
View file @ 24394c0f
---
title: About MetaCentrum Cloud
title: About e-INFRA CZ openstack cloud site in Brno
search:
exclude: false
---
......@@ -30,4 +30,14 @@ More than 400 users are using the MetaCentrum Cloud platform and more than 130k
## Deployed Services
The list of deployed services in MetaCentrum Cloud is available in [Technical reference](../technical-reference/openstack-modules.md).
The list of deployed services in MetaCentrum Cloud is available in [Modules overview](./openstack-modules.md).
## Other techical info specific for this site
* [List of flavors](./flavors.md)
* [How to get access](./get-access.md)
* [Info on networking](./networking.md)
* [Support](./get-support.md)
* [Info on quotas](./quota-limits.md)
* [Specific FAQ](./faq.md)
* [GPU computing](gpu-computing.md)
topics/compute/openstack/docs/technical-reference/brno-site/networking.md 0 → 100644
+ 39
− 0
View file @ 24394c0f
---
title: Networking
search:
exclude: false
---
# Networking
## IPv4 Networking
### IPv4 Personal Floating IPs
Is currently limited to the common internal networks. You can start your machine in network `147-251-115-pers-proj-net` or `78-128-250-pers-proj-net` and allocate floating IP address from pools `public-muni-147-251-115-PERSONAL` or `public-cesnet-78-128-250-PERSONAL` respectively. All VMs need to be connected to the same network. You cannot use virtual routers with personal projects. We encourage users to also use IPv6 addresses for long term use. Unassigned allocated addresses are released daily.
### IPv4 Group Floating IPs
The situation is rather different for group projects. You cannot use the same approach as for personal projects. You should create a virtual network as described on page [Create networking](../../how-to-guides/create-networking.md) instead and select one of the pools with `-GROUP` suffix as show on page [Allocating floating IPs](../../how-to-guides/allocating-floating-ips.md). Namely:
- `public-cesnet-78-128-251-GROUP`
- `public-cesnet-195-113-167-GROUP`
- `public-muni-147-251-21-GROUP`
- `public-muni-147-251-124-GROUP`
- `public-muni-147-251-255-GROUP`
!!! warning
Addresses that are unassigned for longer than 3 months can be released.
!!! tip
If you use a MUNI account, you can use `private-muni-10-16-116` and log into the network via MUNI VPN or you can set up Proxy networking, which is described on page [Proxy networking](../../additional-information/proxy-networking.md).
## IPv6 Networking
### IPv6 Shared Network
We have prepared an IPv6 prefix `public-muni-v6-432`, which is available for both personal and group projects. The network is available as an attachable network for VMs with no limits. For more information please refer to page [Attaching interface](../how-to-guides/attaching-interface.md).
topics/compute/openstack/docs/technical-reference/quota-limits.md → topics/compute/openstack/docs/technical-reference/brno-site/quota-limits.md
+ 1
− 1
View file @ 24394c0f
......@@ -7,7 +7,7 @@ search:
# Quota Limits
Quotas are used to specify individual resources for each project. In the following tables you can see the default resources available for each project. If you need to increase these resources, you can contact [support](../technical-reference/get-support.md).
Quotas are used to specify individual resources for each project. In the following tables you can see the default resources available for each project. If you need to increase these resources, you can contact [support](./get-support.md).
## Compute Resources (Nova)
......
topics/compute/openstack/docs/technical-reference/ostrava-site/faq.md 0 → 100644
+ 10
− 0
View file @ 24394c0f
---
title: "Frequently Asked Questions"
search:
exclude: false
hide:
- toc
---
# Frequently Asked Questions
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment