Merge branch 'blocked-cloud-access-faq' into 'master'

Add blocked access part

See merge request !112

content/cloud/faq/index.md

@@ -111,4 +111,65 @@ All the data is protected against disk failures. We are not responsible for any
 What can I do?
 - Use OpenStack Snapshots for local backup.
 - Use backup software like Borg or Restic to create an offsite incremental backup.
-- Use backup/data storage services provided by MUNI or CESNET (e. g. https://it.muni.cz/sluzby/zalohovani-bacula ).
\ No newline at end of file
+- Use backup/data storage services provided by MUNI or CESNET (e. g. https://it.muni.cz/sluzby/zalohovani-bacula ).
+
+## I can't access my cloud VMs, MetaCentrum OpenStack network security protection
+Access to the MetaCentrum cloud is protected by security teams [CSIRT-MU](https://csirt.muni.cz/?lang=en) and [CSIRT-CESNET](https://csirt.cesnet.cz/en/index).
+
+Some interactions with virtual servers can block access to the allocated cloud resources. This is caused by the fact, that legal SSH access to a virtual machine (VM) can be similar to a brute-force attack.
+
+A newly created VM will respond to SSH connection attempts in different ways as it moves through the setup stages:
+* VM is booting and network is being established. At this stage, there is no functional connection point, and attempts will timeout.
+* SSH connection is being set. At the start of its lifetime, a VM runs the cloud-init process, which enables SSH authentication with the user's SSH key. A connection is refused, because it can't verify the user.
+* Connection is finally successfull. All setup processes are finished.
+
+When a brute-force attack is attempted, repeated unsuccessful (unauthorized) connections to the VM are made, and either timeout or refused are returned. If the attacker guesses the right credentials, he/she will eventually get connect passed.
+
+Therefore, when security systems discover a suspicious series of unsuccessfull connections, they block Your machines. This can also happen, if You were the one who initiated them by mistake.
+
+### Best practices for accessing cloud resources without getting blocked
+As an example, consider a group of virtual machines, where at least one has access to the internet using an IPv4 or IPv6 public address, and they are connected by an internal network (e.g. 10.0.0.0/24).
+
+To access the VM with a public address:
+* Wait for the machine to enter ACTIVE state via Openstack API: `openstack server show <id> -f json | jq -r .status`
+* After VM is in ACTIVE state try to open connection to SSH (or other) port with timeout of approx 5 seconds and period of 30 seconds
+
+To access other VMs on the same cloud internal network (once ssh connection to 1st is established):
+* The recommended method is to create an SSH VPN using sshuttle with `sshuttle -r user@<public-ip> 10.0.0.0/24`
+* Address all internal virtual servers with their internal address (CIDR 10.0.0.0/24) and use the machine with the public address as an entry point for traffic
+* Use the steps above to connect – wait for ACTIVE and try a port connection
+
+### How to check, whether cloud resources are blocked
+Run the following bash script from the machine, where you believe you got blocked (A), and also from another one located in another IP network segment (B, for instance VM in other cloud):
+
+```sh
+# Test Cloud Accessibility
+#
+# Execution example: test_cloud_access 178.128.250.99 22
+
+function test_cloud_access() {
+  local basion_vm_public_ip="$1"
+  local basion_vm_public_port="${2:-22}"
+  local cloud_identity_host=${3:-identity.cloud.muni.cz}
+  local timeout=60
+  set -x
+  cmds=("ip a" "ip -4 r l" "ip -6 r l")
+  for i_cmd in "${cmds[@]}"; do
+    ${i_cmd}; echo "ecode:$?";
+  done
+  for i_cmd in host tracepath traceroute ping ; do
+    timeout --signal=2 ${timeout} ${i_cmd} "${cloud_identity_host}"
+    echo "ecode:$?"
+  done
+  timeout --signal=2 ${timeout} curl -v "https://${cloud_identity_host}"
+  echo "ecode:$?"
+  timeout --signal=2 ${timeout} ncat -z "${basion_vm_public_ip}" "${basion_vm_public_port}"
+  echo "ecode:$?"
+  set +x
+}
+```
+
+### How to report issue and get unblocked
+If You are suspecting, that Your virtual machines are blocked, You should contact support by sending an email to the address cloud@metacentrum.cz. To make things easier and resolve the issue faster, it helps to state Your assumption about the blocked access and what led to it.
+
+Also, it is important to add the outputs of the bash function `test_cloud_access()` above, ran from both VMs (A and B).