Add integration tests

Every time a new feature is added, we need to test wether everything is still working. We typically create a sandbox, instantiate it and test networking by ping and other commands. We need to automate this manual process. Start with simple scripts and also investigate how Gitlab CI/CD could be used in this process.

mentioned in merge request !8 (merged)

Adding integration tests is a long-running activity. This issue covers only the first integration tests focused on compatibility with KYPO CRP.

changed milestone to %Make Creator compatible with KYPO CRP - first phase

added testing label

mentioned in issue #47 (closed)

@487568 Do you have any idea how to use Gitlab CI/CD for this task?

I looked into it a little and running the tests shouldn't be a problem. Since it's a shell script, setting up the pipeline should only require adding a very simple .gitlab-ci.yml file that will execute it. The test script itself might need a few minor adjustments in how it handles errors and stores output but nothing too complicated.

What I'm not sure about is the Runner needed for the job. Since we need to build several virtual machines for each test we can't run it just anywhere. It would be possible to use one of our computers, which shouldn't be hard to set up either, but then we would need to be able to leave the computer on either always or at least sometimes overnight. The complete tests take a couple of hours to finish so it wouldn't make sense to only be able to run them when someone's here.

Thanks, @487568.

Can we use a virtual server for running VirtualBox? Or do we really need a bare metal server?

Using a virtual server should be possible as long as the server allows nested virtualization.

@487568 I've created a new Ubuntu server in our OpenStack cloud for testing purposes. You can test it there. You can connect it using ssh -i <key_sent_via_Mattermost> ubuntu@78.128.250.221. Be careful when creating new user accounts and enabling password authentication. Weak passwords are compromised within a few minutes.

What is the current status?

mentioned in merge request !16 (merged)

mentioned in commit d90bc6c1

closed with merge request !16 (merged)