Starting sandbox-galaxy-docker

- ToDo: Upozorneni na naroky na HW - budou se stahovat desitky GB, vypocentni narocnost -> disk, RAM, CPU

1. Clone source repository

   git clone https://gitlab.ics.muni.cz/europdx/sandbox/sandbox-galaxy-docker.git
   cd sandbox-galaxy-docker

2. Set variable HOST in .env file to FQDN name of your server and ADMIN_PASSWORD to your preferred password.

- ToDo: Zminit tam taky, ze je dobre si nastavit REFERNECE_DATA na nejaky velky disk

3. SSL certificates (optional) - There is self-signed SSL certificate already in the repository to make the the start as easy as possible. You can also use you own SSL cerfificate signed by your favourite certificate authority, if you already have one. Just switch to folder proxy/ssl and replace self-signed key pair in files fullchain.pem and privkey.pem with SSL key pair created for FQDN of your machine. If you prefere not to use self-signed SSL certificate and do not have SSL certificate for your machine, the Let's encrypt SSL certificate can be created. There is a script prepared in swag folder for this case. Just follow this steps:

   1. Switch to swag directory

      cd swag

   2. Launch ./get_certs.sh script

      sudo ./get_certs.sh

   If everything goes well, new key pair and certificate is created and installed in /proxy/ssl directory. Otherwise self-signed SSL certificate is used.

4. Prepare Sandbox's internal data files

   ./prepare_data.sh

5. Copy NGS data to galaxy/import or change IMPORT_DIR in .env If you want to test the import, you can download the samples on the http://download.edirex.ics.muni.cz/data/test-data/test/ into the ./galaxy/import directory and keep the default settings.

6. Start whole environment

   docker-compose up -d

7. Go to https://hostname_or_IP_address:8082 and enter username + password (set in step 2.)

8. Click on "Login or Register" from top panel and log in to Galaxy app with username admin and password password

9. Import your data

If you do not wish to use your own data, skipt this step and download the test data according the step 10.

Select Shared Data / Data Libraries from top panel.

Select + Library and give the new library a name and description, and click on the save icon.

Click on the name of the newly created library and then select + Datasets from import directory.

Select your datasets and import.

Select your datasets in the library and Export to History as Datasets or as a collection. If your data are homogenous, eg. single-end or paired-end Fastq files, it is more useful to use a collection as the workflows will consume the entire collection and process all of the samples simultaneously.

If you wish to process Pair-end data, you should select List of Pairs as the type of the collection. In the next step you will see this window, where you have to specify the regex by which the forward and reverse files are distinguished. The default is _1 and _2.

After specifying the name and creating the collection, the datasets will appear in the selected / newly created History.

10. Download the reference data and indexes

Select the QUICK START tool from the Prepare Reference Data section and select which indexes you want to download.

Optional but recommended: select also the Testing data, as it also downloads the auxiliary inputs needed to run the workflow.

- ToDo:
- jak vyplnit metadata, sampleplatformdata, diagnosis_mapping (testovaci data lze vzit tak jak jsou)
- downgrade/upgrade galaxy, aby tam nebyla chyba "Conda dependency seemingly installed but failed to build job environment."

To run the workflow, select the Workflow option from the top panel, and then select the workflow you wish to run.

- ToDo: 
- pridat zdroj informacii ako vyplnit Metadata.

More info on how to fill the template metadata files ?here?.

Re-starting sandbox-galaxy-docker

- ToDo: Do budoucna asi promylset, jak to udelat, aby po restartu Galaxy tam zustaly data

1. Stop sandbox-galaxy-docker

   docker-compose down
   docker volume prune

2. Prepare data files

   ./delete_previous_data.sh
   ./prepare_data.sh

3. Start whole environment

   docker-compose up -d

- DS: po krocich 1.-3. je API ve stavu down, musim dat /api/reset aby bylo running

App endpoints

• Cbioportal: https://*hostname_or_IP_address*:8080
• Galaxy: https://*hostname_or_IP_address*:8082
• Dataportal: https://*hostname_or_IP_address*
• Datahub API: https://*hostname_or_IP_address*:5000
• Sandbox API: http://*hostname_or_IP_address*:5001
• Backend access: https://*hostname_or_IP_address*:8085
  • ?? chce to login / heslo

SSL certificates (optional)

There are self-signed SSL key and certificate in the repository to make the the start as easy as possible. You can generate SSL key pair for your specific host name a put it in proxy/ssl and replace self-signed pair in files fullchain.pem and privkey.pem.

Sandbox API usage

Resources

• /api/status
• /api/log/import
• /api/log/reset
• /api/upload
• /api/import
• /api/reset

Status values

• running
• importing
• resetting
• down
• failed
• failed_import
• failed_reset

Sandbox API Usage

Check status

curl -H "Authorization: Bearer secret-token" -k https://FQDN:5001/api/status

Import log

curl -H "Authorization: Bearer secret-token" -k https://FQDN:5001/api/log/import

Reset log

curl -H "Authorization: Bearer secret-token" -k https://FQDN:5001/api/log/reset

File upload

curl --form file=@europdx.tgz -H "Authorization: Bearer secret-token" -X POST -k https://FQDN:5001/api/upload

Init data import

curl -X POST -H "Authorization: Bearer secret-token" -k https://FQDN:5001/api/import

Init sendbox reset to default

curl -X POST -H "Authorization: Bearer secret-token" -k https://FQDN:5001/api/reset

Note: -k is there to disable cURL SSL certificate verification

Example

curl -H "Authorization: Bearer secret-token" -k http://localhost:5001/api/status