From 62d5a3cccab5f16c486aa092d8a039e75c5ee609 Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Fri, 25 Nov 2022 23:31:30 +0100 Subject: [PATCH 01/11] Add README.md --- README.md | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..bafd8c3 --- /dev/null +++ b/README.md @@ -0,0 +1,51 @@ +# data.coop infrastructure + +This repository contains the code used to deploy data.coop's services +and websites. We use Ansible to encode our infrastructure setup. Only +the association's administrators have access to deploy the services. + +## Deploying + +To deploy the services, the included `deploy.sh` script can be used. The +Ansible playbook uses two custom-made roles (in the `roles/` directory). +The script has options to deploy only one of the roles, which are namely +`ubuntu_base` and `docker`, the latter of which is for deploying the +services themselves using Docker containers. Select services only can +also be specified. By default, the script deploys everything. + +Here is a summary of the options that can be used: + +```console +$ ./deploy.sh # deploy everything +$ ./deploy.sh base # deploy the ubuntu_base role only +$ ./deploy.sh services # deploy the docker role only +$ ./deploy.sh services SERVICE # deploy SERVICE only +``` + +`SERVICE` should match one of the filenames in +`roles/docker/tasks/services/`, but without the `.yml` extension + +## Testing + +In order for us to be able to test our setup locally, we use Vagrant to +deploy the services in a virtual machine. To do this, Vagrant and +VirtualBox must both be installed on the development machine. Then, the +services can be deployed locally by using the `vagrant` command-line +tool. The working directory needs to be the root of the repository for +this to work properly. + +> Note: As our secrets are contained in an Ansible Vault file, only the +> administrators have the ability to run the deployment in Vagrant. +> However, one could replace the vault file for testing purposes. + +Here is a summary of the commands that are available with the `vagrant` +command-line tool: + +```console +$ vagrant up # Create and provision the VM +$ vagrant provision # Re-provision the VM +$ vagrant ssh # SSH into the VM +$ vagrant halt # Power down the VM +$ vagrant destroy # Power down and delete the VM +``` + From 4611d890f7ef3db0ba946c89fa72b22fa74eee0a Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Sat, 26 Nov 2022 16:32:06 +0100 Subject: [PATCH 02/11] Update README.md --- README.md | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index bafd8c3..114d1a1 100644 --- a/README.md +++ b/README.md @@ -13,17 +13,17 @@ The script has options to deploy only one of the roles, which are namely services themselves using Docker containers. Select services only can also be specified. By default, the script deploys everything. -Here is a summary of the options that can be used: +Here is a summary of the options that can be used with the script: ```console -$ ./deploy.sh # deploy everything -$ ./deploy.sh base # deploy the ubuntu_base role only -$ ./deploy.sh services # deploy the docker role only -$ ./deploy.sh services SERVICE # deploy SERVICE only +$ ./deploy.sh # deploy everything +$ ./deploy.sh base # deploy the ubuntu_base role only +$ ./deploy.sh services # deploy the docker role only +$ ./deploy.sh services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service only ``` -`SERVICE` should match one of the filenames in -`roles/docker/tasks/services/`, but without the `.yml` extension +`SINGLE_SERVICE` should match one of the filenames in +`roles/docker/tasks/services/`, but without the `.yml` extension. ## Testing @@ -49,3 +49,16 @@ $ vagrant halt # Power down the VM $ vagrant destroy # Power down and delete the VM ``` +The `vagrant` command-line tool does not support supplying extra +variables to Ansible on runtime, so to be able to deploy only parts of +the Ansible playbook to Vagrant, the `deploy.sh` script can be used with +the `--vagrant` flag. Here are some examples: + +```console +$ ./deploy.sh --vagrant base # deploy the ubuntu_base role only in the Vagrant VM +$ ./deploy.sh --vagrant services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service only in the Vagrant VM +``` + +Note that the `--vagrant` flag should be the first argument when using +the script. + From 7c7379c42cd5b405844152cd8c70acc554a62842 Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Sat, 26 Nov 2022 20:28:19 +0100 Subject: [PATCH 03/11] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 114d1a1..e756adb 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ $ ./deploy.sh services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service on ``` `SINGLE_SERVICE` should match one of the filenames in -`roles/docker/tasks/services/`, but without the `.yml` extension. +`roles/docker/tasks/services/**/*`, but without the `.yml` extension. ## Testing From c8d603b6aa305f3d8826626af72dc532658628eb Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Sat, 26 Nov 2022 22:50:32 +0100 Subject: [PATCH 04/11] Add J2Live to README.md --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index e756adb..125db88 100644 --- a/README.md +++ b/README.md @@ -62,3 +62,8 @@ $ ./deploy.sh --vagrant services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker Note that the `--vagrant` flag should be the first argument when using the script. +## Nice tools + +- [J2Live](https://j2live.ttl255.com/): A live Jinja2 parser, nice to + test out filters + From 93b1ed60aebdc21a7e71119c5f808af980cc7fe2 Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Sun, 27 Nov 2022 17:20:40 +0100 Subject: [PATCH 05/11] Update README.md --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 125db88..c2651da 100644 --- a/README.md +++ b/README.md @@ -22,8 +22,9 @@ $ ./deploy.sh services # deploy the docker role only $ ./deploy.sh services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service only ``` -`SINGLE_SERVICE` should match one of the filenames in -`roles/docker/tasks/services/**/*`, but without the `.yml` extension. +`SINGLE_SERVICE` should match one of the service names in the `services` +dictionary in `roles/docker/defaults/main.yml` (e.g. `gitea` or +`data_coop_website`). ## Testing From bef767ebd87db52b7de7952d1172c6c019b35e3d Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Sun, 27 Nov 2022 17:35:40 +0100 Subject: [PATCH 06/11] Update README.md --- README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index c2651da..66c06e4 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ also be specified. By default, the script deploys everything. Here is a summary of the options that can be used with the script: -```console +```shell $ ./deploy.sh # deploy everything $ ./deploy.sh base # deploy the ubuntu_base role only $ ./deploy.sh services # deploy the docker role only @@ -42,7 +42,7 @@ this to work properly. Here is a summary of the commands that are available with the `vagrant` command-line tool: -```console +```shell $ vagrant up # Create and provision the VM $ vagrant provision # Re-provision the VM $ vagrant ssh # SSH into the VM @@ -55,7 +55,7 @@ variables to Ansible on runtime, so to be able to deploy only parts of the Ansible playbook to Vagrant, the `deploy.sh` script can be used with the `--vagrant` flag. Here are some examples: -```console +```shell $ ./deploy.sh --vagrant base # deploy the ubuntu_base role only in the Vagrant VM $ ./deploy.sh --vagrant services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service only in the Vagrant VM ``` @@ -67,4 +67,3 @@ the script. - [J2Live](https://j2live.ttl255.com/): A live Jinja2 parser, nice to test out filters - From 3f036ac0eaadb6e95180b3794ab7309b9a381ced Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Sun, 27 Nov 2022 21:00:47 +0100 Subject: [PATCH 07/11] Revert "Update README.md" This reverts commit bef767ebd87db52b7de7952d1172c6c019b35e3d. --- README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 66c06e4..c2651da 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ also be specified. By default, the script deploys everything. Here is a summary of the options that can be used with the script: -```shell +```console $ ./deploy.sh # deploy everything $ ./deploy.sh base # deploy the ubuntu_base role only $ ./deploy.sh services # deploy the docker role only @@ -42,7 +42,7 @@ this to work properly. Here is a summary of the commands that are available with the `vagrant` command-line tool: -```shell +```console $ vagrant up # Create and provision the VM $ vagrant provision # Re-provision the VM $ vagrant ssh # SSH into the VM @@ -55,7 +55,7 @@ variables to Ansible on runtime, so to be able to deploy only parts of the Ansible playbook to Vagrant, the `deploy.sh` script can be used with the `--vagrant` flag. Here are some examples: -```shell +```console $ ./deploy.sh --vagrant base # deploy the ubuntu_base role only in the Vagrant VM $ ./deploy.sh --vagrant services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service only in the Vagrant VM ``` @@ -67,3 +67,4 @@ the script. - [J2Live](https://j2live.ttl255.com/): A live Jinja2 parser, nice to test out filters + From 6a29cdc84d566cfd6581ec47177dfd648d18090f Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Mon, 28 Nov 2022 18:20:12 +0100 Subject: [PATCH 08/11] Apply benjaoming's suggestions --- README.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index c2651da..1e67a00 100644 --- a/README.md +++ b/README.md @@ -7,11 +7,15 @@ the association's administrators have access to deploy the services. ## Deploying To deploy the services, the included `deploy.sh` script can be used. The -Ansible playbook uses two custom-made roles (in the `roles/` directory). -The script has options to deploy only one of the roles, which are namely -`ubuntu_base` and `docker`, the latter of which is for deploying the -services themselves using Docker containers. Select services only can -also be specified. By default, the script deploys everything. +Ansible playbook uses two custom-made roles (in the `roles/` directory): + +- `ubuntu_base` - used for configuring the OS itself and installing the + necessary packages +- `docker` - used to deploy our services and websites with Docker + containers + +The script has options to deploy only one of the roles. Select services only +can also be specified. By default, the script deploys everything. Here is a summary of the options that can be used with the script: From 789caed704b192aba8dceb9be11b875a1b563d80 Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Mon, 28 Nov 2022 18:56:09 +0100 Subject: [PATCH 09/11] Change wording --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1e67a00..c96782a 100644 --- a/README.md +++ b/README.md @@ -9,13 +9,13 @@ the association's administrators have access to deploy the services. To deploy the services, the included `deploy.sh` script can be used. The Ansible playbook uses two custom-made roles (in the `roles/` directory): -- `ubuntu_base` - used for configuring the OS itself and installing the +- `ubuntu_base` - used to configure the host itself and install the necessary packages - `docker` - used to deploy our services and websites with Docker containers -The script has options to deploy only one of the roles. Select services only -can also be specified. By default, the script deploys everything. +The script has options to deploy only one of the roles. Select services +only can also be specified. By default, the script deploys everything. Here is a summary of the options that can be used with the script: From 09215e117aee2f2a4cfe1f391cf4719188579333 Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Mon, 28 Nov 2022 19:24:49 +0100 Subject: [PATCH 10/11] Add 'Contributing' section --- README.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/README.md b/README.md index c96782a..d27ba9f 100644 --- a/README.md +++ b/README.md @@ -67,6 +67,18 @@ $ ./deploy.sh --vagrant services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker Note that the `--vagrant` flag should be the first argument when using the script. +## Contributing + +If you want to contribute, you can fork the repository and submit a pull +request. We use a pre-commit hook for linting the YAML files before +every commit, so please use that. To initialize pre-commit, you need to +have Python and GNU make installed. Then, just run the following shell +command: + +```console +$ make init +``` + ## Nice tools - [J2Live](https://j2live.ttl255.com/): A live Jinja2 parser, nice to From 6cd0eadadeabfee3773415ceb856d4a6ed3e86d6 Mon Sep 17 00:00:00 2001 From: Sam Al-Sapti Date: Mon, 28 Nov 2022 19:31:31 +0100 Subject: [PATCH 11/11] Apply valberg's suggestions --- README.md | 51 +++++++++++++++++++++++++++++++++++---------------- 1 file changed, 35 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index d27ba9f..97e3b76 100644 --- a/README.md +++ b/README.md @@ -19,11 +19,18 @@ only can also be specified. By default, the script deploys everything. Here is a summary of the options that can be used with the script: -```console -$ ./deploy.sh # deploy everything -$ ./deploy.sh base # deploy the ubuntu_base role only -$ ./deploy.sh services # deploy the docker role only -$ ./deploy.sh services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service only +```sh +# deploy everything +./deploy.sh + +# deploy the ubuntu_base role only +./deploy.sh base + +# deploy the docker role only +./deploy.sh services + +# deploy SINGLE_SERVICE Docker service only +./deploy.sh services SINGLE_SERVICE ``` `SINGLE_SERVICE` should match one of the service names in the `services` @@ -46,12 +53,21 @@ this to work properly. Here is a summary of the commands that are available with the `vagrant` command-line tool: -```console -$ vagrant up # Create and provision the VM -$ vagrant provision # Re-provision the VM -$ vagrant ssh # SSH into the VM -$ vagrant halt # Power down the VM -$ vagrant destroy # Power down and delete the VM +```sh +# Create and provision the VM +vagrant up + +# Re-provision the VM +vagrant provision + +# SSH into the VM +vagrant ssh + +# Power down the VM +vagrant halt + +# Power down and delete the VM +vagrant destroy ``` The `vagrant` command-line tool does not support supplying extra @@ -59,9 +75,12 @@ variables to Ansible on runtime, so to be able to deploy only parts of the Ansible playbook to Vagrant, the `deploy.sh` script can be used with the `--vagrant` flag. Here are some examples: -```console -$ ./deploy.sh --vagrant base # deploy the ubuntu_base role only in the Vagrant VM -$ ./deploy.sh --vagrant services SINGLE_SERVICE # deploy SINGLE_SERVICE Docker service only in the Vagrant VM +```sh +# deploy the ubuntu_base role only in the Vagrant VM +./deploy.sh --vagrant base + +# deploy SINGLE_SERVICE Docker service only in the Vagrant VM +./deploy.sh --vagrant services SINGLE_SERVICE ``` Note that the `--vagrant` flag should be the first argument when using @@ -75,8 +94,8 @@ every commit, so please use that. To initialize pre-commit, you need to have Python and GNU make installed. Then, just run the following shell command: -```console -$ make init +```sh +make init ``` ## Nice tools