Use Software Everywhere and IasCable to setup GitOps on a Red Hat OpenShift Cluster in a Virtual Private Cloud on IBM Cloud

This is a long blog post and the third one related to the Software Everywhere project and IasCable. If you are new to Software Everywhere project and IasCable please take a look in these related blog posts.

• Get started with an installable component infrastructure by selecting components from a catalog of available modules with IasCable 
• Use IasCable to create a Virtual Private Cloud and a Red Hat OpenShift cluster on IBM Cloud

Our objective in this blog post is to create a customized initial GitOps setup in an IBM Cloud environment.

The Software Everywhere project and IasCable CLI do provide an awesome way to eliminate writing Terraform modules for various clouds such as IBM Cloud, AWS or Azure to create and configure resources. We are going to reuse Terraform modules which the Software Everywhere catalog does provide.

Surely, we need to know the needed outline for the cloud architecture which does depend on the cloud environment we are going to use.

As I said the Software Everywhere catalog does provide the reuse of existing Terraform modules, which we use by just combining by writing a “Bill of Material file“ and configure the variables for the related Terraform modules (example link to the GitOps Terraform module) when it is needed.

We will not write any Terraform code, we will only combine existing Terraform modules and configure them using IasCable BOM files!

In that scenario we will use IBM Cloud with a Virtual Private Cloud and a Red Hat OpenShift cluster with Argo CD installed and integrated with a GitHub project.

These are the major sections of this blog post:

1. Define an outline of the target architecture
2. Identify the needed Software Everywhere Terraform modules for the target architecture
3. Write a customized BOM to combine the modules
4. Use IasCable to create the scaffolding for a Terraform project
5. Use the IasCable tools container to execute the Terraform modules

Note: Depending on the container engine you are going to use on your computer, you maybe have to copy the Terraform project inside the running tools container, because of access right restrictions to access mapped local volumes to the running containers. That is the reason why I wrote some helper scripts to simplify the copy and deletion of the Terraform code mapped to the local volume of our computer. You can find the helper bash automation script in the GitHub project. IasCable does suggest to use Docker or Colima as container engine.

6. Apply the Terraform modules to create the environment in IBM Cloud and backup the Terraform state to the local computer.
7. Destroy the environment on IBM Cloud.
8. Summary

You can access the source code related to this blog post in the GitHub project I created. The project is under Apache-2.0 license.

git clone https://github.com/thomassuedbroecker/iascable-vpc-openshift-argocd.git
cd example

1. Define an outline of the target architecture

This is our simplified target architecture for our objective to create a customized setup in an IBM Cloud environment for GitOps.

• For the configuration of GitOps in Red Hat OpenShiftWe will use two operators:
  • Red at OpenShift GitOps operator
    • We will create one ArgoCD instance with the Red at OpenShift GitOps operator, that ArgoCD instance will bin initial configured by a newly created GitHub project configure by a Cloud Native Toolkit template for GitOps repositories.
  • Red at OpenShift Pipelines operator.
    • There will be no initial setup for a Tekton pipeline at the moment.

• IBM Cloud infrastructure with Red Hat OpenShift in a Virtual Private Cloud

2. Identify the needed Software Everywhere Terraform modules for the target architecture

Let us first define which Software Everywhere Terraform modules we are going to use for our custom BOM file specification. The Software Everywhere project points to the Automated Solutions project which contains several starting points for various setups, which can be used as a starting point.

In our case we have two major areas for Terraform modules we want to use:

1. Configuration of GitOps
2. IBM Cloud infrastructure

1. Configuration of GitOps

• IBM OpenShift login ocp-login – login to existing OpenShift cluster
• GitOps repo gitops-repo – creates the GitOps Repo
• ArgoCD Bootstrap argocd-bootstrap

2. Cloud infrastructure/services resources related

• IBM VPC ibm-vpc
• IBM VPC Subnets ibm-vpc-subnets
• IBM Cloud VPC Public Gateway ibm-vpc-gateways
• IBM OpenShift VPC cluster ibm-ocp-vpc
• IBM Object Storage ibm-object-storage

3. Write a customized BOM to combine the modules

Step 1: Write the Bill of Material BOM file

Now we combine the existing Terraform modules we got from the Software Everywhere catalog and we specify the variables in the BOM file we need to reflect our target architecture.

Note: When we going to use these variables, we must keep in mind that we need to use the names of the variables defined in the Terraform modules and we should use alias: ibm-vpc to define the prefix in the BOM file.

The BOM file for our architecture is divided in 3 main sections.

• Virtual Private Cloud
• Red Hat OpenShift Cluster (ROKS)
• GitOps

We need to create an IBM Cloud API key and a Personal Access Token for the GitHub account.

apiVersion: cloudnativetoolkit.dev/v1alpha1
kind: BillOfMaterial
metadata:
  name: my-ibm-vpc-roks-argocd
spec:
  modules:
    # Virtual Private Cloud - related
    # - subnets
    # - gateways
    - name: ibm-vpc
      alias: ibm-vpc
      version: v1.16.0
      variables:
      - name: name
        value: "tsued-gitops-sample"
      - name: tags
        value: ["tsuedro"]
    - name: ibm-vpc-subnets
      alias: ibm-vpc-subnets
      version: v1.13.2
      variables:
        - name: _count
          value: 1
        - name: name
          value: "tsued-gitops-sample"
        - name: tags
          value: ["tsuedro"]
    - name: ibm-vpc-gateways
    # ROKS - related
    # - objectstorage
    - name: ibm-ocp-vpc
      alias: ibm-ocp-vpc
      version: v1.15.5
      variables:
        - name: name
          value: "tsued-gitops"
        - name: worker_count
          value: 2
        - name: tags
          value: ["tsuedro"]
    - name: ibm-object-storage
      alias: ibm-object-storage
      version: v4.0.3
      variables:
        - name: name
          value: "cos_tsued_gitops"
        - name: tags
          value: ["tsuedro"]
        - name: label
          value: ["cos_tsued"]
    # Install OpenShift GitOps and Bootstrap GitOps (aka. ArgoCD) - related
    # - argocd
    # - gitops
    - name: argocd-bootstrap
      alias: argocd-bootstrap
      version: v1.12.0
      variables:
        - name: repo_token
    - name: gitops-repo
      alias: gitops-repo
      version: v1.20.2
      variables:
        - name: host
          value: "github.com"
        - name: type
          value: "GIT"
        - name: org
          value: "thomassuedbroecker"
        - name: username
          value: "thomassuedbroecker"
        - name: project
          value: "iascable-gitops"
        - name: repo
          value: "iascable-gitops"

The BOM will result later in following overall dependencies for the used Terraform modules after the usage of IasCable. The dependencies are given in automatic created dependencies.dot file later.

Note: You can use GraphvizOnline for the visualization.

4. Use IasCable to create the scaffolding for a Terraform project

Step 1: Install colima container engine and start the container engine

Example for an installation of colima on macOS.

brew install docker colima
colima start

Step 2: Create a terraform project based on Bill of Material BOM file

• Version

iascable --version

• Output:

2.14.1

• Build

iascable build -i my-vpc-roks-argocd-bom.yaml

• Output:

Loading catalog from url: https://modules.cloudnativetoolkit.dev/index.yaml
Name: my-ibm-vpc-roks-argocd
Writing output to: ./output

Step 3: Copy helper bash scripts into the output folder

cp helper-tools-create-container-workspace.sh ./output
cp helper-tools-execute-apply-and-backup-result.sh ./output
cp helper-tools-execute-destroy-and-delete-backup.sh ./output

Step 4: Start the tools container provided by the IasCable

Note: At the moment we need to change and save the launch.sh script a bit.

1. Open the launch.sh script.

cd output
nano launch.sh

2. Delete the -u "${UID}" parameter

• Before

${DOCKER_CMD} run -itd --name ${CONTAINER_NAME}    -u "${UID}"    -v "${SRC_DIR}:/terraform"    -v "workspace-${AUTOMATION_BASE}:/workspaces"    ${ENV_FILE}    -w /terraform    ${DOCKER_IMAGE}

• After the change

${DOCKER_CMD} run -itd --name ${CONTAINER_NAME} -v "${SRC_DIR}:/terraform"    -v "workspace-${AUTOMATION_BASE}:/workspaces"    ${ENV_FILE}    -w /terraform    ${DOCKER_IMAGE}

3. Execute the launch.sh script

sh launch.sh

5. Use the IasCable tools container to execute the Terraform modules

Step 1 (inside the container): In the running container verify the mapped resources

~/terraform $ ls
helper-tools-create-container-workspace.sh
helper-tools-execute-apply-and-backup-result.sh
helper-tools-execute-destroy-and-delete-backup.sh
launch.sh
my-ibm-vpc-roks-argocd

Step 2 (inside the container): Create a workspace folder in your container and copy your IasCable project into it

sh helper-tools-create-container-workspace.sh
ls /home/devops/workspace

The following tasks are automated in the helper bash script helper-tools-create-container-workspace.sh I wrote.

1. Creates a workspace folder
2. Copies the Terraform project from the mapped volume folder to the workspace folder

• Output:

You can see the copied Terraform project my-ibm-vpc-roks-argocdmy-ibm-vpc-roks-argocd folder inside the container.

helper-tools-create-container-workspace.sh
helper-tools-execute-apply-and-backup-result.sh
helper-tools-execute-destroy-and-delete-backup.sh
launch.sh
my-ibm-vpc-roks-argocdmy-ibm-vpc-roks-argocd

6. Apply the Terraform modules to create the environment in IBM Cloud and backup Terraform configuration

Step 1 (inside the container): Execute the apply.sh and backup the result into the mapped volume

All these tasks are automated in the helper bash script I wrote.

sh helper-tools-execute-apply-and-backup-result.sh

• The script helper-tools-execute-apply-and-backup-result.sh does following:
  1. Navigate to the created workspace
  2. Execute apply.sh script
  3. List the created resources
  4. Copy current start to mapped volume
• Interactive output:

As we see in the output the values we inserted in our custom BOM file are now used as the default values. In our example we only need to insert the values for:

• gitops-repo_token
• ibmcloud_api_key
• resource_group_name
• region

Variables can be provided in a yaml file passed as the first argument

Provide a value for 'gitops-repo_host':
  The host for the git repository. The git host used can be a GitHub, GitHub Enterprise, Gitlab, Bitbucket, Gitea or Azure DevOps server. If the host is null assumes in-cluster Gitea instance will be used.
> (github.com) 
Provide a value for 'gitops-repo_org':
  The org/group where the git repository exists/will be provisioned. If the value is left blank then the username org will be used.
> (thomassuedbroecker) 
Provide a value for 'gitops-repo_project':
  The project that will be used for the git repo. (Primarily used for Azure DevOps repos)
> (iascable-gitops)
Provide a value for 'gitops-repo_username':
  The username of the user with access to the repository
> (thomassuedbroecker) 
Provide a value for 'gitops-repo_token':
  The personal access token used to access the repository
> XXX
> Provide a value for 'ibmcloud_api_key':
> XXX
Provide a value for 'region':
> eu-de
Provide a value for 'worker_count':
  The number of worker nodes that should be provisioned for classic infrastructure
> (2)
Provide a value for 'ibm-ocp-vpc_flavor':
  The machine type that will be provisioned for classic infrastructure
> (bx2.4x16) 
Provide a value for 'ibm-vpc-subnets__count':
  The number of subnets that should be provisioned
> (1) 
Provide a value for 'resource_group_name':
  The name of the resource group
> default

• Output:

We will be asked do we want to move on with the setup and apply Terraform.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

After a while you should get following output.

Apply complete! Resources: 91 added, 0 changed, 0 destroyed.

Major resources which were created, divided in Cloud infrastructure/services resources and Cluster and GitOps configuration.

6.1. Cloud infrastructure/services resources

• 1 x VPC

• 1 x Subnet 

• 4 x Security groups Two were created during the subnet creation and two are related to the created Red Hat OpenShift cluster. 

• 1 x Virtual Private Endpoint 

• 1 x Public Gateway 

• 2 x Access Control Lists One was created for the VPC module and one during the creation by the subnet module. 

• 1 x Routing Table 

• 1 x Red Hat OpenShift Cluster 

• 1 x Object Storage 

6.2. Cluster and GitOps configuration

• Red Hat OpenShift GitOps operator and Red Hat OpenShift Pipelines operator 

• Cluster and GitOps configuration

• Preconfigured ArgoCD project

6.3 Create files or folder

The invoked apply.sh script will create following files or folders:

• Inside the tools container:
  • A temporary workspace/my-ibm-vpc-roks-argocd/variables.yaml.tmp file
  • A workspace/my-ibm-vpc-roks-argocd/variables.yaml file
  • A workspace/my-ibm-vpc-roks-argocd/terraform/variables.tf file
  • A workspace/my-ibm-vpc-roks-argocd/terraform/variables.tfvars file
  • several folders .kube, .terraform, .tmp, bin2, docs
  • Then it creates a terraform.tfvars file based on the entries you gave and executes init and apply command from Terraform.Be aware that the IBM Cloud access key information and GitHub access token are saved in text format in the output/my-ibm-vpc-roks-argocd/terraform/terraform.tfvars file! Don’t share this in a public GitHub repository.

Note: Here you can access a sample of the content of an example for a generated variables.yaml file link and here you can find an example for the created BOM file.

• On GitHub:
  • It creates a GitHub private project which contains preconfigure ArgoCD resource provided by Cloud Native Toolkit.

7. Destroy the environment on IBM Cloud

Step 1 (inside the container): Destroy the created IBM Cloud resources

All these tasks are automated in the helper bash script I wrote.

Note: Ensure you didn’t delete created Terraform files before.

sh helper-tools-execute-destroy-and-delete-backup.sh

• The script helper-tools-execute-destroy-and-delete-backup.sh does following:

1. Navigate to workspace
2. Execute destroy.sh
3. Navigate to the mapped volume
4. Copy the current state to the mapped volume

• Output:

Note: It also will delete the automated created private GitHub project for the GitOps repository.

Destroy complete! Resources: 91 destroyed.

8. Summary

We achieved what we wanted to achieve, create a customized initial setup in an IBM Cloud environment for GitOps.

The Software Everywhere project and IasCable are powerful.

As we have seen there was no need to write any Terraform module!

Yes, when you are going to define you own “Bill of Material (BOM) file” you need to get familiar with the related modules related to your target architecture, when you want to customize it to your needs.

But, as I said: There was no need to write own Terraform modules in our case.

The Software Everywhere project and IasCable project needs some more documentation in the future, I like the power of it and it is under Apache-2.0 license, which means you can use it as your starting point for Software Everywhere with Terraform and contribute to the project.

I hope this was useful to you and let’s see what’s next?

Greetings,

Thomas

#ibmcloud, #vpc, #terraform, #cloudnativetoolkit, #softwareeverywhere, #iascable, #gitops, #argocd, #openshift, #terraform