Run Watson Speech to Text for Embed on an IBM Cloud Kubernetes cluster in a Virtual Private Cloud environment

This blog post is about to deploy the IBM Watson Speech to Text Library for Embed to an IBM Cloud Kubernetes cluster. IBM Cloud Kubernetes cluster is a “certified, managed Kubernetes solution, built for creating a cluster of compute hosts to deploy and manage containerized apps on IBM Cloud“.

The IBM Watson Libraries for Embed are made for IBM Business Partners. Partners can get additional details about embeddable AI on the IBM Partner World page. If you are an IBM Business Partner, you can get a free access to the IBM Speech to Text Library for Embed.

To get started with the libraries you can use the link Watson Speech to Text Library for Embed home. It is an awesome documentation, and it is publicly available.

I used parts of the great content of the IBM Watson documentation for my Kubernetes example. I created a GitHub project with some additional bash scripting. The project is called Run Watson Speech to Text for Embed on an IBM Cloud Kubernetes cluster and I did the same for local execution this project Run Watson Speech to Text for Embed on your local computer with Docker .

This project has two objectives.

The blog post is structured in:

  • Some simplified technical basics about IBM Watson Libraries for Embed
  • Simplified overview of the dependencies for the running Watson STT for Embed container on an IBM Cloud Kubernetes cluster in a VPC environment
  • Some screen shots from a running example by using an IBM Cloud Kubernetes cluster in a VPC environment
  • Setup of the example
  • Summary

1. Some simplified technical basics about IBM Watson Libraries for Embed

The IBM Watson Libraries for Embed do provide a lot of pre-trained models you can find in the related model catalog for the Watson Libraries. Here is a link to the model catalog for Watson STT, the catalog is public available.

There are runtime containers for the AI functionalities like Speech to Text, NLP and so on. You can integrate the container into your application and customize them, for example existing pre-trained models. By separating the container runtime and the models to be loaded into the container, we are free to include the right models we need to integrate in our application to fulfil the needs of our business use case.

The resources are available in a IBM Cloud Container Registry. The image below shows a simplified overview for Watson Speech to Text.

Currently (24.1.2023) you can’t customize the Speech to Text models for Embed, but if you are interested how it works for the public IBM Cloud Watson Speech to Text service, you can just visit my blog post Watson Speech to Text language model customization. You can find the details about Speech to Text API in the Watson Speech to Text API documentation.

2. Simplified overview of the dependencies for the running Watson STT for embed container on an IBM Cloud Kubernetes cluster in a VPC environment

The next image below shows a simplified overview of the dependencies of the running Watson STT pod on the Kubernetes cluster. Which is a bit different from the detailed instructions in the official IBM documentation Run with Kubernetes. It contains an additional Kubernetes load balancer service configuration for an IBM Cloud VPC.

The deployment you can see above is managed with a Helm chart.

The objectives are to provide a load balancer service for a direct access from the internet to Watson STT for Embed. The cluster ip service doesn’t provide this in the VPC environment and to create a pull secret specification with Helm chart and not with a kubectl command.

Helm chart contains the following specifications (the links are pointing to the source code), again this configuration bit different to IBM documentation Run with Kubernetes.

3. Some screen shots from a running example by using an IBM Cloud Kubernetes cluster in a VPC environment

  • Kubernetes cluster
  • Virtual Private Cloud
  • The load balancer service in Kubernetes
  • The load balancer service in Kubernetes creates a network load balancer on IBM Cloud VPC
  • Running pod on Kubernetes
  • Access the container in the pod on Kubernetes

4. Setup of the example

The example setup contains two bash automations:

Step 1: Clone the repository

git clone https://github.com/thomassuedbroecker/terraform-vpc-kubernetes-watson-stt.git
cd terraform-vpc-kubernetes-watson-stt

Create the Kubernetes cluster and VPC

Step 1: Navigate to the terraform_setup

cd code/terraform_setup

Step 2: Create a .env file

cat .env_template > .env

Step 3: Add an IBM Cloud access key to your local .env file

nano .env

Content of the file:

export IC_API_KEY=YOUR_IBM_CLOUD_ACCESS_KEY
export REGION="us-east"
export GROUP="tsuedbro"

Step 4: Verify the global variables in the bash script automation

Inspect the bash automation create_vpc_kubernetes_cluster_with_terraform.sh and adjust the values to your need.

nano create_vpc_kubernetes_cluster_with_terraform.sh


#export TF_LOG=debug
export TF_VAR_flavor="bx2.4x16"
export TF_VAR_worker_count="2"
export TF_VAR_kubernetes_pricing="tiered-pricing"
export TF_VAR_resource_group=$GROUP
export TF_VAR_vpc_name="watson-stt-tsuedbro"
export TF_VAR_region=$REGION
export TF_VAR_kube_version="1.25.5"
export TF_VAR_cluster_name="watson-stt-tsuedbro"

Step 5: Execute the bash automation

The creation can take up to 1 hour, depending on the region you use.

sh create_vpc_kubernetes_cluster_with_terraform.sh
  • Example output:
...
Apply complete! Resources: 6 added, 0 changed, 0 destroyed.
*********************************

Deploy Watson NLP embed with Helm

Step 1: Navigate to the helm_setup

cd code/helm_setup

Step 2: Create a .env file

cat .env_template > .env

Step 3: Add an IBM Cloud access key to your local .env file

export IC_API_KEY=YOUR_IBM_CLOUD_ACCESS_KEY
export IBM_ENTITLEMENT_KEY="YOUR_KEY"
export IBM_ENTITLEMENT_EMAIL="YOUR_EMAIL"
export CLUSTER_ID="YOUR_CLUSTER"
export REGION="us-east"
export GROUP="tsuedbro"

Step 4: Execute the bash automation

sh deploy-watson-stt-to-kubernetes.sh

The script does the ing steps, and the links are pointing to the relevant function in the bash automation:

  1. Log on to IBM Cloud with an IBM Cloud API key.
  2. It ensures that is connected to the cluster.
  3. It creates a Docker Config File which will be used to create a pull secret.
  4. It installs the Helm chart for Watson STT embed configured for REST API usage.
  5. It verifies that the container is running and invokes a REST API call inside the runtime-container of Watson STT embed.
  6. It verifies that the exposed Kubernetes URL with a load balancer service is working and invokes the same REST API call as before from the local machine.

Example output:

*********************
loginIBMCloud
*********************

API endpoint: https://cloud.ibm.com
Region: us-east
Authenticating...
OK

...

*********************
connectToCluster
*********************

OK

...

*********************
createDockerCustomConfigFile
*********************

IBM_ENTITLEMENT_SECRET: 

...

*********************
installHelmChart
*********************

...

==> Linting watson-stt-kubernetes
[INFO] Chart.yaml: icon is recommended

==> Linting ./watson-stt-kubernetes/
[INFO] Chart.yaml: icon is recommended

2 chart(s) linted, 0 chart(s) failed
NAME: watson-stt-kubernetes
LAST DEPLOYED: Tue Jan 24 20:33:04 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

*********************
verifyDeploment
*********************


------------------------------------------------------------------------
Check for (ibm-watson-stt-embed)
(1) from max retrys (4)
Status: ibm-watson-stt-embed
2023-01-24 20:33:06 Status: ibm-watson-stt-embed is created
------------------------------------------------------------------------

*********************
verifyPod could take 10 min
*********************


------------------------------------------------------------------------
Check for (ibm-watson-stt-embed)
(1) from max retrys (10)
Status: 0/1
2023-01-24 20:33:06 Status: ibm-watson-stt-embed(0/1)
------------------------------------------------------------------------
(2) from max retrys (10)
Status: 1/1
2023-01-24 20:34:07 Status: ibm-watson-stt-embed is created
------------------------------------------------------------------------

*********************
verifyWatsonSTTContainer
*********************

* Download audio
Pod: ibm-watson-stt-embed-576544dd4f-9h6tw

Result of download the example audio:

Chuck              example.flac        mkEnv.sh              requirements.txt       runTrial.sh           var

* Invocation of REST API audio
Result of the Watson STT REST API invocation:
Pod: ibm-watson-stt-embed-576544dd4f-9h6tw

Result of download the example audio:

http://localhost:1080/speech-to-text/api/v1/recognize

{
   "result_index": 0,
   "results": [
      {
         "final": true,
         "alternatives": [
            {
               "transcript": "several tornadoes touched down as a line of severe thunderstorms swept through colorado on sunday ",
               "confidence": 0.99
            }
         ]
      }
   ]
}

Verify the running pod on your cluster.
NAME                                    READY   STATUS    RESTARTS   AGE
ibm-watson-stt-embed-576544dd4f-9h6tw   1/1     Running   0          71s
Verify in the deployment in the Kubernetes dashboard.

Verify in the deployment in the Kubernetes dashboard.
https://cloud.ibm.com/kubernetes/clusters/XXXX/overview
Press any key to move on:
  • Open the Kubernetes dashboard:
  • Open the verify the pods:
*********************
verifyWatsonSTTLoadbalancer
this could take up to 10 min
*********************


*********************
verifyLoadbalancer
*********************


------------------------------------------------------------------------
Check for watson-stt-container-vpc-nlb: (0) from max retrys (11)
Status: <pending>
2023-01-25 07:45:39 Status: watson-stt-container-vpc-nlb(<pending>)
--------------------------------------------------
...
Status: <pending>
2023-01-25 07:53:44 Status: watson-stt-container-vpc-nlb(<pending>)
------------------------------------------------------------------------
Status: XXX
2023-01-25 07:54:45 Status: watson-stt-container-vpc-nlb is created (XX)
------------------------------------------------------------------------
EXTERNAL_IP: XXX
Verify invocation of Watson STT API from the local machine:
{
   "result_index": 0,
   "results": [
      {
         "final": true,
         "alternatives": [
            {
               "transcript": "several tornadoes touched down as a line of severe thunderstorms swept through colorado on sunday ",
               "confidence": 0.99
            }
         ]
      }
   ]
}
*********************
uninstallHelmChart
*********************

Press any key to move on:
  • You can verify all on your cluster and press any key to remove the example.
release "watson-stt-kubernetes" uninstalled

5. Summary

Overall it is awesome that Watson Speech to Text Library for Embed is a containerized implementation and you can run it anywhere.

I hope this was useful to you and let’s see what’s next?

Greetings,

Thomas

#ibmcloud, #watsonspeechtotext, #ai, #bashscripting, #kubernetes, #container, #vpc, #stt

One thought on “Run Watson Speech to Text for Embed on an IBM Cloud Kubernetes cluster in a Virtual Private Cloud environment”

Add yours

  1. Pingback: Find simple tutorials for `Watson Libraries for Embed` – Thomas Suedbroecker's Blog

Leave a Reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.

Up ↑

%d bloggers like this: