Deploy an application that uses OpenAI on Azure Kubernetes Service (AKS)

In this article, you deploy the AKS Store Demo application on Azure Kubernetes Service (AKS) and configure Azure OpenAI or the OpenAI API to generate product descriptions.

The sample cloud-native application comprises services written in multiple languages and frameworks, including:

Golang with Gin
Rust with Actix-Web
JavaScript with Vue.js and Fastify
Python with FastAPI

These applications provide front ends for shoppers and store administrators, REST APIs that send data to a RabbitMQ message queue and DocumentDB database, and console apps that simulate traffic.

Note

Don't run stateful containers, such as DocumentDB and RabbitMQ, without persistent storage in production. This article uses them without persistent storage for simplicity. For production workloads, use managed services such as Azure Cosmos DB or Azure Service Bus.

To access the GitHub codebase for the sample application, see AKS Store Demo.

Before you begin

You need an Azure account with an active subscription. If you don't have one, create an account for free.
For this demo, use either Azure OpenAI or the OpenAI API.
- To use Azure OpenAI, you need permissions to create an Azure OpenAI resource and deploy a model. Registration is required only for limited-access models or modified safeguards.
- To use the OpenAI API, sign up on the OpenAI website.

Prerequisites

Use the Bash environment in Azure Cloud Shell. For more information, see Get started with Azure Cloud Shell.
If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.
- If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Authenticate to Azure using Azure CLI.
- When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use and manage extensions with the Azure CLI.
- Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

Create a resource group

An Azure resource group is a logical group in which you deploy and manage Azure resources. When you create a resource group, you're prompted to specify a location. This location is the storage location of your resource group metadata and where your resources run in Azure if you don't specify another region during resource creation.

The following example creates a resource group named myResourceGroup in the eastus location.

Create a resource group using the az group create command.

az group create --name myResourceGroup --location eastus

The following example output shows successful creation of the resource group:

{
  "id": "/subscriptions/<guid>/resourceGroups/myResourceGroup",
  "location": "eastus",
  "managedBy": null,
  "name": "myResourceGroup",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

Create an AKS cluster

The following example creates a cluster named myAKSCluster in myResourceGroup.

Create an AKS cluster using the az aks create command.

az aks create --resource-group myResourceGroup --name myAKSCluster --generate-ssh-keys

After a few minutes, the command completes and returns JSON-formatted information about the cluster.

Connect to the cluster

To manage a Kubernetes cluster, use the Kubernetes command-line client, kubectl. If you use Azure Cloud Shell, kubectl is already installed.

Install kubectl locally using the az aks install-cli command.
```
az aks install-cli
```
Note

If your Linux-based system requires elevated permissions, use the sudo az aks install-cli command.
Configure kubectl to connect to your Kubernetes cluster using the az aks get-credentials command.

This command runs the following operations:
- Downloads credentials and configures the Kubernetes CLI to use them.
- Uses ~/.kube/config, the default location for the Kubernetes configuration file. Specify a different location for your Kubernetes configuration file by using the --file argument.
```
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster
```

Verify the connection to your cluster using the kubectl get command. This command returns a list of the cluster nodes.

kubectl get nodes

The following example output shows the nodes created in the preceding steps. Verify that each node has a status of Ready.

NAME                                STATUS   ROLES   AGE     VERSION
aks-nodepool1-31469198-vmss000000   Ready    agent   3h29m   v1.25.6
aks-nodepool1-31469198-vmss000001   Ready    agent   3h29m   v1.25.6
aks-nodepool1-31469198-vmss000002   Ready    agent   3h29m   v1.25.6

Note

For a private AKS cluster, connect from an endpoint in the same virtual network as the cluster. For configuration instructions, see Create a private AKS cluster.

Deploy the application

The AKS Store application manifest includes the following Kubernetes deployments and services:

Product service: Shows product information.
Order service: Places orders.
Makeline service: Processes orders from the queue and completes the orders.
Store front: Web application for customers to view products and place orders.
Store admin: Web application for store employees to view orders in the queue and manage product information.
Virtual customer: Simulates order creation on a scheduled basis.
Virtual worker: Simulates order completion on a scheduled basis.
DocumentDB: NoSQL instance for persisted data.
RabbitMQ: Message queue for an order queue.

Note

Review the YAML manifest for the application.

Deploy the application using the kubectl apply command and specify the name of your YAML manifest.

kubectl apply -f https://raw.githubusercontent.com/Azure-Samples/aks-store-demo/main/aks-store-all-in-one.yaml

Deploy an OpenAI service

Choose Azure OpenAI or the OpenAI API for the application running on AKS.

Azure OpenAI
OpenAI

In the Azure portal, create an Azure OpenAI instance.
Sign in to Microsoft Foundry.
Turn off the New Foundry toggle, and then select View all resources under Keep building with Foundry.
Select the Azure OpenAI resource you created, and then select Deployments under Shared resources.
Select Deploy model > Deploy base model.
Select the gpt-4o-mini base model, enter a deployment name, and then deploy the model. Save the deployment name for the next section.

For more information about creating an Azure OpenAI deployment, see Get started generating text using Azure OpenAI.

Deploy the AI microservice

Deploy the Python-based AI microservice that uses OpenAI to automatically generate product descriptions. This microservice connects to the AKS Store application you deployed in the previous section.

Azure OpenAI
OpenAI

Use the Azure OpenAI resource and model deployment you created in the Deploy an OpenAI service section. Configure the following environment variables in the AI microservice manifest:

Environment variable	Value
`USE_AZURE_OPENAI`	Set to `"True"` to use Azure OpenAI.
`AZURE_OPENAI_DEPLOYMENT_NAME`	The name of your `gpt-4o-mini` model deployment in Microsoft Foundry.
`AZURE_OPENAI_ENDPOINT`	The endpoint from the Keys and Endpoint page of your Azure OpenAI resource.
`OPENAI_API_KEY`	An API key from the Keys and Endpoint page of your Azure OpenAI resource.

Create a file named ai-service.yaml and copy in the following manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ai-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ai-service
  template:
    metadata:
      labels:
        app: ai-service
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
        - name: ai-service
          image: ghcr.io/azure-samples/aks-store-demo/ai-service:2.2.0
          ports:
            - containerPort: 5001
          env:
            - name: USE_AZURE_OPENAI
              value: "True"
            - name: AZURE_OPENAI_DEPLOYMENT_NAME
              value: ""
            - name: AZURE_OPENAI_ENDPOINT
              value: ""
            - name: OPENAI_API_KEY
              value: ""
          resources:
            requests:
              cpu: 20m
              memory: 50Mi
            limits:
              cpu: 50m
              memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: ai-service
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 5001
      targetPort: 5001
  selector:
    app: ai-service

Set the environment variable USE_AZURE_OPENAI to "True".
In Microsoft Foundry, copy the Azure OpenAI deployment name and set the AZURE_OPENAI_DEPLOYMENT_NAME value.
In the Azure portal, select Keys and Endpoint in your Azure OpenAI resource. Copy the endpoint and API key, and then set the AZURE_OPENAI_ENDPOINT and OPENAI_API_KEY values.
Deploy the application using the kubectl apply command and specify the name of your YAML manifest.
```
kubectl apply -f ai-service.yaml
```
The following example output shows the successfully created deployments and services:
```
  deployment.apps/ai-service created
  service/ai-service created
```

Use the OpenAI secret key and organization ID from the Deploy an OpenAI service section. Configure the following environment variables in the AI microservice manifest:

Environment variable	Value
`USE_AZURE_OPENAI`	Set to `"False"` to use OpenAI.
`OPENAI_API_KEY`	The OpenAI secret key generated on the OpenAI API keys page.
`OPENAI_ORG_ID`	Your organization ID from the OpenAI organization settings page.

Create a file named ai-service.yaml and copy in the following manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ai-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ai-service
  template:
    metadata:
      labels:
        app: ai-service
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
        - name: ai-service
          image: ghcr.io/azure-samples/aks-store-demo/ai-service:2.2.0
          ports:
            - containerPort: 5001
          env:
            - name: USE_AZURE_OPENAI
              value: "False"
            - name: OPENAI_API_KEY
              value: ""
            - name: OPENAI_ORG_ID
              value: ""
          resources:
            requests:
              cpu: 20m
              memory: 50Mi
            limits:
              cpu: 50m
              memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: ai-service
spec:
  type: ClusterIP
  ports:
    - name: http
      port: 5001
      targetPort: 5001
  selector:
    app: ai-service

Set the environment variable USE_AZURE_OPENAI to "False".
Set OPENAI_API_KEY to the OpenAI secret key you generated on the OpenAI API keys page in the Deploy an OpenAI service section.
Set OPENAI_ORG_ID to the organization ID from your OpenAI organization settings page.
Deploy the application using the kubectl apply command and specify the name of your YAML manifest.
```
kubectl apply -f ai-service.yaml
```
The following example output shows the successfully created deployments and services:
```
  deployment.apps/ai-service created
  service/ai-service created
```

Note

Storing API keys directly in Kubernetes manifests is insecure and can expose secrets in source control. This example uses manifest values for simplicity. For production workloads, authenticate to Azure OpenAI by using a managed identity, or store secrets in Azure Key Vault.

Test the application

Check the status of the deployed pods using the kubectl get pods command.
```
kubectl get pods
```
Verify that all pods have a status of Running before you continue.
Get the external IP address for Store Admin by using the kubectl get service command.
```
kubectl get service store-admin
```
A Kubernetes service exposes Store Admin through a public load balancer. The load balancer might take a few minutes to assign an address. The EXTERNAL-IP value displays pending until the address is available.
```
NAME          TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)        AGE
store-admin   LoadBalancer   10.0.142.228   40.64.86.161    80:32494/TCP   50m
```
Repeat the preceding step for the store-front service.
In a web browser, go to the external IP address for Store Admin. In this example, the address is 40.64.86.161.
In Store Admin, select Products > Add Product.
After ai-service starts, the Ask AI Assistant button appears next to the description field. Enter the name, price, and keywords, select Ask AI Assistant to generate a product description, and then select Save Product.
Verify that Dog Smart Collar appears in Store Admin.
Go to the external IP address for Store Front and verify that Dog Smart Collar appears.

Next steps

Now that you added OpenAI functionality to an AKS application, you can Secure access to Azure OpenAI from Azure Kubernetes Service (AKS).

To learn more about generative AI use cases, see the following resources:

Feedback

Was this page helpful?

Last updated on 2026-08-06

Deploy an application that uses OpenAI on Azure Kubernetes Service (AKS)

Before you begin

Prerequisites

Create a resource group

Create an AKS cluster

Connect to the cluster

Deploy the application

Deploy an OpenAI service

Deploy the AI microservice

Test the application

Next steps

Feedback

Additional resources