แก้ไข

Tutorial: Enable Azure Container Apps on Azure Arc-enabled Kubernetes

This tutorial prepares a supported Kubernetes cluster to run Azure Container Apps. It creates an Azure Arc-enabled Kubernetes cluster, installs the Container Apps extension, creates a custom location, and creates a Container Apps connected environment.

Completing this tutorial changes both your Azure subscription and your Kubernetes cluster. You need Azure permissions to create the listed resources and cluster-admin access to the Kubernetes cluster.

This tutorial shows how to enable Azure Container Apps on an Azure Arc-enabled Kubernetes cluster. In this tutorial, you:

  • Connect a Kubernetes cluster to Azure Arc, if it isn't already connected.
  • Optionally create a Log Analytics workspace.
  • Install and validate the Container Apps extension.
  • Create and validate a custom location.
  • Create and validate a Container Apps connected environment.

Before you begin, review Azure Container Apps on Azure Arc. If a resource or in-cluster component doesn't become ready, see Troubleshoot Azure Container Apps on Azure Arc-enabled Kubernetes.

Prerequisites

Before you begin, verify the following requirements:

  • An Azure account with an active subscription. If you don't have one, create one for free.
  • Permission to register resource providers and create resource groups, an Azure Arc-enabled Kubernetes resource, a cluster extension, a custom location, a Container Apps connected environment, and optionally a Log Analytics workspace.
  • The Azure CLI and its prerequisites.
  • A kubectl version compatible with your Kubernetes cluster and network access from your workstation to the Kubernetes API server.
  • A supported Kubernetes cluster with Linux amd64 worker nodes and cluster-admin access through the active kubectl context.
  • A working Kubernetes LoadBalancer service implementation.
  • Outbound connectivity to the endpoints required by Azure Arc and Container Apps.
  • Sufficient allocatable CPU and memory for the extension and application workloads.

Before changing a production cluster, review Plan a Container Apps deployment on Azure Arc-enabled Kubernetes.

Setup

Install the following Azure CLI extensions.

az extension add --name connectedk8s --upgrade --yes
az extension add --name k8s-extension --upgrade --yes
az extension add --name customlocation --upgrade --yes
az extension add --name containerapp --upgrade --yes

Register the required namespaces.

az provider register --namespace Microsoft.ExtendedLocation --wait
az provider register --namespace Microsoft.KubernetesConfiguration --wait
az provider register --namespace Microsoft.App --wait
az provider register --namespace Microsoft.Web --wait
az provider register --namespace Microsoft.OperationalInsights --wait

Set environment variables based on your Kubernetes cluster deployment.

GROUP_NAME="my-arc-cluster-group"
AKS_CLUSTER_GROUP_NAME="my-aks-cluster-group"
AKS_NAME="my-aks-cluster"
LOCATION="eastus"

Create a connected cluster

If you already have a supported Azure Arc-enabled Kubernetes cluster in $GROUP_NAME, don't create another cluster. Set CLUSTER_NAME to the existing connected-cluster resource name, verify access with kubectl get nodes, and continue to Create a Log Analytics workspace.

The following steps create an AKS cluster and connect it to Azure Arc. This path is intended only as an Azure-hosted evaluation environment for the tutorial. For an existing on-premises or multicloud cluster, follow Quickstart: Connect an existing Kubernetes cluster to Azure Arc, and then return to this tutorial.

  1. Create a cluster in Azure Kubernetes Service.

    az group create --name $AKS_CLUSTER_GROUP_NAME --location $LOCATION
az aks create \
   --resource-group $AKS_CLUSTER_GROUP_NAME \
   --name $AKS_NAME \
   --enable-aad \
   --generate-ssh-keys

  2. Get the kubeconfig file and test your connection to the cluster. By default, the kubeconfig file is saved to ~/.kube/config.

    az aks get-credentials --resource-group $AKS_CLUSTER_GROUP_NAME --name $AKS_NAME --admin

kubectl get ns

  3. Create a resource group to contain your Azure Arc resources.

    az group create --name $GROUP_NAME --location $LOCATION

  4. Connect the cluster you created to Azure Arc.

    CLUSTER_NAME="${GROUP_NAME}-cluster" # Name of the connected cluster resource

az connectedk8s connect --resource-group $GROUP_NAME --name $CLUSTER_NAME

  5. Wait for the connected cluster to finish provisioning.

    CONNECTED_CLUSTER_ID=$(az connectedk8s show \
    --resource-group $GROUP_NAME \
    --name $CLUSTER_NAME \
    --query id \
    --output tsv)

az resource wait --ids $CONNECTED_CLUSTER_ID --created --timeout 600

az connectedk8s show \
    --resource-group $GROUP_NAME \
    --name $CLUSTER_NAME \
    --query "{State:provisioningState,Connectivity:connectivityStatus}" \
    --output table

    Continue only when State is Succeeded and Connectivity is Connected. If the command times out, see Connected cluster isn't ready.

Create a Log Analytics workspace

A Log Analytics workspace provides access to application logs for Container Apps running in the Azure Arc-enabled Kubernetes cluster. A Log Analytics workspace is optional but recommended for application diagnostics.

Important

Decide whether to use Log Analytics before installing the Container Apps extension. Supply the Log Analytics configuration during extension installation. You can't add Log Analytics configuration to that extension instance later.

  1. Create a Log Analytics workspace.

    WORKSPACE_NAME="$GROUP_NAME-workspace" # Name of the Log Analytics workspace

az monitor log-analytics workspace create \
    --resource-group $GROUP_NAME \
    --workspace-name $WORKSPACE_NAME

  2. Run the following commands to get the encoded workspace ID and shared key for an existing Log Analytics workspace. You need them in the next step.

    Caution

    The workspace shared key is a credential. Don't print it, commit it to source control, store it in shell history, or include it in support bundles. The following commands keep it in a shell variable and pass it to Azure as a protected extension setting.

    LOG_ANALYTICS_WORKSPACE_ID=$(az monitor log-analytics workspace show \
    --resource-group $GROUP_NAME \
    --workspace-name $WORKSPACE_NAME \
    --query customerId \
    --output tsv)
LOG_ANALYTICS_WORKSPACE_ID_ENC=$(printf %s $LOG_ANALYTICS_WORKSPACE_ID | base64 -w0) # Needed for the next step
LOG_ANALYTICS_KEY=$(az monitor log-analytics workspace get-shared-keys \
    --resource-group $GROUP_NAME \
    --workspace-name $WORKSPACE_NAME \
    --query primarySharedKey \
    --output tsv)
LOG_ANALYTICS_KEY_ENC=$(printf %s $LOG_ANALYTICS_KEY | base64 -w0) # Needed for the next step

Check for an existing KEDA installation

The Container Apps extension installs KEDA. Before installing the extension, check the cluster for existing KEDA components:

kubectl get deployments -A -o custom-columns="NAMESPACE:.metadata.namespace,NAME:.metadata.name"
kubectl get crd scaledobjects.keda.sh

Inspect the deployment list for KEDA components. If either command identifies an existing KEDA installation, stop and confirm the supported coexistence configuration before continuing. Don't remove the existing KEDA installation or apply undocumented extension settings because other workloads might depend on it.

Install the Container Apps extension

Important

If you're deploying onto AKS on Azure Local, ensure that you set up HAProxy or a custom load balancer before attempting to install the extension. You can also use az containerapp arc setup-core-dns --distro AksAzureLocal to set up CoreDNS for local contexts.

  1. Set names for the Container Apps extension, its Kubernetes namespace, and the connected environment.

    • EXTENSION_NAME identifies the Azure cluster-extension resource.
    • NAMESPACE is created in the Kubernetes cluster and contains extension components and Container Apps-managed resources. The appsNamespace setting must exactly match the extension release namespace.
    • CONNECTED_ENVIRONMENT_NAME becomes part of the default application domain. Use a DNS-compatible name that's unique within the resource group.

    Don't install unrelated workloads in the extension namespace.

    EXTENSION_NAME="appenv-ext"
NAMESPACE="appplat-ns"
CONNECTED_ENVIRONMENT_NAME="<connected-environment-name>"

  2. Install the Container Apps extension to your Azure Arc-connected cluster with Log Analytics enabled. Log Analytics can't be added to the extension later.

    az k8s-extension create \
    --resource-group $GROUP_NAME \
    --name $EXTENSION_NAME \
    --cluster-type connectedClusters \
    --cluster-name $CLUSTER_NAME \
    --extension-type 'Microsoft.App.Environment' \
    --release-train stable \
    --auto-upgrade-minor-version true \
    --scope cluster \
    --release-namespace $NAMESPACE \
    --configuration-settings "Microsoft.CustomLocation.ServiceAccount=default" \
    --configuration-settings "appsNamespace=${NAMESPACE}" \
    --configuration-settings "clusterName=${CONNECTED_ENVIRONMENT_NAME}" \
    --configuration-settings "logProcessor.appLogs.destination=log-analytics" \
    --config-protected-settings "logProcessor.appLogs.logAnalyticsConfig.customerId=${LOG_ANALYTICS_WORKSPACE_ID_ENC}" \
    --config-protected-settings "logProcessor.appLogs.logAnalyticsConfig.sharedKey=${LOG_ANALYTICS_KEY_ENC}"

    Note

    To install the extension without Log Analytics integration, remove the three logging-related parameters from the command.

    For a cluster that uses a custom load balancer, set loadBalancerIp to an address reserved for Container Apps ingress:

    --configuration-settings "loadBalancerIp=<LOAD_BALANCER_INGRESS_IP>"

    The address must be reachable by application clients and must not be assigned to another service. After installation, use kubectl get service -n $NAMESPACE -o wide to verify that the extension ingress service reports this address. Configure wildcard DNS only after the address is assigned. See DNS requirements.

    The following table describes the various --configuration-settings parameters when running the command:

    Parameter Description
    Microsoft.CustomLocation.ServiceAccount The service account created for the custom location. Set the value to default.
    appsNamespace The namespace used to create the app definitions and revisions. It must match that of the extension release namespace.
    clusterName The name of the Container Apps extension Kubernetes environment created against this extension.
    logProcessor.appLogs.destination Optional. Destination for application logs. Accepts log-analytics or none, choosing none disables platform logs.
    logProcessor.appLogs.logAnalyticsConfig.customerId Required only when logProcessor.appLogs.destination is set to log-analytics. The base64-encoded Log Analytics workspace ID. This parameter should be configured as a protected setting.
    logProcessor.appLogs.logAnalyticsConfig.sharedKey Required only when logProcessor.appLogs.destination is set to log-analytics. The base64-encoded Log Analytics workspace shared key. This parameter should be configured as a protected setting.
    loadBalancerIp The ingress IP of the load balancer.

  3. Save the id property of the Container Apps extension for later.

    EXTENSION_ID=$(az k8s-extension show \
    --cluster-type connectedClusters \
    --cluster-name $CLUSTER_NAME \
    --resource-group $GROUP_NAME \
    --name $EXTENSION_NAME \
    --query id \
    --output tsv)

  4. Wait for the extension to fully install before proceeding.

    az resource wait --ids $EXTENSION_ID --created --timeout 1200

az k8s-extension show \
    --cluster-type connectedClusters \
    --cluster-name $CLUSTER_NAME \
    --resource-group $GROUP_NAME \
    --name $EXTENSION_NAME \
    --query "{State:provisioningState,Version:currentVersion}" \
    --output table

    Continue only when State is Succeeded. If the command times out or reports a failure, see Extension installation fails or times out.

  5. Inspect the extension workloads, services, and recent events:

    kubectl get pods -n $NAMESPACE
kubectl get services -n $NAMESPACE -o wide
kubectl get events -n $NAMESPACE --sort-by=.lastTimestamp

    Don't create the custom location while an extension pod is pending, repeatedly restarting, or unexpectedly not ready. To learn more about these pods and their role in the system, see Azure Arc overview.

  6. If you configured Log Analytics, clear the local variables that contain the workspace key.

    unset LOG_ANALYTICS_KEY LOG_ANALYTICS_KEY_ENC

Create a custom location

The custom location is an Azure location that you assign to the Azure Container Apps connected environment.

  1. Set the following environment variables to the desired name of the custom location and for the ID of the Azure Arc-connected cluster.

    CUSTOM_LOCATION_NAME="my-custom-location" # Name of the custom location
CONNECTED_CLUSTER_ID=$(az connectedk8s show --resource-group $GROUP_NAME --name $CLUSTER_NAME --query id --output tsv)

  2. Create the custom location:

    az customlocation create \
    --resource-group $GROUP_NAME \
    --name $CUSTOM_LOCATION_NAME \
    --host-resource-id $CONNECTED_CLUSTER_ID \
    --namespace $NAMESPACE \
    --cluster-extension-ids $EXTENSION_ID

    Note

    If you have trouble creating a custom location on your cluster, you might need to enable the custom location feature on your cluster. Enable this feature when you sign in to the CLI by using a service principal or a Microsoft Entra user with restricted permissions on the cluster resource.

  3. Wait for the custom location to finish provisioning and save its resource ID.

    CUSTOM_LOCATION_ID=$(az customlocation show \
    --resource-group $GROUP_NAME \
    --name $CUSTOM_LOCATION_NAME \
    --query id \
    --output tsv)

az resource wait --ids $CUSTOM_LOCATION_ID --created --timeout 600

az customlocation show \
    --resource-group $GROUP_NAME \
    --name $CUSTOM_LOCATION_NAME \
    --query "{State:provisioningState,Host:hostResourceId,Namespace:namespace}" \
    --output table

    Check that State is Succeeded, Host identifies the connected cluster you want, and Namespace matches $NAMESPACE. If provisioning fails, see Custom location creation fails.

Create the Azure Container Apps connected environment

Before you can start creating apps in the custom location, you need an Azure Container Apps connected environment.

  1. Create the Container Apps connected environment:

    az containerapp connected-env create \
    --resource-group $GROUP_NAME \
    --name $CONNECTED_ENVIRONMENT_NAME \
    --custom-location $CUSTOM_LOCATION_ID \
    --location $LOCATION

  2. Wait for the connected environment to finish provisioning.

    CONNECTED_ENVIRONMENT_ID=$(az containerapp connected-env show \
    --resource-group $GROUP_NAME \
    --name $CONNECTED_ENVIRONMENT_NAME \
    --query id \
    --output tsv)

az resource wait --ids $CONNECTED_ENVIRONMENT_ID --created --timeout 900

az containerapp connected-env show \
    --resource-group $GROUP_NAME \
    --name $CONNECTED_ENVIRONMENT_NAME \
    --query "{State:properties.provisioningState,Location:location,CustomLocation:extendedLocation.name,Domain:properties.defaultDomain}" \
    --output yaml

    Continue only when State is Succeeded and CustomLocation matches the custom location you created in this tutorial. Save the displayed Domain for application DNS configuration. If provisioning fails, see Connected environment creation fails.

Next steps

Create a container app on Azure Arc

  • Last updated on