Preparación de la infraestructura para implementar Kafka en Azure Kubernetes Service (AKS)

Establecimiento de variables de entorno

• Antes de ejecutar los comandos de la CLI, establezca las siguientes variables de entorno para usarlas en esta guía con valores que cumplan sus requisitos:

  export RESOURCE_GROUP_NAME="rg-kafka"  
  export LOCATION="canadacentral"  
  export VNET_NAME="vnet-aks-kafka"  
  export SUBNET_NAME="node-subnet"  
  export AKS_CLUSTER_NAME="aks-kafka-cluster"  
  export AKS_TIER=standard
  export NAT_GATEWAY_NAME="nat-kafka"  
  export ADDRESS_SPACE="10.31.0.0/20"  
  export SUBNET_PREFIX="10.31.0.0/21"  
  export SYSTEM_NODE_COUNT_MIN=3
  export SYSTEM_NODE_COUNT_MAX=6
  export SYSTEM_NODE_VM_SIZE="Standard_D4ds_v5"   
  export KAFKA_NODE_COUNT_MIN=1  
  export KAFKA_NODE_COUNT_MAX=3 
  export KAFKA_NODE_COUNT=1
  export KAFKA_NODE_VM_SIZE="Standard_D16ds_v5"  
  export LOG_ANALYTICS_WORKSPACE_NAME="law-monitoring"  
  export DIAGNOSTIC_SETTINGS_NAME="aks-diagnostic-settings"  
  export ACR_NAME="aksacr123"  
  export ACR_SKU="Premium"  
  export USER_ASSIGNED_IDENTITY_NAME="uami-aks"  
  export KUBERNETES_VERSION="1.30.0"  
  export AAD_ADMIN_GROUP_OBJECT_IDS="<your-admin-group-object-id>"    
  export AAD_TENANT_ID="<your-tenant-id>"  
  export GRAFANA_NAME="grafana-kafka-aks"  
  export PROMETHEUS_WORKSPACE_NAME="prometheus-aks"
  

Implementaciones de red anteriores al clúster

Antes de implementar el clúster de AKS para Kafka, implemente los recursos de red de requisitos previos que admiten la implementación del clúster de AKS.

1. Cree un grupo de recursos con el comando az group create.

   az group create --name $RESOURCE_GROUP_NAME --location $LOCATION
   

2. Cree una red virtual mediante el comando az network vnet create.

   az network vnet create \
   --resource-group $RESOURCE_GROUP_NAME \
   --name $VNET_NAME \
   --address-prefix $ADDRESS_SPACE \
   --location $LOCATION
   

3. Cree una subred mediante el az network vnet subnet create comando .

   az network vnet subnet create \
   --resource-group $RESOURCE_GROUP_NAME \
   --vnet-name $VNET_NAME \
   --name $SUBNET_NAME \
   --address-prefix $SUBNET_PREFIX
   

4. Cree una dirección IP pública para la puerta de enlace NAT mediante el az network public-ip create comando .

   az network public-ip create \
   --resource-group $RESOURCE_GROUP_NAME \
   --name ${NAT_GATEWAY_NAME}-public-ip \
   --sku Standard \
   --location $LOCATION
   

5. Cree una puerta de enlace NAT con el az network nat gateway create comando .

   az network nat gateway create \
   --resource-group $RESOURCE_GROUP_NAME \
   --name $NAT_GATEWAY_NAME \
   --public-ip-addresses ${NAT_GATEWAY_NAME}-public-ip \
   --location $LOCATION
   

6. Asocie la puerta de enlace NAT a la subred del nodo mediante el comando az network vnet subnet update.

   az network vnet subnet update \
   --resource-group $RESOURCE_GROUP_NAME \
   --vnet-name $VNET_NAME \
   --name $SUBNET_NAME \
   --nat-gateway $NAT_GATEWAY_NAME
   

Implementaciones previas a la supervisión y gobernanza del clúster

Antes de implementar el clúster de AKS para Kafka, implemente los recursos de gobernanza y supervisión de requisitos previos que admiten la implementación del clúster de AKS.

1. Cree un área de trabajo de Log Analytics mediante el az monitor log-analytics workspace create comando .

   az monitor log-analytics workspace create \
   --resource-group $RESOURCE_GROUP_NAME \
   --workspace-name $LOG_ANALYTICS_WORKSPACE_NAME \
   --location $LOCATION
   

2. Cree un área de trabajo de Azure Monitor para Prometheus mediante el az monitor account create comando .

   az monitor account create \
   --resource-group $RESOURCE_GROUP_NAME \
   --name $PROMETHEUS_WORKSPACE_NAME \
   --location $LOCATION
   

3. Cree una instancia de Grafana administrada de Azure mediante el az grafana create comando .

   az grafana create \
   --resource-group $RESOURCE_GROUP_NAME \
   --name $GRAFANA_NAME \
   --location $LOCATION \
   --api-key Enabled \
   --deterministic-outbound-ip Enabled \
   --public-network-access Enabled \
   --grafana-major-version 11
   

   Nota:

   Azure Managed Grafana tiene redundancia de zona disponible en regiones seleccionadas. Si la región de destino tiene redundancia de zona, use el --zone-redundancy Enabled argumento .

4. Cree un registro de contenedor de Azure mediante el comando az acr create.

   az acr create \
   --resource-group $RESOURCE_GROUP_NAME \
   --name $ACR_NAME \
   --sku $ACR_SKU \
   --location $LOCATION \
   --admin-enabled false \
   --zone-redundancy Enabled
   

5. Use el comando az identity create para crear una identidad administrada asignada por el usuario.

   az identity create \
   --resource-group $RESOURCE_GROUP_NAME \
   --name $USER_ASSIGNED_IDENTITY_NAME \
   --location $LOCATION
   

6. Asigne permisos de RBAC a la identidad administrada de la instancia de Grafana mediante el az role assignment create comando .

   az role assignment create \
   --assignee $(az grafana show --resource-group $RESOURCE_GROUP_NAME --name $GRAFANA_NAME --query identity.principalId -o tsv) \
   --role "Monitoring Reader" --scope $(az group show --name $RESOURCE_GROUP_NAME --query id -o tsv)
   

Implementación de clústeres de AKS

Implemente el clúster de AKS con grupos de nodos dedicados para Kafka por zona de disponibilidad mediante la CLI de Azure.

1. Asigne el rol de colaborador de red a la identidad administrada asignada por el usuario para AKS mediante el az role assignment create comando .

   az role assignment create \
   --assignee $(az identity show --resource-group $RESOURCE_GROUP_NAME --name $USER_ASSIGNED_IDENTITY_NAME --query principalId -o tsv) \
   --role "Network Contributor" \
   --scope $(az group show --name $RESOURCE_GROUP_NAME --query id -o tsv)
   

2. Cree un clúster de AKS con el comando az aks create.

   az aks create \
   --name $AKS_CLUSTER_NAME \
   --aad-admin-group-object-ids $AAD_ADMIN_GROUP_OBJECT_IDS \
   --aad-tenant-id $AAD_TENANT_ID \
   --assign-identity $(az identity show --resource-group $RESOURCE_GROUP_NAME --name $USER_ASSIGNED_IDENTITY_NAME --query id -o tsv) \
   --attach-acr $(az acr show --resource-group $RESOURCE_GROUP_NAME --name $ACR_NAME --query id -o tsv) \
   --auto-upgrade-channel patch \
   --enable-aad \
   --enable-addons monitoring \
   --enable-azure-monitor-metrics \
   --enable-cluster-autoscaler \
   --enable-managed-identity \
   --enable-oidc-issuer \
   --enable-workload-identity \
   --kubernetes-version $KUBERNETES_VERSION \
   --load-balancer-sku standard \
   --location $LOCATION \
   --max-count $SYSTEM_NODE_COUNT_MAX \
   --max-pods 110 \
   --min-count $SYSTEM_NODE_COUNT_MIN \
   --network-dataplane cilium \
   --network-plugin azure \
   --network-plugin-mode overlay \
   --network-policy cilium \
   --node-osdisk-type Ephemeral \
   --node-os-upgrade-channel NodeImage \
   --node-vm-size $SYSTEM_NODE_VM_SIZE \
   --nodepool-labels "role=system" \
   --nodepool-name systempool \
   --nodepool-tags "env=production" \
   --os-sku AzureLinux \
   --outbound-type userAssignedNATGateway \
   --pod-cidr 10.244.0.0/16 \
   --resource-group $RESOURCE_GROUP_NAME \
   --tags "env=production" \
   --tier $AKS_TIER \
   --vnet-subnet-id $(az network vnet subnet show --resource-group $RESOURCE_GROUP_NAME --vnet-name $VNET_NAME --name $SUBNET_NAME --query id -o tsv) \
   --workspace-resource-id $(az monitor log-analytics workspace show --resource-group $RESOURCE_GROUP_NAME --workspace-name $LOG_ANALYTICS_WORKSPACE_NAME --query id -o tsv) \
   --zones 1 2 3
   

3. Cree un grupo de nodos adicional por zona de disponibilidad mediante un bucle for y el az aks nodepool add comando .

   for zone in 1 2 3; do
     az aks nodepool add \
     --cluster-name $AKS_CLUSTER_NAME \
     --enable-cluster-autoscaler \
     --labels app=kafka \
     --max-count $KAFKA_NODE_COUNT_MAX \
     --max-surge 10% \
     --min-count $KAFKA_NODE_COUNT_MIN \
     --node-count $KAFKA_NODE_COUNT \
     --mode User \
     --name "kafka$zone" \
     --node-osdisk-type Ephemeral \
     --node-vm-size $KAFKA_NODE_VM_SIZE \
     --os-sku AzureLinux \
     --resource-group $RESOURCE_GROUP_NAME \
     --vnet-subnet-id $(az network vnet subnet show --resource-group $RESOURCE_GROUP_NAME --vnet-name $VNET_NAME --name $SUBNET_NAME --query id -o tsv) \
     --zones $zone
   done
   

4. Habilite la integración de Prometheus y Grafana administrados de Azure mediante el az aks update comando .

   az aks update \
   --name $AKS_CLUSTER_NAME \
   --resource-group $RESOURCE_GROUP_NAME \
   --enable-azure-monitor-metrics \
   --azure-monitor-workspace-resource-id $(az monitor account show --resource-group $RESOURCE_GROUP_NAME --name $PROMETHEUS_WORKSPACE_NAME --query id -o tsv) \
   --grafana-resource-id $(az grafana show --resource-group $RESOURCE_GROUP_NAME --name $GRAFANA_NAME --query id -o tsv)
   

5. Opcional: configure la configuración de diagnóstico para el clúster de AKS mediante el az monitor diagnostic-settings create comando .

   az monitor diagnostic-settings create \
   --resource $(az aks show --resource-group $RESOURCE_GROUP_NAME --name $AKS_CLUSTER_NAME --query id -o tsv) \
   --name $DIAGNOSTIC_SETTINGS_NAME \
   --workspace $(az monitor log-analytics workspace show --resource-group $RESOURCE_GROUP_NAME --workspace-name $LOG_ANALYTICS_WORKSPACE_NAME --query id -o tsv) \
   --logs '[{"category": "kube-apiserver", "enabled": true}, {"category": "kube-audit", "enabled": true}, {"category": "kube-audit-admin", "enabled": true}, {"category": "kube-controller-manager", "enabled": true}, {"category": "kube-scheduler", "enabled": true}, {"category": "cluster-autoscaler", "enabled": true}, {"category": "cloud-controller-manager", "enabled": true}, {"category": "guard", "enabled": true}, {"category": "csi-azuredisk-controller", "enabled": true}, {"category": "csi-azurefile-controller", "enabled": true}, {"category": "csi-snapshot-controller", "enabled": true}]' \
   --metrics '[{"category": "AllMetrics", "enabled": true}]'
   

En esta sección, implementa un clúster de AKS y despliega los recursos de infraestructura necesarios mediante Terraform.

• Un clúster de AKS privado con un grupo de nodos por zona de disponibilidad mediante el módulo comprobado de Azure (AVM).
• Configuraciones de red virtual y subred.
• Puerta de enlace NAT para la conectividad saliente.
• Azure Container Registry con punto de conexión privado.
• Identidad administrada asignada por el usuario para AKS.
• Área de trabajo de Azure Monitor para métricas de Prometheus.
• Panel de Grafana administrado de Azure con integración de Prometheus.
• Grupos de nodos dedicados para cargas de trabajo de Kafka con etiquetas adecuadas.
• Controlador CSI de disco de Azure para volúmenes persistentes (habilitado de forma predeterminada).

1. Copie el variables.tf en su directorio de Terraform.

   variable "azure_subscription_id" {
   type        = string
   description = "The Azure subscription ID to use for the resources."
   
   }
   variable "enable_telemetry" {
   type        = bool
   default     = true
   description = "This variable controls whether or not telemetry is enabled for the module."
   }
   
   variable "kubernetes_cluster_name" {
   type        = string
   default     = "kafka-cluster"
   description = "The name of the Kubernetes cluster."
   }
   
   variable "kubernetes_version" {
   type        = string
   default     = "1.30"
   description = "The version of Kubernetes to use for the cluster."
   }
   
   variable "resource_group_name" {
   type        = string
   description = "The name of the resource group in which to create the resources."
   }
   
   variable "rbac_aad_admin_group_object_ids" {
   type        = list(string)
   description = "The object IDs of the Azure AD groups that should be granted admin access to the Kubernetes cluster."    
   }
   
   variable "location" {
   type        = string
   description = "The location in which to create the resources."
   }
   

2. Revise las variables y cree una kafka.tfvars según sea necesario. Actualice con valores que cumplan sus requisitos:

   # Replace placeholder values with your actual configuration
   
   azure_subscription_id = "00000000-0000-0000-0000-000000000000" # Replace with your actual subscription ID
   location              = "Canada Central"
   enable_telemetry      = true
   kubernetes_cluster_name = "kafka-aks-cluster"
   kubernetes_version    = "1.30"
   resource_group_name   = "rg-kafka-prod"
   rbac_aad_admin_group_object_ids = [
   "0000-0000-0000-0000", 
   # Add additional admin group object IDs as needed
   ]
   

3. Copie el main.tf en su directorio de Terraform.

   terraform {
     required_version = ">= 1.3.0"
     required_providers {
       azurerm = {
         source  = "hashicorp/azurerm"
         version = ">= 4, <5"
       }
     }
   }
   provider "azurerm" {
     features {
       resource_group {
         prevent_deletion_if_contains_resources = false
       }
     }
     subscription_id = var.azure_subscription_id
   }
   module "naming" {
     source  = "Azure/naming/azurerm"
     version = ">= 0.3.0"
   }
   
   resource "azurerm_user_assigned_identity" "this" {
     location            = var.location
     name                = "uami-${var.kubernetes_cluster_name}"
     resource_group_name = var.resource_group_name
   }
   
   data "azurerm_client_config" "current" {}
   
   module "avm-ptn-aks-production" {
     source = "github.com/Azure/terraform-azurerm-avm-ptn-aks-production"
     kubernetes_version  = "1.30"
     enable_telemetry    = var.enable_telemetry 
     name                = var.kubernetes_cluster_name
     resource_group_name = var.resource_group_name
     location = var.location 
     default_node_pool_vm_sku = "Standard_D8ds_v5"
     network = {
       name                = module.avm_res_network_virtualnetwork.name
       resource_group_name = var.resource_group_name
       node_subnet_id      = module.avm_res_network_virtualnetwork.subnets["subnet"].resource_id
       pod_cidr            = "192.168.0.0/16"
     }
     acr = {
       name                          = module.naming.container_registry.name_unique
       subnet_resource_id            = module.avm_res_network_virtualnetwork.subnets["private_link_subnet"].resource_id
       private_dns_zone_resource_ids = [azurerm_private_dns_zone.this.id]
     }
     managed_identities = {
       user_assigned_resource_ids = [
         azurerm_user_assigned_identity.this.id
       ]
     }
     rbac_aad_tenant_id = data.azurerm_client_config.current.tenant_id
     rbac_aad_admin_group_object_ids =  var.rbac_aad_admin_group_object_ids
     rbac_aad_azure_rbac_enabled = true
   
     node_pools = {
       kafka = {
         name                 = "kafka"
         vm_size              = "Standard_D16ds_v5"
         orchestrator_version = "1.30"
         max_count            = 3
         min_count            = 1
         os_sku               = "AzureLinux"
         mode                 = "User"
         os_disk_size_gb      = 128
         labels = {
           "app" = "kafka"
         }
       }
     }
   }
   
   resource "azurerm_private_dns_zone" "this" {
     name                = "privatelink.azurecr.io"
     resource_group_name = var.resource_group_name
   }
   
   resource "azurerm_nat_gateway" "this" {
     location            = var.location
     name                = module.naming.nat_gateway.name_unique
     resource_group_name = var.resource_group_name
   }
   
   resource "azurerm_public_ip" "this" {
     name                = module.naming.public_ip.name_unique
     location            = var.location
     resource_group_name = var.resource_group_name
     allocation_method   = "Static"
     sku                 = "Standard"
   }
   
   resource "azurerm_nat_gateway_public_ip_association" "this" {
     nat_gateway_id       = azurerm_nat_gateway.this.id
     public_ip_address_id = azurerm_public_ip.this.id  
   }
   
   module "avm_res_network_virtualnetwork" {
     source  = "Azure/avm-res-network-virtualnetwork/azurerm"
     version = "0.7.1"
   
     address_space       = ["10.31.0.0/16"]
     location            = var.location
     name                = "vnet-aks-lab"
     resource_group_name = var.resource_group_name
     subnets = {
       "subnet" = {
         name             = "nodecidr"
         address_prefixes = ["10.31.0.0/17"]
         nat_gateway = {
           id = azurerm_nat_gateway.this.id
         }
         private_link_service_network_policies_enabled = false
       }
       "private_link_subnet" = {
         name             = "private_link_subnet"
         address_prefixes = ["10.31.129.0/24"]
       }
     }
   }
   
   resource "azurerm_monitor_workspace" "this" {
     name                = "prometheus-aks"
     location            = var.location
     resource_group_name = var.resource_group_name
   }
   
   resource "azurerm_monitor_data_collection_endpoint" "dataCollectionEndpoint" {
     name                = "prom-aks-endpoint"
     location            = var.location
     resource_group_name = var.resource_group_name
     kind                = "Linux"
   }
   
   resource "azurerm_monitor_data_collection_rule" "dataCollectionRule" {
     name      = "prom-aks-dcr"
     location            = var.location
     resource_group_name = var.resource_group_name
     data_collection_endpoint_id = azurerm_monitor_data_collection_endpoint.dataCollectionEndpoint.id
     kind                        = "Linux"
     description = "DCR for Azure Monitor Metrics Profile (Managed Prometheus)"
     destinations {
       monitor_account {
         monitor_account_id = azurerm_monitor_workspace.this.id
         name               = "PrometheusAzMonitorAccount"
       }
     }
     data_flow {
       streams      = ["Microsoft-PrometheusMetrics"]
       destinations = ["PrometheusAzMonitorAccount"]
     }
     data_sources {
       prometheus_forwarder {
         streams = ["Microsoft-PrometheusMetrics"]
         name    = "PrometheusDataSource"
       }
     }
   
   }
   
   resource "azurerm_monitor_data_collection_rule_association" "dataCollectionRuleAssociation" {
     name                    = "prom-aks-dcra"
     target_resource_id      = module.avm-ptn-aks-production.resource_id
     data_collection_rule_id = azurerm_monitor_data_collection_rule.dataCollectionRule.id
     description             = "Association of data collection rule. Deleting this association will break the data collection for this AKS Cluster."
   }
   
   resource "azurerm_dashboard_grafana" "this" {
     name                              = "grafana-kafka-aks"
     location                          = var.location
     resource_group_name               = var.resource_group_name
     api_key_enabled                   = true
     deterministic_outbound_ip_enabled = true
     public_network_access_enabled     = true
     grafana_major_version             = 11
   
     azure_monitor_workspace_integrations {
       resource_id = azurerm_monitor_workspace.this.id
     }
   
     identity {
       type = "SystemAssigned"
     }
   }
   
   data "azurerm_resource_group" "current" {
     name       = var.resource_group_name
     depends_on = [azurerm_dashboard_grafana.this]
   }
   
   resource "azurerm_role_assignment" "grafana_monitoring_reader" {
     scope                            = data.azurerm_resource_group.current.id
     role_definition_name             = "Monitoring Reader"
     principal_id                     = azurerm_dashboard_grafana.this.identity[0].principal_id
     skip_service_principal_aad_check = true
   }
   
   resource "azurerm_kubernetes_cluster_extension" "container_storage" {
     name           = "microsoft-azurecontainerstorage"
     cluster_id     = module.avm-ptn-aks-production.resource_id
     extension_type = "microsoft.azurecontainerstorage"
     configuration_settings = {
       "enable-azure-container-storage" : "azureDisk",
     }
   }
   

4. Inicialice Terraform mediante el terraform init comando .

   terraform init  
   

5. Cree un plan de implementación mediante el terraform plan comando .

   terraform plan -var-file="kafka.tfvars"
   

6. Aplique la configuración mediante el terraform apply comando .

   terraform apply -var-file="kafka.tfvars"