Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Applies to: ✔️ Hosted control planes architecture
This article describes how to create an Azure Red Hat OpenShift with hosted control planes (HCP) cluster by using the Azure CLI or a Bicep file.
The Bicep file defines all of the Azure resources required for the cluster, including the network infrastructure, managed identities, role assignments, cluster configuration, and node pool.
The CLI commands create all of the Azure resources required for the cluster, including the network infrastructure, managed identities, role assignments, cluster configuration, and node pool.
Prerequisites
- You planned your cluster configuration
- You planned your cluster network
- If you're using an Azure Managed HSM for etcd encryption, verify the following prerequisites. For more information, see Quickstart: Provision and activate a Managed HSM using Azure CLI.
- The Managed HSM is activated. An unactivated HSM can't process key operations.
- Purge protection is enabled on the Managed HSM.
- The Managed HSM setting
AllowKeyManagementOperationsThroughARMis enabled. Enable it by usingaz keyvault setting update --hsm-name <managed-hsm-name> --name AllowKeyManagementOperationsThroughARM --value true.
Prepare your environment
Before you create the cluster, verify that your Azure environment meets the following requirements.
Azure CLI
- Azure CLI version 2.93.0 or higher. Use
az --versionto find your installed version. If you need to install or upgrade, see Install Azure CLI.
Resource quota
Azure Red Hat OpenShift with hosted control planes requires at least 20 cores to create and run a cluster. The default Azure resource quota for a new Azure subscription doesn't meet this requirement. To request an increase in your resource limit, see Standard quota: Increase limits by VM series. Verify that you have enough quota for the VM family you choose from the supported worker node VM sizes.
To check the current quota for the default virtual machine family:
LOCATION=eastus
az vm list-usage -l $LOCATION \
--query "[?contains(name.value, 'standardDSv5Family')]" -o table
Permissions
You need Contributor and User Access Administrator permissions, or Owner permissions, on the resource group or subscription where you create the cluster. These permissions are required to create the resource group, virtual network, managed identities, and role assignments.
For more information, see Verify your permissions.
Register resource providers
Register the following resource providers in your Azure subscription:
Microsoft.RedHatOpenShiftMicrosoft.ComputeMicrosoft.StorageMicrosoft.Authorization
To check whether a resource provider is registered:
az provider list --query "[?namespace=='Microsoft.RedHatOpenShift'].registrationState" \
--output table
If you find any resource providers that aren't registered, register them.
az provider register --namespace Microsoft.RedHatOpenShift --wait
az provider register --namespace Microsoft.Compute --wait
az provider register --namespace Microsoft.Storage --wait
az provider register --namespace Microsoft.Authorization --wait
For more information about how to register resource providers, see Register the resource providers.
Set environment variables
Set the following environment variables to define the names of the resources that you create. Replace the placeholder values with your own.
# Azure resource location and subscription
LOCATION="<region>"
SUBSCRIPTION_ID="$(az account show --query id --output tsv)"
CUSTOMER_RG_NAME="<resource-group-name>"
# Network resources
CUSTOMER_NSG="<nsg-name>"
CUSTOMER_VNET_NAME="<vnet-name>"
CUSTOMER_VNET_SUBNET1="<worker-subnet-name>"
CUSTOMER_VNET_INTEGRATION_SUBNET_NAME="<vnet-integration-subnet-name>"
# Cluster and node pool
CLUSTER_NAME="<cluster-name>"
MANAGED_RESOURCE_GROUP="${CLUSTER_NAME}-managed-rg"
NP_NAME="<node-pool-name>"
CLUSTER_VERSION="<major.minor>" # Example: 4.22
NP_VERSION="<major.minor.patch>" # Example: 4.22.1
Note
For more information about cluster and node pool version requirements, review the supported version combinations.
Note
For a list of the supported regions, see Choose the cluster region.
Create a resource group
Create a resource group to hold the cluster resource, virtual network, and managed identities.
az group create \
--name "${CUSTOMER_RG_NAME}" \
--subscription "${SUBSCRIPTION_ID}" \
--location "${LOCATION}"
(Managed HSM only) Create the KMS managed identity and role assignments
If you're using Azure Managed HSM for etcd encryption, create a managed identity and grant it the Managed HSM Crypto User role. You also need to grant the same role to the identity that runs the Bicep deployment, so it can create the encryption key in the HSM.
Note
If you're using Azure Key Vault (the default), skip this section. The Bicep template assigns the Key Vault role directly through Azure RBAC. Managed HSM uses its own per-HSM RBAC system that you can't configure through ARM role assignments. Complete the following steps before deploying the template.
Create the KMS managed identity. The Bicep template references this identity as a pre-existing resource for the cluster's KMS operator.
az identity create \ --name "${CLUSTER_NAME}-cp-kms" \ --resource-group "${CUSTOMER_RG_NAME}"Get the principal ID of the KMS managed identity. You'll use this ID in the next step.
KMS_MI_PID=$(az identity show \ --name "${CLUSTER_NAME}-cp-kms" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query principalId --output tsv)Grant the KMS managed identity the Managed HSM Crypto User role. This role enables the cluster's KMS operator to perform key operations (wrap, unwrap, sign, verify) on keys in the HSM at runtime.
az keyvault role assignment create \ --hsm-name "<managed-hsm-name>" \ --assignee-object-id "${KMS_MI_PID}" \ --assignee-principal-type ServicePrincipal \ --role "Managed HSM Crypto User" \ --scope /keysGrant your deploying identity the Managed HSM Crypto User role. This role allows the Bicep deployment to create the etcd encryption key inside the HSM. If you skip this step, the deployment fails with an
AccessDeniederror onMicrosoft.KeyVault/managedHsm/keys/read/action.For a signed-in user:
DEPLOYER_OID=$(az ad signed-in-user show --query id --output tsv) az keyvault role assignment create \ --hsm-name "<managed-hsm-name>" \ --assignee-object-id "${DEPLOYER_OID}" \ --assignee-principal-type User \ --role "Managed HSM Crypto User" \ --scope /keysNote
If you're deploying from a CI/CD pipeline or automation script that authenticates as a service principal, set
--assignee-principal-type ServicePrincipal.
Create the Bicep file
Create a file named azuredeploy.bicep with the following content.
This Bicep template defines the Azure resources required for the cluster, including the network security group, virtual network, managed identities, role assignments, key store (Azure Key Vault or Managed HSM), KMS encryption key, cluster, and node pool. When using Managed HSM, the KMS managed identity must be created separately (see the preceding section).
The template deploys a cluster with OVN-Kubernetes networking with default CIDR ranges, customer-managed etcd encryption, load balancer outbound connectivity, and a single node pool with two Standard_D8s_v6 worker nodes.
To customize the cluster or node pool configuration, see Bicep file reference.
Note
If you add custom rules to the network security group (NSG), make sure that the NSG allows the traffic described in Required network security group traffic.
@description('Network Security Group Name')
param customerNsgName string
@description('Virtual Network Name')
param customerVnetName string
@description('Subnet Name')
param customerVnetSubnetName string
@description('Virtual Network Integration Subnet Name')
param customerVirtualNetworkIntegrationSubnetName string
@description('Name of the cluster')
param clusterName string
@description('The cluster managed resource group name')
param managedResourceGroupName string
@description('The name of the node pool')
param nodePoolName string
@description('The OpenShift version for the cluster (X.Y format, e.g. 4.20)')
param clusterVersion string
@description('The OpenShift version for the node pool (X.Y.Z format, e.g. 4.20.8)')
param nodePoolVersion string
@description('Deploy ARO HCP with a private key store')
param privateKeyVault bool
@description('Key store type for customer-managed etcd encryption')
@allowed([
'KeyVault'
'ManagedHSM'
])
param keyVaultType string = 'KeyVault'
@description('Name of a pre-existing, activated Managed HSM (required when keyVaultType is ManagedHSM)')
param customerMhsmName string = ''
@description('Name of the pre-created KMS managed identity (required when keyVaultType is ManagedHSM)')
param kmsIdentityName string = ''
@description('API server visibility')
@allowed([
'Public'
'Private'
])
param apiVisibility string = 'Public'
@description('Default ingress type')
@allowed([
'Public'
'Private'
])
param ingressType string = 'Public'
@description('Cryptographic restrictions for kernel and userspace libraries (immutable after creation)')
@allowed(['None', 'FIPS'])
param cryptoRestrictions string = 'None'
var etcdEncryptionKeyName = 'etcd-data-kms-encryption-key'
var randomKeyVaultSuffix = toLower(uniqueString(resourceGroup().id))
var customerKeyVaultName string = 'cust-kv-${randomKeyVaultSuffix}'
var addressPrefix = '10.0.0.0/16'
var subnetPrefix = '10.0.0.0/24'
var virtualNetworkIntegrationSubnetPrefix = '10.0.1.0/24'
var useKeyVault = keyVaultType == 'KeyVault'
var useMhsm = keyVaultType == 'ManagedHSM'
// Network Security Group
resource customerNsg 'Microsoft.Network/networkSecurityGroups@2023-05-01' = {
name: customerNsgName
location: resourceGroup().location
}
// Virtual network with worker subnet and VNet integration subnet
resource customerVnet 'Microsoft.Network/virtualNetworks@2023-05-01' = {
name: customerVnetName
location: resourceGroup().location
properties: {
addressSpace: {
addressPrefixes: [
addressPrefix
]
}
subnets: [
{
name: customerVnetSubnetName
properties: {
addressPrefix: subnetPrefix
networkSecurityGroup: {
id: customerNsg.id
}
}
}
{
name: customerVirtualNetworkIntegrationSubnetName
properties: {
addressPrefix: virtualNetworkIntegrationSubnetPrefix
networkSecurityGroup: {
id: customerNsg.id
}
delegations: [
{
name: 'aro-hcp-delegation'
properties: {
serviceName: 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters'
}
}
]
}
}
]
}
}
resource subnet 'Microsoft.Network/virtualNetworks/subnets@2022-07-01' existing = {
name: customerVnetSubnetName
parent: customerVnet
}
resource vnetIntegrationSubnet 'Microsoft.Network/virtualNetworks/subnets@2022-07-01' existing = {
name: customerVirtualNetworkIntegrationSubnetName
parent: customerVnet
}
resource customerKeyVault 'Microsoft.KeyVault/vaults@2024-12-01-preview' = if (useKeyVault) {
name: customerKeyVaultName
location: resourceGroup().location
properties: {
enableRbacAuthorization: true
tenantId: subscription().tenantId
publicNetworkAccess: privateKeyVault ? 'Disabled' : 'Enabled'
sku: {
family: 'A'
name: 'standard'
}
}
}
resource etcdEncryptionKey 'Microsoft.KeyVault/vaults/keys@2024-12-01-preview' = if (useKeyVault) {
parent: customerKeyVault
name: 'etcd-data-kms-encryption-key'
properties: {
kty: 'RSA'
keySize: 2048
}
}
resource customerMhsm 'Microsoft.KeyVault/managedHsms@2024-12-01-preview' existing = if (useMhsm) {
name: customerMhsmName
}
resource etcdEncryptionKeyMhsm 'Microsoft.KeyVault/managedHsms/keys@2024-12-01-preview' = if (useMhsm) {
parent: customerMhsm
name: 'etcd-data-kms-encryption-key'
properties: {
kty: 'RSA-HSM'
keySize: 2048
}
}
// Private endpoint resources for Azure Key Vault
resource privateEndpointDnsZone 'Microsoft.Network/privateDnsZones@2020-06-01' = if (privateKeyVault && useKeyVault) {
name: 'privatelink.vaultcore.azure.net'
location: 'global'
properties: {}
dependsOn: [
privateEndpoint
]
}
resource privateEndpoint 'Microsoft.Network/privateEndpoints@2023-09-01' = if (privateKeyVault && useKeyVault) {
name: 'kv-private-endpoint'
properties: {
privateLinkServiceConnections: [
{
name: 'kv-private-endpoint'
properties: {
privateLinkServiceId: customerKeyVault.id
groupIds: ['vault']
}
}
]
subnet: {
id: customerVnet.properties.subnets[0].id
}
}
location: resourceGroup().location
}
resource privateEndpointDnsGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-09-01' = if (privateKeyVault && useKeyVault) {
name: 'kv-private-ep-dns-group'
parent: privateEndpoint
properties: {
privateDnsZoneConfigs: [
{
name: 'config1'
properties: {
privateDnsZoneId: privateEndpointDnsZone.id
}
}
]
}
dependsOn: [
privateDnsZoneVnetLink
]
}
resource privateDnsZoneVnetLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = if (privateKeyVault && useKeyVault) {
name: uniqueString('kv-private-dns-zone-link')
parent: privateEndpointDnsZone
location: 'global'
properties: {
registrationEnabled: false
virtualNetwork: {
id: customerVnet.id
}
}
}
// Private endpoint resources for Managed HSM
resource mhsmPrivateEndpointDnsZone 'Microsoft.Network/privateDnsZones@2020-06-01' = if (privateKeyVault && useMhsm) {
name: 'privatelink.managedhsm.azure.net'
location: 'global'
properties: {}
dependsOn: [
mhsmPrivateEndpoint
]
}
resource mhsmPrivateEndpoint 'Microsoft.Network/privateEndpoints@2023-09-01' = if (privateKeyVault && useMhsm) {
name: 'mhsm-private-endpoint'
properties: {
privateLinkServiceConnections: [
{
name: 'mhsm-private-endpoint'
properties: {
privateLinkServiceId: customerMhsm.id
groupIds: ['managedhsm']
}
}
]
subnet: {
id: customerVnet.properties.subnets[0].id
}
}
location: resourceGroup().location
}
resource mhsmPrivateEndpointDnsGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-09-01' = if (privateKeyVault && useMhsm) {
name: 'mhsm-private-ep-dns-group'
parent: mhsmPrivateEndpoint
properties: {
privateDnsZoneConfigs: [
{
name: 'config1'
properties: {
privateDnsZoneId: mhsmPrivateEndpointDnsZone.id
}
}
]
}
dependsOn: [
mhsmPrivateDnsZoneVnetLink
]
}
resource mhsmPrivateDnsZoneVnetLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = if (privateKeyVault && useMhsm) {
name: uniqueString('mhsm-private-dns-zone-link')
parent: mhsmPrivateEndpointDnsZone
location: 'global'
properties: {
registrationEnabled: false
virtualNetwork: {
id: customerVnet.id
}
}
}
//
// Control plane identities
//
// Reader
var readerRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'acdd72a7-3385-48ef-bd42-f606fba81ae7'
)
//
// Cluster API Azure managed identity
//
resource clusterApiAzureMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-cluster-api-azure'
location: resourceGroup().location
}
// Azure Red Hat OpenShift with hosted control planes Cluster API Provider
var hcpClusterApiProviderRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'88366f10-ed47-4cc0-9fab-c8a06148393e'
)
resource hcpClusterApiProviderRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, clusterApiAzureMi.id, hcpClusterApiProviderRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: clusterApiAzureMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpClusterApiProviderRoleId
}
}
resource serviceManagedIdentityReaderOnClusterApiAzureMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, clusterApiAzureMi.id)
scope: clusterApiAzureMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// KMS managed identity
//
resource kmsMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = if (useKeyVault) {
name: '${clusterName}-cp-kms'
location: resourceGroup().location
}
resource kmsMiExisting 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' existing = if (useMhsm) {
name: kmsIdentityName
}
var keyVaultCryptoUserRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'12338af0-0e69-4776-bea7-57ae8d297424'
)
resource keyVaultCryptoUserToKeyVaultRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = if (useKeyVault) {
name: guid(resourceGroup().id, kmsMi.id, keyVaultCryptoUserRoleId, customerKeyVault.id)
scope: customerKeyVault
properties: {
principalId: kmsMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: keyVaultCryptoUserRoleId
}
}
resource serviceManagedIdentityReaderOnKmsMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = if (useKeyVault) {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, kmsMi.id)
scope: kmsMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
resource serviceManagedIdentityReaderOnKmsMiMhsm 'Microsoft.Authorization/roleAssignments@2022-04-01' = if (useMhsm) {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, kmsMiExisting.id)
scope: kmsMiExisting
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Control plane operator managed identity
//
resource controlPlaneMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-control-plane'
location: resourceGroup().location
}
// Azure Red Hat OpenShift with hosted control planes Control Plane Operator
var hcpControlPlaneOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'fc0c873f-45e9-4d0d-a7d1-585aab30c6ed'
)
resource hcpControlPlaneOperatorVnetRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, controlPlaneMi.id, hcpControlPlaneOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: controlPlaneMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpControlPlaneOperatorRoleId
}
}
resource hcpControlPlaneOperatorNsgRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, controlPlaneMi.id, hcpControlPlaneOperatorRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: controlPlaneMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpControlPlaneOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnControlPlaneMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, controlPlaneMi.id)
scope: controlPlaneMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Cloud controller manager managed identity
//
resource cloudControllerManagerMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-cloud-controller-manager'
location: resourceGroup().location
}
// Azure Red Hat OpenShift Cloud Controller Manager
var cloudControllerManagerRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'a1f96423-95ce-4224-ab27-4e3dc72facd4'
)
resource cloudControllerManagerRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: cloudControllerManagerMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: cloudControllerManagerRoleId
}
}
resource cloudControllerManagerRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: cloudControllerManagerMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: cloudControllerManagerRoleId
}
}
resource serviceManagedIdentityReaderOnCloudControllerManagerMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, cloudControllerManagerMi.id)
scope: cloudControllerManagerMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Ingress managed identity
//
resource ingressMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-ingress'
location: resourceGroup().location
}
// Azure Red Hat OpenShift Cluster Ingress Operator
var ingressOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'0336e1d3-7a87-462b-b6db-342b63f7802c'
)
// Azure Red Hat OpenShift Image Registry Operator
var imageRegistryOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'8b32b316-c2f5-4ddf-b05b-83dacd2d08b5'
)
resource ingressOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, ingressMi.id, ingressOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: ingressMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: ingressOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnIngressMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, ingressMi.id)
scope: ingressMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Disk CSI driver managed identity
//
resource diskCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-disk-csi-driver'
location: resourceGroup().location
}
resource serviceManagedIdentityReaderOnDiskCsiDriverMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, diskCsiDriverMi.id)
scope: diskCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// File CSI driver managed identity
//
resource fileCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-file-csi-driver'
location: resourceGroup().location
}
// Azure Red Hat OpenShift File Storage Operator
var fileStorageOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'0d7aedc0-15fd-4a67-a412-efad370c947e'
)
resource fileStorageOperatorRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: fileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource fileStorageOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: fileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnFileCsiDriverMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, fileCsiDriverMi.id)
scope: fileCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Image registry managed identity
//
resource imageRegistryMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-image-registry'
location: resourceGroup().location
}
resource serviceManagedIdentityReaderOnImageRegistryMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, imageRegistryMi.id)
scope: imageRegistryMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
resource imageRegistryOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, imageRegistryMi.id, imageRegistryOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: imageRegistryMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: imageRegistryOperatorRoleId
}
}
//
// Cloud network config managed identity
//
resource cloudNetworkConfigMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-cloud-network-config'
location: resourceGroup().location
}
// Azure Red Hat OpenShift Network Operator
var networkOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'be7a6435-15ae-4171-8f30-4a343eff9e8f'
)
resource networkOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudNetworkConfigMi.id, networkOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: cloudNetworkConfigMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: networkOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnCloudNetworkMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, cloudNetworkConfigMi.id)
scope: cloudNetworkConfigMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Data plane identities
//
// Azure Red Hat OpenShift Federated Credential
var federatedCredentialsRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'ef318e2a-8334-4a05-9e4a-295a196c6a6e'
)
resource dpDiskCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-dp-disk-csi-driver'
location: resourceGroup().location
}
resource dpDiskCsiDriverMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpDiskCsiDriverMi.id, federatedCredentialsRoleId)
scope: dpDiskCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: federatedCredentialsRoleId
}
}
resource dpFileCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-dp-file-csi-driver'
location: resourceGroup().location
}
resource dpFileCsiDriverMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpFileCsiDriverMi.id, federatedCredentialsRoleId)
scope: dpFileCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: federatedCredentialsRoleId
}
}
resource dpFileCsiDriverFileStorageOperatorRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: dpFileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource dpFileCsiDriverFileStorageOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: dpFileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource dpImageRegistryMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-dp-image-registry'
location: resourceGroup().location
}
resource dpImageRegistryMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpImageRegistryMi.id, federatedCredentialsRoleId)
scope: dpImageRegistryMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: federatedCredentialsRoleId
}
}
resource dpImageRegistryOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpImageRegistryMi.id, imageRegistryOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: dpImageRegistryMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: imageRegistryOperatorRoleId
}
}
//
// Service managed identity
//
resource serviceManagedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-service-managed-identity'
location: resourceGroup().location
}
// Azure Red Hat OpenShift with hosted control planes Service Managed Identity
var hcpServiceManagedIdentityRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'c0ff367d-66d8-445e-917c-583feb0ef0d4'
)
resource serviceManagedIdentityRoleAssignmentVnet 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, hcpServiceManagedIdentityRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpServiceManagedIdentityRoleId
}
}
resource serviceManagedIdentityRoleAssignmentNSG 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, hcpServiceManagedIdentityRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpServiceManagedIdentityRoleId
}
}
// Cluster resource
resource hcp 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters@2026-10-01' = {
name: clusterName
location: resourceGroup().location
properties: {
version: {
id: clusterVersion
channelGroup: 'stable'
}
dns: {}
network: {
networkType: 'OVNKubernetes'
podCidr: '10.128.0.0/14'
serviceCidr: '172.30.0.0/16'
machineCidr: '10.0.0.0/16'
hostPrefix: 23
}
etcd: {
dataEncryption: {
keyManagementMode: 'CustomerManaged'
customerManaged: {
encryptionType: 'KMS'
kms: {
activeKey: {
name: etcdEncryptionKeyName
version: useKeyVault
? last(split(etcdEncryptionKey.properties.keyUriWithVersion, '/'))
: last(split(etcdEncryptionKeyMhsm.properties.keyUriWithVersion, '/'))
}
vaultName: useKeyVault ? customerKeyVaultName : customerMhsmName
keyVaultType: keyVaultType
visibility: privateKeyVault ? 'Private' : 'Public'
}
}
}
}
api: {
visibility: apiVisibility
}
ingress: {
type: ingressType
}
cryptoRestrictions: cryptoRestrictions
clusterImageRegistry: {
state: 'Enabled'
}
platform: {
managedResourceGroup: managedResourceGroupName
subnetId: subnet.id
vnetIntegrationSubnetId: vnetIntegrationSubnet.id
outboundType: 'LoadBalancer'
networkSecurityGroupId: customerNsg.id
operatorsAuthentication: {
userAssignedIdentities: {
controlPlaneOperators: {
'cluster-api-azure': clusterApiAzureMi.id
'control-plane': controlPlaneMi.id
'cloud-controller-manager': cloudControllerManagerMi.id
ingress: ingressMi.id
'disk-csi-driver': diskCsiDriverMi.id
'file-csi-driver': fileCsiDriverMi.id
'image-registry': imageRegistryMi.id
'cloud-network-config': cloudNetworkConfigMi.id
kms: useKeyVault ? kmsMi.id : kmsMiExisting.id
}
dataPlaneOperators: {
'disk-csi-driver': dpDiskCsiDriverMi.id
'file-csi-driver': dpFileCsiDriverMi.id
'image-registry': dpImageRegistryMi.id
}
serviceManagedIdentity: serviceManagedIdentity.id
}
}
}
}
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${serviceManagedIdentity.id}': {}
'${clusterApiAzureMi.id}': {}
'${controlPlaneMi.id}': {}
'${cloudControllerManagerMi.id}': {}
'${ingressMi.id}': {}
'${diskCsiDriverMi.id}': {}
'${fileCsiDriverMi.id}': {}
'${imageRegistryMi.id}': {}
'${cloudNetworkConfigMi.id}': {}
'${useKeyVault ? kmsMi.id : kmsMiExisting.id}': {}
}
}
dependsOn: [
hcpClusterApiProviderRoleVnetAssignment
keyVaultCryptoUserToKeyVaultRoleAssignment
hcpControlPlaneOperatorVnetRoleAssignment
hcpControlPlaneOperatorNsgRoleAssignment
cloudControllerManagerRoleNsgAssignment
cloudControllerManagerRoleVnetAssignment
ingressOperatorRoleVnetAssignment
fileStorageOperatorRoleNsgAssignment
fileStorageOperatorRoleVnetAssignment
networkOperatorRoleVnetAssignment
dpDiskCsiDriverMiFederatedCredentialsRoleAssignment
dpFileCsiDriverMiFederatedCredentialsRoleAssignment
dpImageRegistryMiFederatedCredentialsRoleAssignment
serviceManagedIdentityRoleAssignmentVnet
serviceManagedIdentityRoleAssignmentNSG
dpFileCsiDriverFileStorageOperatorRoleNsgAssignment
dpFileCsiDriverFileStorageOperatorRoleVnetAssignment
imageRegistryOperatorRoleVnetAssignment
dpImageRegistryOperatorRoleVnetAssignment
serviceManagedIdentityReaderOnControlPlaneMi
serviceManagedIdentityReaderOnCloudControllerManagerMi
serviceManagedIdentityReaderOnIngressMi
serviceManagedIdentityReaderOnDiskCsiDriverMi
serviceManagedIdentityReaderOnFileCsiDriverMi
serviceManagedIdentityReaderOnImageRegistryMi
serviceManagedIdentityReaderOnCloudNetworkMi
serviceManagedIdentityReaderOnClusterApiAzureMi
serviceManagedIdentityReaderOnKmsMi
serviceManagedIdentityReaderOnKmsMiMhsm
]
}
// Node pool resource
resource nodepool 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters/nodePools@2026-10-01' = {
parent: hcp
name: nodePoolName
location: resourceGroup().location
properties: {
version: {
id: nodePoolVersion
channelGroup: 'stable'
}
platform: {
subnetId: hcp.properties.platform.subnetId
vmSize: 'Standard_D8s_v6'
osDisk: {
sizeGiB: 64
diskStorageAccountType: 'StandardSSD_LRS'
}
}
replicas: 2
}
}
Deploy the Bicep file
Deploy the Bicep template to create the cluster and all required resources.
az deployment group create \
--name 'aro-hcp' \
--subscription "${SUBSCRIPTION_ID}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--template-file azuredeploy.bicep \
--parameters \
customerNsgName="${CUSTOMER_NSG}" \
customerVnetName="${CUSTOMER_VNET_NAME}" \
customerVirtualNetworkIntegrationSubnetName="${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \
customerVnetSubnetName="${CUSTOMER_VNET_SUBNET1}" \
clusterName="${CLUSTER_NAME}" \
managedResourceGroupName="${MANAGED_RESOURCE_GROUP}" \
nodePoolName="${NP_NAME}" \
clusterVersion="${CLUSTER_VERSION}" \
nodePoolVersion="${NP_VERSION}" \
apiVisibility="<Public-or-Private>" \
ingressType="<Public-or-Private>" \
privateKeyVault=<true-or-false> \
keyVaultType="<KeyVault-or-ManagedHSM>" \
customerMhsmName="<managed-hsm-name-or-empty-for-KeyVault>" \
kmsIdentityName="${CLUSTER_NAME}-cp-kms" \
cryptoRestrictions="<None-or-FIPS>"
Set the following deployment parameters:
apiVisibility-Publicfor an internet-accessible API server, orPrivatefor an API server reachable only through a private network connection.ingressType-Publicfor internet-accessible application routes, orPrivatefor application routes reachable only through a private network connection.privateKeyVault-trueto deploy private endpoint resources and set vault visibility toPrivateon your cluster, orfalseto use public connectivity. For Azure Key Vault, the template also disables public network access when this value istrue. For an existing Managed HSM, you must disable public network access yourself before deploying withprivateKeyVault=true; the template doesn't modify the HSM's network settings.keyVaultType-KeyVault(default) to use a standard Azure Key Vault, orManagedHSMto use Azure Key Vault Managed HSM for FIPS 140-2 Level 3 validated, single-tenant key storage. For more information, see Choose the key store type.cryptoRestrictions-FIPSto restrict the cluster to FIPS-validated cryptographic modules, orNone(default) to deploy without FIPS restrictions. You can't changecryptoRestrictionsafter cluster creation. The control plane is always hosted on a FIPS-enabled cluster regardless of this setting;cryptoRestrictionsonly controls whether the worker node pools use FIPS-validated cryptographic modules.
The deployment typically takes 15 to 20 minutes to complete.
Verify the deployment
After the deployment finishes, verify that the cluster and node pool were created successfully.
Verify that the cluster was created successfully.
az aro hcp cluster show \ --name "${CLUSTER_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query "properties.provisioningState" \ --output tsvWhen the cluster is ready, the output is
Succeeded.Verify that the node pool was created successfully.
az aro hcp cluster nodepool show \ --cluster-name "${CLUSTER_NAME}" \ --name "${NP_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query "properties.provisioningState" \ --output tsvWhen the node pool is ready, the output is
Succeeded.
Create the network infrastructure
Create a network security group, a virtual network with a worker subnet, and a VNet integration subnet with delegation.
Create the network security group (NSG).
Note
If you add custom rules to the NSG, ensure that it allows the traffic described in Required network security group traffic.
az network nsg create \ --name "${CUSTOMER_NSG}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --location "${LOCATION}"Create the virtual network with the worker subnet.
az network vnet create \ --name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --location "${LOCATION}" \ --address-prefixes 10.0.0.0/16 \ --subnet-name "${CUSTOMER_VNET_SUBNET1}" \ --subnet-prefixes 10.0.0.0/24 \ --nsg "${CUSTOMER_NSG}"Create the VNet integration subnet with the required delegation.
az network vnet subnet create \ --name "${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --address-prefixes 10.0.1.0/24 \ --delegations Microsoft.RedHatOpenShift/hcpOpenShiftClustersGet the resource IDs for the networking resources.
NSG_ID=$(az network nsg show \ --name "${CUSTOMER_NSG}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv) SUBNET_ID=$(az network vnet subnet show \ --name "${CUSTOMER_VNET_SUBNET1}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv) VNET_ID=$(az network vnet show \ --name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv) VNET_INTEGRATION_SUBNET_ID=$(az network vnet subnet show \ --name "${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv)
Create the managed identities
Create the user-assigned managed identities for the cluster operators and the service managed identity. For a description of each identity and its purpose, see Required managed identities and role assignments.
# Control plane operator identities
az identity create --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}"
# Data plane operator identities
az identity create --name "${CLUSTER_NAME}-dp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}"
# Service managed identity
az identity create --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}"
Get the resource IDs and principal IDs for the managed identities.
# Control plane identity resource IDs and principal IDs
CLUSTER_API_AZURE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLUSTER_API_AZURE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
CONTROL_PLANE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CONTROL_PLANE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
CLOUD_CONTROLLER_MANAGER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLOUD_CONTROLLER_MANAGER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
INGRESS_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
INGRESS_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
DISK_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
FILE_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
FILE_CSI_DRIVER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
IMAGE_REGISTRY_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
IMAGE_REGISTRY_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
CLOUD_NETWORK_CONFIG_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLOUD_NETWORK_CONFIG_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
# Data plane identity resource IDs and principal IDs
DP_DISK_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_FILE_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_FILE_CSI_DRIVER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
DP_IMAGE_REGISTRY_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_IMAGE_REGISTRY_MI_PID=$(az identity show --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
# Service managed identity resource ID and principal ID
SERVICE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
SERVICE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
Create the role assignments
Assign the required Azure roles to each managed identity. For a detailed description of each identity and its required role assignments, see Required managed identities and role assignments.
Set the role definition GUIDs.
READER_ROLE="acdd72a7-3385-48ef-bd42-f606fba81ae7" HCP_CLUSTER_API_PROVIDER_ROLE="88366f10-ed47-4cc0-9fab-c8a06148393e" HCP_CONTROL_PLANE_OPERATOR_ROLE="fc0c873f-45e9-4d0d-a7d1-585aab30c6ed" CLOUD_CONTROLLER_MANAGER_ROLE="a1f96423-95ce-4224-ab27-4e3dc72facd4" INGRESS_OPERATOR_ROLE="0336e1d3-7a87-462b-b6db-342b63f7802c" FILE_STORAGE_OPERATOR_ROLE="0d7aedc0-15fd-4a67-a412-efad370c947e" NETWORK_OPERATOR_ROLE="be7a6435-15ae-4171-8f30-4a343eff9e8f" FEDERATED_CREDENTIAL_ROLE="ef318e2a-8334-4a05-9e4a-295a196c6a6e" HCP_SERVICE_MI_ROLE="c0ff367d-66d8-445e-917c-583feb0ef0d4" IMAGE_REGISTRY_OPERATOR_ROLE="8b32b316-c2f5-4ddf-b05b-83dacd2d08b5"Assign control plane operator roles.
# Cluster API Provider on the VNet az role assignment create --assignee-object-id "${CLUSTER_API_AZURE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CLUSTER_API_PROVIDER_ROLE}" --scope "${VNET_ID}" # Control Plane Operator on VNet and NSG az role assignment create --assignee-object-id "${CONTROL_PLANE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CONTROL_PLANE_OPERATOR_ROLE}" --scope "${VNET_ID}" az role assignment create --assignee-object-id "${CONTROL_PLANE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CONTROL_PLANE_OPERATOR_ROLE}" --scope "${NSG_ID}" # Cloud Controller Manager on VNet and NSG az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${NSG_ID}" az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${VNET_ID}" # Ingress Operator on VNet az role assignment create --assignee-object-id "${INGRESS_MI_PID}" --assignee-principal-type ServicePrincipal --role "${INGRESS_OPERATOR_ROLE}" --scope "${VNET_ID}" # File Storage Operator on VNet and NSG az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${NSG_ID}" az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${VNET_ID}" # Network Operator on VNet az role assignment create --assignee-object-id "${CLOUD_NETWORK_CONFIG_MI_PID}" --assignee-principal-type ServicePrincipal --role "${NETWORK_OPERATOR_ROLE}" --scope "${VNET_ID}" # Image Registry Operator on VNet az role assignment create --assignee-object-id "${IMAGE_REGISTRY_MI_PID}" --assignee-principal-type ServicePrincipal --role "${IMAGE_REGISTRY_OPERATOR_ROLE}" --scope "${VNET_ID}"Assign service managed identity roles.
# Service Managed Identity on VNet and NSG az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_SERVICE_MI_ROLE}" --scope "${VNET_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_SERVICE_MI_ROLE}" --scope "${NSG_ID}" # Reader on each control plane identity az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLUSTER_API_AZURE_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CONTROL_PLANE_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLOUD_CONTROLLER_MANAGER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${INGRESS_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${DISK_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${FILE_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${IMAGE_REGISTRY_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLOUD_NETWORK_CONFIG_MI_ID}"Assign data plane operator roles.
# Federated Credential on each data plane identity az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_DISK_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_FILE_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_IMAGE_REGISTRY_MI_ID}" # File Storage Operator on VNet and NSG az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${NSG_ID}" az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${VNET_ID}" # Image Registry Operator on VNet az role assignment create --assignee-object-id "${DP_IMAGE_REGISTRY_MI_PID}" --assignee-principal-type ServicePrincipal --role "${IMAGE_REGISTRY_OPERATOR_ROLE}" --scope "${VNET_ID}"
Set up customer-managed etcd encryption
Azure Red Hat OpenShift with hosted control planes encrypts etcd data by using customer-managed keys.
Create the KMS managed identity
Set environment variables for the KMS key.
KMS_KEY_NAME="etcd-kms-key"Create the KMS managed identity.
az identity create \ --name "${CLUSTER_NAME}-cp-kms" \ --resource-group "${CUSTOMER_RG_NAME}"Get the resource ID and principal ID for the KMS managed identity.
KMS_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv) KMS_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
Create the KMS role assignments
Assign the service managed identity the Reader role on the KMS managed identity.
az role assignment create \
--assignee-object-id "${SERVICE_MI_PID}" \
--assignee-principal-type ServicePrincipal \
--role "${READER_ROLE}" \
--scope "${KMS_MI_ID}"
Create the key store and KMS key
Create an Azure Key Vault or Managed HSM with RBAC authorization and an RSA 2048-bit encryption key for customer-managed etcd encryption. For more information about choosing between Key Vault and Managed HSM, see Choose the key store type.
Set environment variables for the Key Vault.
KEYVAULT_NAME="<key-vault-name>"Create the Key Vault.
az keyvault create \ --name "${KEYVAULT_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --location "${LOCATION}" \ --enable-rbac-authorization trueAssign yourself the Key Vault Crypto Officer role. This role grants you the permissions to create the KMS encryption key.
az role assignment create \ --role "Key Vault Crypto Officer" \ --assignee "$(az account show --query user.name -o tsv)" \ --scope "$(az keyvault show --name "${KEYVAULT_NAME}" --resource-group "${CUSTOMER_RG_NAME}" --query id -o tsv)"Create the KMS encryption key.
az keyvault key create \ --vault-name "${KEYVAULT_NAME}" \ --name "${KMS_KEY_NAME}" \ --kty RSA \ --size 2048Get the key version.
KMS_KEY_VERSION=$(az keyvault key show \ --vault-name "${KEYVAULT_NAME}" \ --name "${KMS_KEY_NAME}" \ --query "key.kid" --output tsv | rev | cut -d'/' -f1 | rev)Assign the Key Vault Crypto User role to the KMS managed identity on the Key Vault.
KEY_VAULT_CRYPTO_USER_ROLE="12338af0-0e69-4776-bea7-57ae8d297424" KEYVAULT_ID=$(az keyvault show --name "${KEYVAULT_NAME}" --query id --output tsv) az role assignment create \ --assignee-object-id "${KMS_MI_PID}" \ --assignee-principal-type ServicePrincipal \ --role "${KEY_VAULT_CRYPTO_USER_ROLE}" \ --scope "${KEYVAULT_ID}"
(Optional) Restrict key store access to a private endpoint
If you want to restrict key store access to a private endpoint, create the following resources. If you don't need a private key store, skip to Create the cluster.
Important
The steps in this section configure private access on the key store only. When you create your Azure Red Hat OpenShift with hosted control planes cluster, also set --vault-visibility Private. Both settings must match.
Disable public network access on the Key Vault.
az keyvault update \ --name "${KEYVAULT_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --public-network-access DisabledCreate a private endpoint for the Key Vault.
az network private-endpoint create \ --name "kv-private-endpoint" \ --resource-group "${CUSTOMER_RG_NAME}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --subnet "${CUSTOMER_VNET_SUBNET1}" \ --private-connection-resource-id "${KEYVAULT_ID}" \ --group-ids vault \ --connection-name "kv-private-endpoint"Create a private DNS zone and link it to the virtual network.
az network private-dns zone create \ --name "privatelink.vaultcore.azure.net" \ --resource-group "${CUSTOMER_RG_NAME}" az network private-dns link vnet create \ --name "kv-private-dns-zone-link" \ --resource-group "${CUSTOMER_RG_NAME}" \ --zone-name "privatelink.vaultcore.azure.net" \ --virtual-network "${CUSTOMER_VNET_NAME}" \ --registration-enabled falseCreate a DNS zone group for the private endpoint.
az network private-endpoint dns-zone-group create \ --name "kv-private-ep-dns-group" \ --resource-group "${CUSTOMER_RG_NAME}" \ --endpoint-name "kv-private-endpoint" \ --private-dns-zone "privatelink.vaultcore.azure.net" \ --zone-name "config1"
Create the cluster
Create the Azure Red Hat OpenShift with hosted control planes cluster by using the network infrastructure, managed identities, and role assignments you configured in the previous steps.
Note
For information about all customizable cluster parameters, see the command help (az aro hcp cluster create -h).
Create the cluster with the following command.
az aro hcp cluster create \ --name "${CLUSTER_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --location "${LOCATION}" \ --version "${CLUSTER_VERSION}" \ --channel-group stable \ --subnet-id "${SUBNET_ID}" \ --vnet-integration-subnet-id "${VNET_INTEGRATION_SUBNET_ID}" \ --nsg "${NSG_ID}" \ --managed-resource-group-name "${MANAGED_RESOURCE_GROUP}" \ --key-management-mode CustomerManaged \ --kms-vault-name <AzureKeyVaultName-or-ManagedHSMName> \ --key-vault-type <AzureKeyVault-or-ManagedHSM> \ --vault-visibility <Public-or-Private> \ --kms-active-key "{name:${KMS_KEY_NAME},version:${KMS_KEY_VERSION}}" \ --api-visibility <Public-or-Private> \ --ingress-visibility <Public-or-Private> \ --assign-service-managed-identity "${SERVICE_MI_ID}" \ --assign-control-plane-operator-identity cluster-api-azure "${CLUSTER_API_AZURE_MI_ID}" \ --assign-control-plane-operator-identity control-plane "${CONTROL_PLANE_MI_ID}" \ --assign-control-plane-operator-identity cloud-controller-manager "${CLOUD_CONTROLLER_MANAGER_MI_ID}" \ --assign-control-plane-operator-identity ingress "${INGRESS_MI_ID}" \ --assign-control-plane-operator-identity disk-csi-driver "${DISK_CSI_DRIVER_MI_ID}" \ --assign-control-plane-operator-identity file-csi-driver "${FILE_CSI_DRIVER_MI_ID}" \ --assign-control-plane-operator-identity image-registry "${IMAGE_REGISTRY_MI_ID}" \ --assign-control-plane-operator-identity cloud-network-config "${CLOUD_NETWORK_CONFIG_MI_ID}" \ --assign-control-plane-operator-identity kms "${KMS_MI_ID}" \ --assign-data-plane-operator-identity disk-csi-driver "${DP_DISK_CSI_DRIVER_MI_ID}" \ --assign-data-plane-operator-identity file-csi-driver "${DP_FILE_CSI_DRIVER_MI_ID}" \ --assign-data-plane-operator-identity image-registry "${DP_IMAGE_REGISTRY_MI_ID}"If you're using Managed HSM, set
--kms-vault-nameto the Managed HSM name (for example,"${MHSM_NAME}") and set--key-vault-typetoManagedHSM. If you're using standard Key Vault, set--key-vault-typetoAzureKeyVault(default).If you restricted the key store to a private endpoint, set
--vault-visibilitytoPrivateinstead ofPublicso your cluster setting matches the key store network configuration.Set
--api-visibilitytoPublicfor an internet-accessible API server, orPrivatefor an API server reachable only through a private network connection.Set
--ingress-visibilitytoPublicfor internet-accessible application routes, orPrivatefor application routes reachable only through a private network connection.The deployment typically takes 15 to 20 minutes to complete.
Verify that the cluster was created successfully.
az aro hcp cluster show \ --name "${CLUSTER_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query "properties.provisioningState" \ --output tsvWhen the cluster is ready, the output is
Succeeded.
Create the node pool
Create a node pool to add worker compute capacity to your cluster.
The following command creates a node pool with two Standard_D8s_v6 worker nodes.
Important
Before creating the node pool, verify that the NSG rules allow the required traffic described in Plan your cluster network. A matching Deny rule can prevent the node pool from completing provisioning.
az aro hcp cluster nodepool create \
--cluster-name "${CLUSTER_NAME}" \
--name "${NP_NAME}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--replicas 2 \
--vm-size Standard_D8s_v6 \
--version "${NP_VERSION}" \
--channel-group stable
Verify that the node pool was created successfully.
az aro hcp cluster nodepool show \
--cluster-name "${CLUSTER_NAME}" \
--name "${NP_NAME}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--query "properties.provisioningState" \
--output tsv
When the node pool is ready, the output is Succeeded.
Bicep file reference
The example Bicep file deploys a cluster with default configuration values. This section describes each resource and property in the Bicep file that you can customize before deploying.
Networking resources
The Bicep file defines a network security group (NSG), a virtual network with a worker subnet and a VNet integration subnet, and the subnet delegation for the VNet integration subnet. For subnet sizing, placement, and NSG requirements, see Plan your cluster network.
Managed identities and role assignments
The Bicep file defines a user-assigned managed identity for each OpenShift cluster Operator, and role assignments that grant each identity the permissions it needs. For a detailed description of each identity and its required role assignments, see Required managed identities and role assignments.
Etcd encryption resources
The Bicep file includes the following resources for customer-managed etcd encryption:
- KMS managed identity - A user-assigned managed identity to access the KMS key. This identity requires:
- A Key Vault Crypto User role assignment (for Azure Key Vault) or a Managed HSM Crypto User role assignment (for Managed HSM) scoped to the key store.
- A Reader role assignment that grants the service managed identity read access over the KMS managed identity
- Azure Key Vault or Managed HSM - The key store with RBAC authorization enabled. Set the
keyVaultTypeparameter toKeyVault(default) orManagedHSMto select the key store type. - KMS encryption key - An RSA 2048 key created in the key store.
If you set the privateKeyVault parameter to true, the Bicep file also deploys the following resources to restrict access to the key store to a private endpoint:
- A private endpoint for the key store.
- A private DNS zone (
privatelink.vaultcore.azure.netfor Key Vault orprivatelink.managedhsm.azure.netfor Managed HSM). - A private DNS zone group that associates the private endpoint with the DNS zone
- A virtual network link that connects the private DNS zone to the cluster virtual network
Cluster resource properties
The cluster resource (Microsoft.RedHatOpenShift/hcpOpenShiftClusters) defines the configuration for your cluster.
For more information about each configuration option, see Choose your permanent cluster settings.
| Property | Required | Description |
|---|---|---|
version.id |
Yes | The OpenShift minor version to install. For example, 4.20. To list available versions, see List available OpenShift versions. |
version.channelGroup |
Yes | The OpenShift update channel. The only supported option is stable. |
dns.baseDomainPrefix |
No | A custom prefix for the cluster's base domain name. Maximum length of 15 characters. Must start with a lowercase letter, and can contain only lowercase letters, numbers, and hyphens (-). |
network.networkType |
No | The Container Network Interface (CNI) plugin. Valid values are OVNKubernetes (default) or Other. |
network.podCidr |
No | The IP address range for pods. Default: 10.128.0.0/14. |
network.serviceCidr |
No | The IP address range for services. Default: 172.30.0.0/16. |
network.machineCidr |
No | The IP address range for compute nodes. Default: 10.0.0.0/16. |
network.hostPrefix |
No | The subnet prefix length allocated to each node for its pods. Default: 23. |
api.visibility |
No | The visibility of the API server. Valid values are Public (default) or Private. This setting is immutable. |
api.authorizedCidrs |
No | A list of authorized CIDR blocks (IPv4 only) permitted to access the API server. Maximum of 500 entries. |
ingress.type |
No | The type of the default cluster ingress. Valid values are Public (default) or Private. This setting is immutable. |
cryptoRestrictions |
No | Cryptographic restrictions for kernel and userspace libraries. Valid values are None (default) or FIPS. When set to FIPS, all cryptographic operations use FIPS-validated modules and all node pools inherit the FIPS configuration. This setting is immutable after cluster creation. |
etcd.dataEncryption.keyManagementMode |
Yes | The encryption key management mode. The only supported value is CustomerManaged. |
etcd.dataEncryption.customerManaged.encryptionType |
Yes | The encryption type. The only supported value is KMS. |
etcd.dataEncryption.customerManaged.kms.vaultName |
Yes | The name of the Azure Key Vault or Managed HSM. |
etcd.dataEncryption.customerManaged.kms.keyVaultType |
No | The type of key store. Valid values are KeyVault (default) or ManagedHSM. |
etcd.dataEncryption.customerManaged.kms.visibility |
Yes | How your Azure Red Hat OpenShift with hosted control planes cluster reaches the key store. Valid values are Public or Private. If you set this to Private, you must also disable public network access on the key store and create a private endpoint in the cluster virtual network. |
etcd.dataEncryption.customerManaged.kms.activeKey.name |
Yes | The name of the encryption key. |
etcd.dataEncryption.customerManaged.kms.activeKey.version |
Yes | The version of the encryption key. |
clusterImageRegistry.state |
No | The state of the internal image registry. Valid values are Enabled (default) or Disabled. |
platform.managedResourceGroup |
No | The name for the managed resource group. Maximum length of 90 characters. |
platform.subnetId |
Yes | The Azure resource ID of the worker subnet. |
platform.vnetIntegrationSubnetId |
Yes | The Azure resource ID of the VNet integration subnet. |
platform.outboundType |
No | The outbound connectivity model. The only supported value is LoadBalancer. |
platform.networkSecurityGroupId |
Yes | The Azure resource ID of the network security group. |
platform.operatorsAuthentication |
Yes | The managed identity assignments for the cluster operators. |
autoscaling.maxNodesTotal |
No | The maximum number of nodes the cluster autoscaler can create. If you omit this property, the autoscaler has no maximum limit. |
autoscaling.maxPodGracePeriodSeconds |
No | The maximum seconds for graceful pod termination before scale-down. |
autoscaling.maxNodeProvisionTimeSeconds |
No | The maximum seconds for node provisioning. Default: 900. |
autoscaling.podPriorityThreshold |
No | The priority threshold below which pods are treated as best-effort by the autoscaler. Default: -10. |
imageDigestMirrors |
No | Image digest mirror entries that redirect image pulls from a source registry to mirror registries. Maximum of 240 entries. |
nodeDrainTimeoutMinutes |
No | The default node drain grace period for all node pools, in minutes. Range: 0 to 10080. Default: 0 (no time limit). |
identity.type |
Yes | The identity type. Must be UserAssigned. |
identity.userAssignedIdentities |
Yes | A map of user-assigned managed identity resource IDs. |
Node pool resource properties
The node pool resource (Microsoft.RedHatOpenShift/hcpOpenShiftClusters/nodePools) defines a group of worker nodes that share the same configuration.
| Property | Required | Description |
|---|---|---|
version.id |
Yes | The OpenShift patch version for this node pool. For example, 4.20.8. The node pool version must be a patch version within the cluster's minor version. To list available versions, see List available OpenShift versions. |
version.channelGroup |
Yes | The OpenShift update channel. The only supported option is stable. |
platform.vmSize |
Yes | The Azure virtual machine size for the nodes. To list available VM sizes, see List available VM sizes. |
platform.subnetId |
No | The Azure resource ID of the subnet. If you don't specify this property, the node pool uses the cluster's worker subnet. |
platform.osDisk.sizeGiB |
No | The OS disk size in GiB. Minimum: 64. Maximum: 4095. Default: 64. For Ephemeral disks, the maximum is 2040, and the VM size can lower it further. |
platform.osDisk.diskStorageAccountType |
No | The disk storage account type. Valid values are Premium_LRS (default), StandardSSD_LRS, or Standard_LRS. |
platform.osDisk.encryptionSetId |
No | The Azure resource ID of a DiskEncryptionSet to encrypt OS disks. |
platform.osDisk.diskType |
No | The OS disk type. Valid values are Managed (default) or Ephemeral. |
platform.availabilityZone |
No | The Azure availability zone for the node pool. |
platform.enableEncryptionAtHost |
No | Enables host-level encryption for VMs. Default: false. |
replicas |
Conditional | The number of nodes. Required if autoScaling isn't configured. |
autoScaling.min |
Conditional | The minimum node count. Required if replicas isn't configured. |
autoScaling.max |
Conditional | The maximum node count. Required if replicas isn't configured. |
autoRepair |
No | Enables health checks for nodes. Default: true. |
labels |
No | Kubernetes labels applied to nodes, as an array of key-value pairs. |
taints |
No | Kubernetes taints applied to nodes. Each taint has a key, value, and effect (NoSchedule, PreferNoSchedule, or NoExecute). |
nodeDrainTimeoutMinutes |
No | The node drain grace period for this node pool, in minutes. Overrides the cluster default. Range: 0 to 10080. |
List available OpenShift versions
The cluster version.id uses a minor version (for example, 4.20), while the node pool version.id uses a patch version (for example, 4.20.8).
To list the available OpenShift versions in your region, run the following command:
az aro hcp get-versions --location "${LOCATION}" --output table
List available VM sizes
The availability of VM sizes depends on the Azure region of your cluster. To list VM sizes available in your region, run the following command:
az vm list-sizes --location "${LOCATION}" --output table
For the list of supported worker node VM sizes for Azure Red Hat OpenShift with hosted control planes, see Supported virtual machine sizes. Use the regional list to confirm that a supported size is available in your cluster's location.