Staging - Update Spark Compute

Service:: Environment

API Version:: v1

Update environment staging spark compute. If you want to delete a spark property, set its value as null.

Note

This API is a release version of a beta version due to be deprecated on August 31, 2026. When calling this API - callers must set the query parameter beta to the value false

Permissions

Write permission for the environment item.

Required Delegated Scopes

Item.ReadWrite.All or Environment.ReadWrite.All

Microsoft Entra supported identities

This API supports the Microsoft identities listed in this section.

Identity	Support
User	Yes
Service principal and Managed identities	Yes

Interface

PATCH https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/environments/{environmentId}/staging/sparkcompute?beta={beta}

URI Parameters

Name	In	Required	Type	Description
environmentId	path	True	string (uuid)	The environment ID.
workspaceId	path	True	string (uuid)	The workspace ID.
beta	query	True	boolean	This parameter specifies which version of the API to use. Set to `false` to use the release version.

Request Body

Name	Type	Description
customLivePoolSettings	CustomLivePoolSettings	Live pool hydration settings. Pass `null` to remove the hydration settings entirely. Omit to leave the existing value unchanged.
customLivePoolSupport	CustomLivePoolSupport	Controls whether the live pool is active. Setting `Disabled` turns it off but retains the hydration settings. Omit to leave the existing value unchanged.
driverCores	integer (int32)	Spark driver core. Must be one of the following values: 4, 8, 16, 32, 64.
driverMemory	CustomPoolMemory	Spark driver memory.
dynamicExecutorAllocation	DynamicExecutorAllocationProperties	Dynamic executor allocation.
executorCores	integer (int32)	Spark executor core. Must be one of the following values: 4, 8, 16, 32, 64.
executorMemory	CustomPoolMemory	Spark executor memory.
instancePool	InstancePool	Environment pool has to be a valid custom pool. The name for a default workspace pool is Starter Pool.
runtimeVersion	string	Runtime version, find the supported fabric runtimes. For example: 1.3
sparkProperties	SparkProperty[]	Spark properties.

Responses

Name Type Description

200 OK

Name	Type	Description
200 OK	EnvironmentSparkCompute	Request completed successfully.
429 Too Many Requests	ErrorResponse	The service rate limit was exceeded. The server returns a `Retry-After` header indicating, in seconds, how long the client must wait before sending additional requests. Headers Retry-After: integer
Other Status Codes	ErrorResponse	Common error codes: UnknownError - An error occurred.

EnvironmentSparkCompute

Request completed successfully.

429 Too Many Requests

ErrorResponse

The service rate limit was exceeded. The server returns a Retry-After header indicating, in seconds, how long the client must wait before sending additional requests.

Headers

Retry-After: integer

Other Status Codes

ErrorResponse

Common error codes:

UnknownError - An error occurred.

Examples

Update environment staging spark compute example

Update environment staging spark compute instance pool using pool ID example

Update environment staging spark compute example

Sample request

HTTP

PATCH https://api.fabric.microsoft.com/v1/workspaces/f089354e-8366-4e18-aea3-4cb4a3a50b48/environments/707cfd07-cbf1-41da-aad7-dd157ddb8c11/staging/sparkcompute?beta=false

{
  "instancePool": {
    "name": "MyWorkspacePool",
    "type": "Workspace"
  },
  "driverCores": 4,
  "driverMemory": "56g",
  "executorCores": 4,
  "executorMemory": "56g",
  "dynamicExecutorAllocation": {
    "enabled": false,
    "minExecutors": 1,
    "maxExecutors": 1
  },
  "sparkProperties": [
    {
      "key": "spark.acls.enable",
      "value": "false"
    },
    {
      "key": "spark.admin.acls",
      "value": null
    }
  ],
  "runtimeVersion": "1.2",
  "customLivePoolSupport": "Enabled",
  "customLivePoolSettings": {
    "maxClustersToHydrate": 4,
    "clusterIdleTimeout": "PT20M",
    "customLivePoolLifespan": "PT1H"
  }
}

Sample response

Status code:: 200

{
  "instancePool": {
    "name": "MyWorkspacePool",
    "type": "Workspace",
    "id": "78942136-106c-4f3e-80fc-7ff4eae11603",
    "maxClustersToHydrateLimit": 10
  },
  "driverCores": 4,
  "driverMemory": "56g",
  "executorCores": 4,
  "executorMemory": "56g",
  "dynamicExecutorAllocation": {
    "enabled": false,
    "minExecutors": 1,
    "maxExecutors": 1
  },
  "sparkProperties": [
    {
      "key": "spark.acls.enable",
      "value": "false"
    }
  ],
  "runtimeVersion": "1.2",
  "customLivePoolSupport": "Enabled",
  "customLivePoolSettings": {
    "maxClustersToHydrate": 4,
    "clusterIdleTimeout": "PT20M",
    "customLivePoolLifespan": "PT1H"
  }
}

Update environment staging spark compute instance pool using pool ID example

Sample request

HTTP

PATCH https://api.fabric.microsoft.com/v1/workspaces/f089354e-8366-4e18-aea3-4cb4a3a50b48/environments/707cfd07-cbf1-41da-aad7-dd157ddb8c11/staging/sparkcompute?beta=false

{
  "instancePool": {
    "id": "78942136-106c-4f3e-80fc-7ff4eae11603"
  }
}

Sample response

Status code:: 200

{
  "instancePool": {
    "name": "MyWorkspacePool",
    "type": "Workspace",
    "id": "78942136-106c-4f3e-80fc-7ff4eae11603",
    "maxClustersToHydrateLimit": 10
  },
  "driverCores": 4,
  "driverMemory": "56g",
  "executorCores": 4,
  "executorMemory": "56g",
  "dynamicExecutorAllocation": {
    "enabled": false,
    "minExecutors": 1,
    "maxExecutors": 1
  },
  "sparkProperties": [
    {
      "key": "spark.acls.enable",
      "value": "false"
    }
  ],
  "runtimeVersion": "1.2"
}

Definitions

Name	Description
CustomLivePoolSettings	Live pool hydration settings on the environment's spark compute.
CustomLivePoolSupport	Flag controlling whether live pool support is active. Additional `CustomLivePoolSupport` values may be added over time.
CustomPoolMemory	Custom pool memory for Spark driver or Spark executor. Additional `CustomPoolMemory` types may be added over time.
CustomPoolType	Custom pool type. Additional `CustomPoolType` types may be added over time.
DynamicExecutorAllocationProperties	Dynamic executor allocation proerties.
EnvironmentSparkCompute
ErrorRelatedResource	The error related resource details object.
ErrorResponse	The error response.
ErrorResponseDetails	The error response details.
InstancePool	The instance pool.
SparkProperty	A Spark property key and its value.
UpdateEnvironmentSparkComputeRequest

CustomLivePoolSettings

Object

Live pool hydration settings on the environment's spark compute.

Name	Type	Description
clusterIdleTimeout	string (duration)	Idle-time threshold after which a hydrated cluster is released, using the ISO 8601 duration format. Minimum: PT20M. Maximum: PT24H.
customLivePoolLifespan	string (duration)	Maximum lifetime of a live pool activation, using the ISO 8601 duration format. The live pool job runs for this duration and then terminates. Minimum: PT30M. Maximum: PT24H.
maxClustersToHydrate	integer (int32) minimum: 1	Maximum number of clusters to pre-hydrate during each activation cycle. Minimum: 1. The maximum allowed value is determined by the capacity size and is exposed on the instance pool settings as `InstancePool.maxClustersToHydrateLimit`.

CustomLivePoolSupport

Enumeration

Flag controlling whether live pool support is active. Additional CustomLivePoolSupport values may be added over time.

Value	Description
Enabled	Live pool support is active.
Disabled	Live pool support is inactive. Configuration is retained.

CustomPoolMemory

Enumeration

Custom pool memory for Spark driver or Spark executor. Additional CustomPoolMemory types may be added over time.

Value	Description
28g	28GB memory.
56g	56G memory.
112g	112G memory.
224g	224G memory.
400g	400G memory.

CustomPoolType

Enumeration

Custom pool type. Additional CustomPoolType types may be added over time.

Value	Description
Workspace	Workspace level custom pool
Capacity	Capacity level custom pool

DynamicExecutorAllocationProperties

Object

Dynamic executor allocation proerties.

Name	Type	Description
enabled	boolean	The status of the dynamic executor allocation. False - Disabled, true - Enabled.
maxExecutors	integer (int32) minimum: 1	The maximum executor number for dynamic allocation and the minimum for this property is 1. The maximum value has to be lower than the instance pool `maxNodeCount`.
minExecutors	integer (int32) minimum: 1	The minimum executor number for dynamic allocation and the minimum for this property is 1. The maximum value has to be lower than `maxExecutors`.

EnvironmentSparkCompute

Object

Name	Type	Description
customLivePoolSettings	CustomLivePoolSettings	Live pool hydration settings. Controls how clusters are pre-hydrated when live pool support is enabled.
customLivePoolSupport	CustomLivePoolSupport	Controls whether the live pool is enabled. When `Enabled`, clusters are pre-hydrated according to the hydration settings so that workloads start without cold-start latency.
driverCores	integer (int32)	Spark driver core. Must be one of the following values: 4, 8, 16, 32, 64.
driverMemory	CustomPoolMemory	Spark driver memory.
dynamicExecutorAllocation	DynamicExecutorAllocationProperties	Dynamic executor allocation.
executorCores	integer (int32)	Spark executor core. Must be one of the following values: 4, 8, 16, 32, 64.
executorMemory	CustomPoolMemory	Spark executor memory.
instancePool	InstancePool	Environment pool has to be a valid custom pool. "Starter Pool" means use starter pool.
runtimeVersion	string	Runtime version, find the supported fabric runtimes. For example: 1.3
sparkProperties	SparkProperty[]	Spark properties.

ErrorRelatedResource

Object

The error related resource details object.

Name	Type	Description
resourceId	string	The resource ID that's involved in the error.
resourceType	string	The type of the resource that's involved in the error.

ErrorResponse

Object

The error response.

Name	Type	Description
errorCode	string	A specific identifier that provides information about an error condition, allowing for standardized communication between our service and its users.
isRetriable	boolean	When true, the request can be retried. Use the `Retry-After` response header to determine the delay, if available.
message	string	A human readable representation of the error.
moreDetails	ErrorResponseDetails[]	List of additional error details.
relatedResource	ErrorRelatedResource	The error related resource details.
requestId	string (uuid)	ID of the request associated with the error.

ErrorResponseDetails

Object

The error response details.

Name	Type	Description
errorCode	string	A specific identifier that provides information about an error condition, allowing for standardized communication between our service and its users.
message	string	A human readable representation of the error.
relatedResource	ErrorRelatedResource	The error related resource details.

InstancePool

Object

The instance pool.

Name	Type	Description
id	string (uuid)	Instance pool ID.
maxClustersToHydrateLimit	integer (int32)	Server-computed upper bound for the live pool `maxClustersToHydrate` setting, determined by the capacity size associated with this pool.
name	string	Instance pool name.
type	CustomPoolType	Instance pool type.

SparkProperty

Object

A Spark property key and its value.

Name	Type	Description
key	string	The Spark property key.
value	string	The Spark property value.

UpdateEnvironmentSparkComputeRequest

Object

Name	Type	Description
customLivePoolSettings	CustomLivePoolSettings	Live pool hydration settings. Pass `null` to remove the hydration settings entirely. Omit to leave the existing value unchanged.
customLivePoolSupport	CustomLivePoolSupport	Controls whether the live pool is active. Setting `Disabled` turns it off but retains the hydration settings. Omit to leave the existing value unchanged.
driverCores	integer (int32)	Spark driver core. Must be one of the following values: 4, 8, 16, 32, 64.
driverMemory	CustomPoolMemory	Spark driver memory.
dynamicExecutorAllocation	DynamicExecutorAllocationProperties	Dynamic executor allocation.
executorCores	integer (int32)	Spark executor core. Must be one of the following values: 4, 8, 16, 32, 64.
executorMemory	CustomPoolMemory	Spark executor memory.
instancePool	InstancePool	Environment pool has to be a valid custom pool. The name for a default workspace pool is Starter Pool.
runtimeVersion	string	Runtime version, find the supported fabric runtimes. For example: 1.3
sparkProperties	SparkProperty[]	Spark properties.