An Azure service for ingesting, preparing, and transforming data at scale.
ADF Power Query cannot evaluate Azure SQL source with system-assigned managed identity
Issue In Azure Data Factory Power Query (Data Wrangling), an Azure SQL Database dataset cannot be evaluated in the Power Query Online editor. The following error is displayed: Credentials are invalid or missing, so this query could not be…
Azure Data Factory
Trying to filter csv file based on content
I have an existing ADF pipeline that is creating an error due to some bad (or not acceptable data) displayed on the csv file. Basically, instead of having proper header on the first row with certain amount columns (with data), csv file displays data like…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Example for Fabric Data Pipeline CICD with variable substitution.
Hi All, I am hoping someone can help me figure out how to implement CICD with connection substitution for a Fabric Data Factory pipeline. An example would be awesome - All I want to do is to promote a Data Factory pipeline to another environment and…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory - self-hosted integration runtime - missing metrics in Data Factory (v2) overview
We just noticed, that the metrics from the self hosted integration runtime, like Integration Runtime CPU or Integration Runtime Memory are missing. The last metrics, we have are from Apr. 29 of this year (around 6:00 CEST) - before this time, we have…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
The specified network name is no longer available - error code - 22852
Hello Team, Today I faced an error, when i am trying to do test connection. Please resolved as soon as possible. I need to resolve please help me to resolve. 
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory Data Flow Trigger Runs Successfully but Source File Is Not Updated (Debug Works Fine)
We are currently experiencing an issue with Azure ADF data flows. While the data flow functions as expected in debug mode and correctly updates the storage file in sink, it fails to update the storage file when triggered, even though the logs indicate…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Vulnerable libcurl.dll detected in Microsoft Integration Runtime 5.0 bundled ODBC drivers - supported remediation?
Hello, Our security vulnerability scanner has identified vulnerable versions of libcurl.dll within the Microsoft Integration Runtime 5.0 installation directory: C:\Program Files\Microsoft Integration Runtime\5.0\Shared\ODBC Drivers\ We verified that the…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory Private endpoint for Portal Endpoint
Hi Team, We are looking for recommendations and best practices around configuring the portal private endpoint for Azure Data Factory in a multi-VNET environment. Current setup: We have multiple ADF instances owned by separate teams. Each ADF has its…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
How to copy image from sqlserver in varbinary to Azure Blob storage using azure data factory
Hi all, “We are looking for a sample Copy Activity in Azure Data Factory to copy image data from Oracle/SQL databases into Azure Blob Storage as image files. The challenge we face is that when copying from relational databases, we are unable to perform a…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Does Azure Data Factory support Federated Identity Credential / Workload Identity Federation for ADLS Gen2 linked services?
I am trying to validate whether Azure Data Factory supports a secretless cross-tenant authentication pattern using Federated Identity Credential / Workload Identity Federation. Scenario: We have an Azure Data Factory pipeline / Copy Activity that needs…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Can't reach Azure Data Factory
Hi I can't reach adf.azure.com. Clearing cache and cookies or changing to a different browser (Firefox, Chrome, Edge) on the same laptop doesn't work. I can still open the browser on my phone and access just fine. What am I missing?
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Certain Decimal values from Parquet file are returned as NULL in Data Flow
Hi, when reading a Parquet file in Data Flow, decimal values that exceed a certain size/precision are being returned as NULL instead of being correctly parsed. Example 72232493.28310600000000. The issue is not related to the sink or any downstream…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF pipeline is failing while updating the single field COSMOS document
We have similar ADF pipelines in lower & Prod environments! But the pipeline is failing in PROD environment while updating the data to COSMOS document! Below is the error message it uses Dataflow to update: Operation on target…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
CLOUD_PROVIDER_RESOURCE_STOCKOUT (CLOUD_FAILURE)
I run a pipeline every hour, 24/7. Lately I have been receiving the CLOUD_PROVIDER_RESOURCE_STOCKOUT frequently during business hours.
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Data Factory Parameter Limitation Issue
Hi, I am having problem when deploying data factory for my batch data. The error is regarding the parameter limitation of 256 parameters, but the number of parameters were 259. I believe this error root cause is the dependency to ARM which having hard…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Migrating varbinary data from onprem SQL table to Azure blob storage.
Hi, I need to move the varbinary data in one of my on Prem SQL data table column Column(binary_data) which needs to be uploaded to azure blob storage. We have 2 million rows in the table and every row should create one file in the blob storage for…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
SQL Server Integration Services
A Microsoft platform for building enterprise-level data integration and data transformations solutions.
SQL Server | Other
Additional SQL Server features and topics not covered by specific categories
Transfer documents from sqlserver in varbinary to Azure Blob storage
I am looking to move a large set of documents from a Managed Instance of SQL Server database where the documents are saved in varbinary data type on the sql server table. The ADF dataflow I created doesn't let me select a Azure Blob 'Binary' as the sink…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
SQL Server | Other
Additional SQL Server features and topics not covered by specific categories
FABRIC ON PREMISED GATEWAY DATA ACCESS USING A COPY DATA ACTIVITY FROM A PIPE LINE
Hello guys i have been facing this issue with my gateway recently on the Fabric data plateform, my gateway appears to be online and updated to the latest version yet, anytime i want to pull data using a pipeline and a copy data activity, i face this…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
From AME unable to create ADF Linked Service connection for CORP KUSTO using CORP SPN
Hello Team, I am trying to create a Linked Service in AME ADF for CORP Kusto cluster. For that I have created SPN in AME and marked it as multi-tenant. This SPN is also created in CORP. However, AME ADF is throwing error that SPN USER does not have…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
ADF Cosmos DB Change Feed Connector Missing Intermediate Events/Documents for Separate Records
Hi team, I am experiencing an issue where the Azure Data Factory (ADF) Mapping Data Flow Cosmos DB Change Feed connector is missing specific intermediate document updates. Environment Setup: Source: Azure Cosmos DB (NoSQL API) container. Timestamps are…
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.