Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
ODBC (Open Database Connectivity) is a widely adopted standard that enables client applications to connect to and work with data from databases and big data platforms.
The Microsoft ODBC Driver for Fabric Data Engineering lets you connect, query, and manage Spark workloads in Fabric with the reliability and simplicity of the ODBC standard. Built on Fabric's Livy APIs, the driver provides secure and flexible Spark SQL connectivity to your C/C++, .NET, Python, and other ODBC-compatible applications on Linux.
Key features
- ODBC 3.x compliant: Full implementation of the ODBC 3.x specification.
- Microsoft Entra ID authentication: Multiple authentication flows, including Azure CLI, client credentials, certificate-based, and access token authentication.
- Spark SQL query support: Direct execution of Spark SQL statements.
- Comprehensive data type support: Support for all Spark SQL data types, including complex types (
ARRAY,MAP, andSTRUCT). - Session reuse: Built-in session management for improved performance.
- High-concurrency mode: Opt in to shared, warm server-side Livy sessions to reduce connection startup latency and Spark cluster sprawl.
- Large table support: Optimized handling for large result sets with configurable page sizes.
- Async prefetch: Background data loading for improved performance.
- Proxy support: HTTP proxy configuration for enterprise environments.
- Multi-schema lakehouse support: Connect to a specific schema within a lakehouse.
Note
In open-source Apache Spark, database and schema are used synonymously. For example, running SHOW SCHEMAS or SHOW DATABASES in a Fabric notebook returns the same result: a list of all schemas in the lakehouse.
Prerequisites
Before you use the Microsoft ODBC Driver for Microsoft Fabric Data Engineering on Linux, ensure you have the following prerequisites:
- Operating system: Ubuntu 22.04 or later, Debian 11 or later, or Red Hat Enterprise Linux (RHEL) 8 or later on x86-64.
- unixODBC: The ODBC driver manager for Linux. Install the
unixodbcandunixodbc-devpackages. - libcurl: The HTTP runtime dependency, such as
libcurl4or the equivalent package for your distribution. - OpenSSL: The TLS and cryptography runtime dependency, such as
opensslandlibssl3or equivalent packages. - Fabric access: Access to a Fabric workspace.
- Microsoft Entra ID credentials: Appropriate credentials for authentication.
- Workspace and lakehouse IDs: The GUID identifiers for your Fabric workspace and lakehouse.
- Azure CLI (optional): Required when you use Azure CLI authentication.
Download and install on Linux
To install the driver:
Extract
ms-sparksql-odbc-linux-2.0.0.tar.Open a terminal in the extracted directory.
Install the Debian package:
sudo dpkg -i microsoft-fabric-odbc-driver-2.0.0-Linux.deb
The package installs the following files:
| File | Installed location |
|---|---|
| Driver library | /usr/lib/libmicrosoftfabricodbc.so |
| Driver registration template | /usr/share/microsoft-fabric-odbc-driver/odbcinst.ini.template |
| DSN configuration template | /usr/share/microsoft-fabric-odbc-driver/odbc.ini.template |
| License | /usr/share/doc/microsoft-fabric-odbc-driver/LICENSE |
| Usage guide | /usr/share/doc/microsoft-fabric-odbc-driver/USAGE_Linux.md |
Install an RPM package
On RHEL, Fedora, or CentOS, install the RPM package from the extracted driver archive:
sudo dnf install ./microsoft-fabric-odbc-driver-*.x86_64.rpm
Alternatively, use rpm:
sudo rpm -i microsoft-fabric-odbc-driver-*.x86_64.rpm
Verify that the RPM registered the driver:
odbcinst -q -d | grep "Microsoft"
To uninstall the RPM package, run:
sudo dnf remove microsoft-fabric-odbc-driver
Install a tarball
For distributions without Debian or RPM package support, or for a custom installation location, extract the .tar.gz package and run its installation script:
tar -xzf microsoft-fabric-odbc-driver-2.0.0-linux-x86_64.tar.gz
cd microsoft-fabric-odbc-driver-2.0.0
# Install to /opt/microsoft/fabricodbc.
sudo ./install.sh
# Or install to a custom writable location.
./install.sh --prefix=$HOME/fabricodbc
The installation script supports these options:
| Option | Description |
|---|---|
--prefix=<path> |
Install to a custom directory. The default is /opt/microsoft/fabricodbc. |
--no-register |
Skip unixODBC driver registration. |
--dry-run |
Show the planned changes without installing files. |
--uninstall |
Remove the installed files and unregister the driver. |
--help |
Show command help. |
To verify the default tarball installation, run:
odbcinst -q -d
ls -la /opt/microsoft/fabricodbc/lib/libmicrosoftfabricodbc.so
To uninstall it, run sudo ./install.sh --uninstall from the extracted package directory.
Register driver manually
The package automatically registers the driver with unixODBC. To register the driver manually, run:
sudo odbcinst -i -d -f /usr/share/microsoft-fabric-odbc-driver/odbcinst.ini.template
Verify the installation
Verify that the driver is registered and that the library is installed:
odbcinst -q -d
ls -la /usr/lib/libmicrosoftfabricodbc.so
The odbcinst command should list [Microsoft ODBC Driver for Microsoft Fabric Data Engineering].
Uninstall the driver
To uninstall the driver, run the following command:
sudo dpkg -r microsoft-fabric-odbc-driver
This command removes the driver files and unregisters the driver from unixODBC.
Quick start example
The following examples connect to Fabric and run a Spark SQL query. Complete the prerequisites and install the driver before you run an example.
Python example
import pyodbc
connection_string = (
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=AZURE_CLI;"
)
conn = pyodbc.connect(connection_string, timeout=30)
cursor = conn.cursor()
cursor.execute("SELECT 'Hello from Fabric!' AS message")
row = cursor.fetchone()
print(row.message)
conn.close()
C/C++ example
#include <sql.h>
#include <sqlext.h>
#include <iostream>
int main() {
SQLHENV henv = SQL_NULL_HENV;
SQLHDBC hdbc = SQL_NULL_HDBC;
SQLHSTMT hstmt = SQL_NULL_HSTMT;
SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &henv);
SQLSetEnvAttr(henv, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, 0);
SQLAllocHandle(SQL_HANDLE_DBC, henv, &hdbc);
const char* connectionString =
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=AZURE_CLI;";
SQLRETURN result = SQLDriverConnect(
hdbc,
NULL,
(SQLCHAR*)connectionString,
SQL_NTS,
NULL,
0,
NULL,
SQL_DRIVER_NOPROMPT);
if (SQL_SUCCEEDED(result)) {
std::cout << "Connected successfully!" << std::endl;
SQLAllocHandle(SQL_HANDLE_STMT, hdbc, &hstmt);
result = SQLExecDirect(
hstmt,
(SQLCHAR*)"SELECT 'Hello from Fabric!' AS message",
SQL_NTS);
if (SQL_SUCCEEDED(result)) {
char message[256];
SQLLEN indicator;
while (SQLFetch(hstmt) == SQL_SUCCESS) {
SQLGetData(
hstmt,
1,
SQL_C_CHAR,
message,
sizeof(message),
&indicator);
std::cout << message << std::endl;
}
}
SQLFreeHandle(SQL_HANDLE_STMT, hstmt);
SQLDisconnect(hdbc);
}
SQLFreeHandle(SQL_HANDLE_DBC, hdbc);
SQLFreeHandle(SQL_HANDLE_ENV, henv);
return 0;
}
Build and run the example:
g++ -o fabric_test fabric_test.cpp -lodbc -std=c++17
./fabric_test
.NET example
using System.Data.Odbc;
string connectionString =
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};" +
"WorkspaceId=<workspace-id>;" +
"LakehouseId=<lakehouse-id>;" +
"AuthFlow=AZURE_CLI;";
using var connection = new OdbcConnection(connectionString);
await connection.OpenAsync();
Console.WriteLine("Connected successfully!");
using var command = new OdbcCommand(
"SELECT 'Hello from Fabric!' AS message",
connection);
using var reader = await command.ExecuteReaderAsync();
if (await reader.ReadAsync())
{
Console.WriteLine(reader.GetString(0));
}
Connection string format
Basic connection string
Use the following connection string format:
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};<parameter1>=<value1>;<parameter2>=<value2>;...
Connection string components
| Component | Description | Example |
|---|---|---|
DRIVER |
ODBC driver identifier | {Microsoft ODBC Driver for Microsoft Fabric Data Engineering} |
WorkspaceId |
Fabric workspace identifier (GUID) | 4bbf89a8-66bb-443f-91af-df31e6a7560b |
LakehouseId |
Fabric lakehouse identifier (GUID) | d8faa650-1343-496b-b9cc-d4168a676f90 |
AuthFlow |
Authentication method | AZURE_CLI, CLIENT_CREDENTIAL, CLIENT_CERTIFICATE, or ACCESS_TOKEN |
Example connection strings
Basic connection
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};WorkspaceId=<workspace-id>;LakehouseId=<lakehouse-id>;AuthFlow=AZURE_CLI
Connection with performance options
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};WorkspaceId=<workspace-id>;LakehouseId=<lakehouse-id>;AuthFlow=AZURE_CLI;ReuseSession=true;LargeTableSupport=true;PageSizeBytes=18874368
Connection with logging
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};WorkspaceId=<workspace-id>;LakehouseId=<lakehouse-id>;AuthFlow=AZURE_CLI;LogLevel=DEBUG;LogFile=/tmp/odbc_driver.log
Connection with high-concurrency mode
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};WorkspaceId=<workspace-id>;LakehouseId=<lakehouse-id>;AuthFlow=AZURE_CLI;LivyMode=HighConcurrency;SessionTag=bi-dashboard
For complete configuration guidance, see High-concurrency (HC) mode).
Authentication
The Microsoft ODBC Driver for Microsoft Fabric Data Engineering supports multiple authentication methods through Microsoft Entra ID. Configure authentication by using the AuthFlow parameter in the connection string or DSN.
Authentication methods
AuthFlow value |
Description |
|---|---|
AZURE_CLI |
Development using Azure CLI credentials |
CLIENT_CREDENTIAL |
Service principal with a client secret |
CLIENT_CERTIFICATE |
Service principal with a certificate |
ACCESS_TOKEN |
Pre-acquired bearer access token |
FILE_TOKEN |
Reads an authentication token from a file |
Note
Interactive browser authentication isn't available on headless Linux servers. Use Azure CLI, client credentials, certificate-based, or access token authentication instead.
Azure CLI authentication
Use Azure CLI authentication for development and interactive applications.
connection_string = (
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=AZURE_CLI;"
"Scope=https://api.fabric.microsoft.com/.default;"
)
conn = pyodbc.connect(connection_string)
Before you connect, verify that Azure CLI is installed and sign in:
az --version
az login
To install Azure CLI on Debian or Ubuntu, use the package manager:
sudo apt-get update
sudo apt-get install -y azure-cli
Client credentials authentication
Use client credentials authentication for automated services and background jobs.
connection_string = (
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=CLIENT_CREDENTIAL;"
f"TenantId={tenant_id};"
f"ClientId={client_id};"
f"ClientSecret={client_secret};"
)
Provide the following parameters:
TenantId: The Microsoft Entra tenant ID.ClientId: The application (client) ID.ClientSecret: The client secret.
Store secrets in a secure secret store or environment variables. Don't store secrets in plain-text connection strings or INI files.
Certificate-based authentication
Use certificate-based authentication for enterprise applications that require certificate credentials.
connection_string = (
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=CLIENT_CERTIFICATE;"
"TenantId=<tenant-id>;"
"ClientId=<client-id>;"
"CertificatePath=/path/to/cert.pfx;"
"CertificatePassword=<password>;"
)
Provide the following parameters:
TenantId: The Microsoft Entra tenant ID.ClientId: The application (client) ID.CertificatePath: The path to the PFX or PKCS12 certificate file.CertificatePassword: The certificate password.
Access token authentication
Use access token authentication when your application acquires a token through another mechanism.
access_token = acquire_token_from_custom_source()
connection_string = (
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=ACCESS_TOKEN;"
f"AccessToken={access_token};"
)
File token authentication
Use file token authentication in containerized environments and automated pipelines where another process writes a bearer token to the driver's well-known token file:
connection_string = (
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=FILE_TOKEN;"
)
For supported cross-driver property aliases, see Cross-driver property aliases.
Configuration parameters
Required parameters
Include these parameters in every connection string:
| Parameter | Type | Description | Example |
|---|---|---|---|
WorkspaceId |
UUID | Fabric workspace identifier | 4bbf89a8-... |
LakehouseId |
UUID | Fabric lakehouse identifier | d8faa650-... |
AuthFlow |
String | Authentication flow type | AZURE_CLI |
Optional parameters
Connection settings
| Parameter | Type | Default | Description |
|---|---|---|---|
Database |
String | None | Initial database for classic mode. HC mode doesn't apply it automatically; run USE <database> after connecting or use fully qualified table names. |
Scope |
String | https://api.fabric.microsoft.com/.default |
OAuth scope |
Performance settings
| Parameter | Type | Default | Description |
|---|---|---|---|
ReuseSession |
Boolean | true |
Reuse an existing Spark session |
LargeTableSupport |
Boolean | false |
Enable optimizations for large result sets |
EnableAsyncPrefetch |
Boolean | false |
Enable background data prefetching |
PageSizeBytes |
Integer | 18874368 (18 MB) |
Page size for result pagination from 1 through 18 MB |
Logging settings
| Parameter | Type | Default | Description |
|---|---|---|---|
LogLevel |
String | INFO |
Log level: DEBUG, INFO, WARN, or ERROR. Use DEBUG for detailed driver diagnostics. |
LogFile |
String | odbc_driver.log |
Absolute or relative log file path |
Proxy settings
| Parameter | Type | Default | Description |
|---|---|---|---|
UseProxy |
Boolean | false |
Enable a proxy |
ProxyHost |
String | None | Proxy host name |
ProxyPort |
Integer | None | Proxy port |
ProxyUsername |
String | None | Proxy authentication user name |
ProxyPassword |
String | None | Proxy authentication password |
High-concurrency (HC) settings
Use these parameters to enable and configure high-concurrency (HC) mode. HC mode is opt-in. If you don't set LivyMode or UseHighConcurrency, the driver uses the classic single-session path. For guidance and examples, see High-concurrency (HC) mode.
| Parameter | Type | Default | Description |
|---|---|---|---|
LivyMode |
String | Not set | Controls session acquisition. Supported values are Classic, HighConcurrency, and Auto. Auto currently uses the classic path. |
UseHighConcurrency |
Boolean | false |
Enables HC mode as an alternative to setting LivyMode=HighConcurrency. |
SessionTag |
String | None | Provides a server-side packing hint. Connections must use matching server-side configuration to share a warm Livy session. |
IdempotencyKey |
String | None | Provides a client-generated key for safe HC acquisition retries. The driver validates the character set and never logs the value. |
EnvironmentId |
String | None | Specifies the Fabric environment identifier sent as spark.fabric.environmentDetails in the HC acquisition configuration. |
HeartbeatTimeoutInSecond |
Integer | Server default | Sets the server-side HC session idle timeout, in seconds. The value must be a positive integer. |
AcquireTimeoutSeconds |
Integer | 300 |
Sets how long the driver waits for an HC session to reach the Idle state, in seconds. The value must be a positive integer. |
conf.<key> |
String | None | Specifies a Spark configuration override sent in the HC acquisition request. For example, conf.spark.sql.shuffle.partitions=200. |
Mode precedence and validation
LivyModetakes precedence overUseHighConcurrencywhen you set both parameters.- Parameter values aren't case-sensitive.
UseHighConcurrencyacceptstrue,1,yes, oronto enable HC mode. It acceptsfalse,0,no, oroffto disable HC mode.- For other nonempty values,
SQLDriverConnectreturns an error. DSN-onlySQLConnectlogs the validation failure and uses the classic path.
Compatibility settings
The driver accepts TenantPrincipal and PoolMax, including their environment-variable fallbacks, for compatibility with earlier HC configurations. The driver parses and validates these values, but they don't affect HC session matching or runtime pool capacity. Don't rely on them for tenant isolation or connection limits.
Environment variable fallbacks
The driver uses the following environment variables only when the corresponding DSN or connection-string parameter isn't set:
| Environment variable | Maps to |
|---|---|
FABRIC_ODBC_USE_HC |
UseHighConcurrency |
FABRIC_ODBC_WORKSPACE_ID |
WorkspaceId |
FABRIC_ODBC_LAKEHOUSE_ID |
LakehouseId |
FABRIC_ODBC_ENVIRONMENT_ID |
EnvironmentId |
FABRIC_ODBC_SESSION_TAG |
SessionTag |
FABRIC_ODBC_TENANT_PRINCIPAL |
TenantPrincipal (compatibility only) |
FABRIC_ODBC_HC_HEARTBEAT_SECONDS |
HeartbeatTimeoutInSecond |
FABRIC_ODBC_HC_ACQUIRE_TIMEOUT |
AcquireTimeoutSeconds |
FABRIC_ODBC_HC_POOL_MAX |
PoolMax (compatibility only) |
For FABRIC_ODBC_USE_HC, environment-only enablement recognizes true, 1, or yes (case-insensitive). Other values leave HC mode disabled. The on and off aliases apply only to DSN and connection-string values.
High-concurrency (HC) mode
High-concurrency (HC) mode lets ODBC connections to the same workspace and lakehouse attach to shared, warm server-side Livy Spark sessions instead of provisioning a separate cluster for each connection. Use it for workloads such as BI dashboards, notebook kernels, and ETL fan-out that open many short-lived connections and can share server-managed Spark capacity.
HC mode is opt-in. If you don't configure an HC setting, the driver uses the classic single-session path.
Choose a connection mode
Use HC mode when:
- Your workload opens many short-lived ODBC connections, such as BI dashboards, notebook kernels, or ETL fan-out.
- Connections target the same workspace and lakehouse and can share a server-managed Spark session.
- Reducing per-connection cold-start latency is important.
Use classic mode when:
- Each connection requires an isolated Spark session because of different Spark configurations or strict resource isolation.
- The Fabric Livy endpoint for the target workspace and lakehouse doesn't support HC mode.
Enable high-concurrency mode
Add a high-concurrency mode parameter to the DSN or connection string.
Recommended: Set LivyMode
Use LivyMode to select the session-acquisition mode explicitly:
LivyMode=HighConcurrency
Supported values are Classic, HighConcurrency, and Auto. Values aren't case-sensitive. Auto currently selects classic mode.
Alternative: Set UseHighConcurrency
Use the Boolean setting for compatibility with existing configurations:
UseHighConcurrency=true
To enable high-concurrency mode, use true, 1, yes, or on. To disable it, use false, 0, no, or off. Values aren't case-sensitive. SQLDriverConnect rejects other nonempty values with SQLSTATE HY024. DSN-only SQLConnect logs the validation failure and uses classic mode.
Note
If you set both parameters, LivyMode takes precedence over UseHighConcurrency.
You can also enable high-concurrency mode for the current shell by using environment variables:
export FABRIC_ODBC_USE_HC=true
export FABRIC_ODBC_WORKSPACE_ID=<workspace-id>
export FABRIC_ODBC_LAKEHOUSE_ID=<lakehouse-id>
export FABRIC_ODBC_SESSION_TAG=bi-dashboard
Required parameters in HC mode
HC activation requires the following valid identifiers:
| Parameter | Notes |
|---|---|
WorkspaceId |
Must be a valid Fabric workspace GUID. |
LakehouseId |
Must be a valid Fabric lakehouse GUID. |
Include SessionTag because it influences server-side matching for warm sessions.
Session matching
HC mode doesn't guarantee that every connection reuses an existing session. For each session-acquisition request, the driver sends:
- The
SessionTagvalue. - The effective Spark configuration, including
conf.*settings andEnvironmentId.
The Fabric Livy service evaluates these values and decides whether to attach the connection to a warm session or create a new session. To improve session reuse, use consistent values across connections that are intended to share server-managed Spark capacity.
Example connection strings
Minimal HC connection with Azure CLI authentication:
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};WorkspaceId=<workspace-id>;LakehouseId=<lakehouse-id>;AuthFlow=AZURE_CLI;LivyMode=HighConcurrency;SessionTag=bi-dashboard
HC connection with tuned timeouts:
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};WorkspaceId=<workspace-id>;LakehouseId=<lakehouse-id>;AuthFlow=AZURE_CLI;LivyMode=HighConcurrency;SessionTag=etl-prod;AcquireTimeoutSeconds=180;HeartbeatTimeoutInSecond=600
HC connection with an environment and Spark configuration overrides:
DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};WorkspaceId=<workspace-id>;LakehouseId=<lakehouse-id>;AuthFlow=AZURE_CLI;LivyMode=HighConcurrency;EnvironmentId=<environment-id>;SessionTag=ml-team;conf.spark.sql.shuffle.partitions=200;conf.spark.executor.memory=8g
Python with pyodbc:
import pyodbc
conn = pyodbc.connect(
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=AZURE_CLI;"
"LivyMode=HighConcurrency;"
"SessionTag=notebook-kernel"
)
Configure HC mode in a Linux DSN
Instead of repeating HC settings in each connection string, store them in ~/.odbc.ini for a user-specific DSN or /etc/odbc.ini for a system-wide DSN.
[FabricHCDSN]
Description = Microsoft Fabric Data Engineering with HC mode
Driver = Microsoft ODBC Driver for Microsoft Fabric Data Engineering
WorkspaceId = <workspace-id>
LakehouseId = <lakehouse-id>
AuthFlow = AZURE_CLI
LivyMode = HighConcurrency
SessionTag = bi-dashboard
AcquireTimeoutSeconds = 300
HeartbeatTimeoutInSecond = 600
After you save the DSN, verify it and test the connection.
odbcinst -q -s
isql -k -v "DSN=FabricHCDSN;"
When a connection string doesn't specify a value, the driver reads these settings from the DSN:
LivyModeUseHighConcurrencyEnvironmentIdSessionTagIdempotencyKeyHeartbeatTimeoutInSecondAcquireTimeoutSeconds
The driver also reads the compatibility-only TenantPrincipal and PoolMax settings. Use the parameter names in the HC settings table.
Important
The driver doesn't read conf.* Spark configuration overrides from the DSN. Add them to the connection string at runtime or inject them before calling SQLDriverConnect.
Disable HC mode
You can disable new HC session acquisitions without redeploying the driver. Use the method that corresponds to how you enabled HC mode:
| HC configuration source | How to disable HC mode |
|---|---|
| Environment variable only | Set FABRIC_ODBC_USE_HC=false, or remove the variable. |
| DSN | Set LivyMode=Classic. Alternatively, remove LivyMode and set UseHighConcurrency=false. |
| Connection string | Set LivyMode=Classic, or remove the HC parameters. |
Connection-string and DSN settings take precedence over the environment-variable fallback. These changes affect only new connections. Existing HC sessions continue until they're released.
Understand HC runtime behavior
- Validation:
SQLDriverConnectrejects invalid HC Boolean and positive-integer values with SQLSTATEHY024. It rejects an invalidLivyModeand invalid or oversizedWorkspaceId,LakehouseId, andIdempotencyKeyvalues with SQLSTATEHY000. An oversized compatibilityTenantPrincipalvalue also returnsHY000. For DSN-onlySQLConnect, the driver logs ordinary HC validation failures and uses classic mode. Oversized security-sensitive values still fail the connection. - Sensitive data: The driver excludes or scrubs
IdempotencyKeyand other sensitive fields from HC logs and diagnostic output. - Cancellation: Statement cancellation is cooperative. Server work already in progress winds down cleanly instead of being terminated immediately.
- Circuit breakers: HC and classic modes use separate circuit breakers. Exhausting one mode doesn't affect the other.
- Initial database: HC connections don't execute the initial
USEstatement for theDatabaseparameter. RunUSE <database>after connecting or use fully qualified table names. - Diagnostics: Set
LogLevel=DEBUGto log HC acquisition, polling, release, and error-mapping decisions. For focused troubleshooting, setLIVY_HC_VERBOSE=1to include additional scrubbed HC HTTP diagnostics.
DSN configuration
On Linux, configure data source names (DSNs) in INI files instead of the Windows registry.
| File | Scope | Access |
|---|---|---|
/etc/odbc.ini |
System-wide DSNs | Requires sudo |
~/.odbc.ini |
User-specific DSNs | Current user only |
Create a DSN
If ~/.odbc.ini exists, add the new DSN section without replacing the existing file. Copy the installed template only when you create the file:
if [ -e "$HOME/.odbc.ini" ]; then
echo "Preserve ~/.odbc.ini and add the new DSN section manually."
else
cp /usr/share/microsoft-fabric-odbc-driver/odbc.ini.template "$HOME/.odbc.ini"
fi
Edit ~/.odbc.ini with your Fabric workspace details:
[FabricDSN]
Description = Microsoft Fabric Data Engineering
Driver = Microsoft ODBC Driver for Microsoft Fabric Data Engineering
WorkspaceId = <workspace-id>
LakehouseId = <lakehouse-id>
AuthFlow = AZURE_CLI
LogLevel = INFO
# LogFile = /tmp/fabric_odbc.log
# LargeTableSupport = true
# ReuseSession = true
Verify the DSN
List the configured DSNs, and then test the connection:
odbcinst -q -s
isql -k -v "DSN=FabricDSN;"
The isql command requires the unixODBC command-line tools. The -k option uses SQLDriverConnect, which lets the driver process DSN connection attributes.
Use a DSN in applications
conn = pyodbc.connect("DSN=FabricDSN")
using var connection = new OdbcConnection("DSN=FabricDSN");
await connection.OpenAsync();
SQLRETURN result = SQLConnect(
hdbc,
(SQLCHAR*)"FabricDSN",
SQL_NTS,
NULL,
0,
NULL,
0);
Usage examples
Test a connection with isql
Start an interactive SQL session:
isql -k -v "DSN=FabricDSN;"
Run a single query:
echo "SELECT 1 AS test" | isql -k -v "DSN=FabricDSN;" -b
Use -k for Fabric DSNs so isql connects through SQLDriverConnect instead of SQLConnect.
Work with large result sets
import pyodbc
connection_string = (
"DRIVER={Microsoft ODBC Driver for Microsoft Fabric Data Engineering};"
"WorkspaceId=<workspace-id>;"
"LakehouseId=<lakehouse-id>;"
"AuthFlow=AZURE_CLI;"
"LargeTableSupport=true;"
"PageSizeBytes=18874368;"
"EnableAsyncPrefetch=1;"
)
conn = pyodbc.connect(connection_string)
cursor = conn.cursor()
cursor.execute("SELECT * FROM large_table")
row_count = 0
while True:
rows = cursor.fetchmany(1000)
if not rows:
break
for row in rows:
row_count += 1
if row_count % 10000 == 0:
print(f"Processed {row_count} rows")
print(f"Total rows processed: {row_count}")
conn.close()
Discover schemas and tables
import pyodbc
conn = pyodbc.connect(connection_string)
cursor = conn.cursor()
cursor.execute("SHOW TABLES")
for table in cursor.fetchall():
print(table)
cursor.execute("DESCRIBE employees")
for column in cursor.fetchall():
print(column)
cursor.execute("SHOW SCHEMAS")
for schema in cursor.fetchall():
print(schema)
conn.close()
Data type mapping
The driver maps Spark SQL data types to ODBC SQL types:
| Spark SQL type | ODBC SQL type | C/C++ type | Python type | .NET type |
|---|---|---|---|---|
BOOLEAN |
SQL_BIT |
SQLCHAR |
bool |
bool |
BYTE |
SQL_TINYINT |
SQLSCHAR |
int |
sbyte |
SHORT |
SQL_SMALLINT |
SQLSMALLINT |
int |
short |
INT |
SQL_INTEGER |
SQLINTEGER |
int |
int |
LONG |
SQL_BIGINT |
SQLBIGINT |
int |
long |
FLOAT |
SQL_REAL |
SQLREAL |
float |
float |
DOUBLE |
SQL_DOUBLE |
SQLDOUBLE |
float |
double |
DECIMAL |
SQL_DECIMAL |
SQLCHAR* |
decimal.Decimal |
decimal |
STRING |
SQL_VARCHAR |
SQLCHAR* |
str |
string |
VARCHAR(n) |
SQL_VARCHAR |
SQLCHAR* |
str |
string |
CHAR(n) |
SQL_CHAR |
SQLCHAR* |
str |
string |
BINARY |
SQL_VARBINARY |
SQLCHAR* |
bytes |
byte[] |
DATE |
SQL_TYPE_DATE |
SQL_DATE_STRUCT |
datetime.date |
DateTime |
TIMESTAMP |
SQL_TYPE_TIMESTAMP |
SQL_TIMESTAMP_STRUCT |
datetime.datetime |
DateTime |
ARRAY |
SQL_VARCHAR |
SQLCHAR* |
JSON string | string |
MAP |
SQL_VARCHAR |
SQLCHAR* |
JSON string | string |
STRUCT |
SQL_VARCHAR |
SQLCHAR* |
JSON string | string |
Platform differences
| Feature | Windows | Linux |
|---|---|---|
| Driver manager | Microsoft ODBC Driver Manager | unixODBC |
| Driver binary | microsoftfabricodbc.dll |
libmicrosoftfabricodbc.so |
| DSN configuration | Windows registry and GUI | /etc/odbc.ini and ~/.odbc.ini |
| Driver registration | Registry and odbcad32.exe |
odbcinst -i -d -f |
| HTTP client | WinHTTP | libcurl |
| TLS | Windows built-in support | OpenSSL |
| Certificate authentication | Windows CryptoAPI | OpenSSL with RS256 and PEM or PFX files |
| Interactive authentication | Browser window | Not available on headless servers |
| Packaging | MSI installer | .deb, .rpm, or .tar.gz package |
Troubleshooting
Driver not found
Problem: The connection fails with [IM002] Data source name not found and no default driver specified.
Solutions:
- Verify the driver registration by running
odbcinst -q -d. - Verify that
/usr/lib/libmicrosoftfabricodbc.soexists. - Register the driver by running
sudo odbcinst -i -d -f /usr/share/microsoft-fabric-odbc-driver/odbcinst.ini.template. - Reinstall the package by running
sudo dpkg -i microsoft-fabric-odbc-driver-2.0.0-Linux.deb.
Driver not found after a tarball installation
Problem: The driver is registered, but the shared library isn't present at the registered path.
Solutions:
- Verify that
/opt/microsoft/fabricodbc/lib/libmicrosoftfabricodbc.soexists. - Rerun
sudo ./install.sh --prefix=/opt/microsoft/fabricodbc. - If you used a custom prefix, verify that the registered driver path uses the same prefix.
DSN not found
Problem: The connection fails with [IM002] Data source name not found.
Solutions:
- Verify the DSN configuration by running
odbcinst -q -s. - Verify that
~/.odbc.inior/etc/odbc.inicontains the DSN section. - Make sure the
Drivervalue exactly matches the registered driver name.
Connection failures
Problem: The driver can't connect to Fabric.
Solutions:
- Verify that the workspace ID and lakehouse ID are valid GUIDs.
- Check Azure CLI authentication by running
az account show. - Make sure you have the required Fabric workspace permissions.
- Check network connectivity and proxy settings.
Authentication errors
Problem: Azure CLI authentication fails.
Solutions:
- Run
az loginto refresh your credentials. - Set the correct subscription by running
az account set --subscription <subscription-id>. - Check the token by running
az account get-access-token --resource https://api.fabric.microsoft.com. - Ensure your account has the required Fabric workspace permissions.
Shared library errors
Problem: The driver reports error while loading shared libraries: libmicrosoftfabricodbc.so.
Solutions:
- Reinstall the package by running
sudo dpkg -i microsoft-fabric-odbc-driver-2.0.0-Linux.deb. - Verify that
/usr/lib/libmicrosoftfabricodbc.soexists. - Run
sudo ldconfigto refresh the shared library cache.
Query timeouts
Problem: Queries time out on large tables.
Solutions:
- Add
LargeTableSupport=trueto the connection string. - Adjust
PageSizeBytesfor the result size. - Add
EnableAsyncPrefetch=1to the connection string. - Use a
LIMITclause to restrict the result size.
Enable logging
Enable detailed logging in a DSN:
[FabricDSN]
LogLevel = DEBUG
LogFile = /tmp/fabric_odbc_debug.log
Alternatively, add logging parameters to the connection string:
LogLevel=DEBUG;LogFile=/tmp/fabric_odbc_debug.log;
The driver supports the following log levels:
DEBUG: Includes detailed debugging information and is the most verbose driver log level.INFO: Includes general information and is the default.WARN: Includes warnings only.ERROR: Includes errors only.
Enable unixODBC tracing
For low-level ODBC call diagnostics, add the following configuration to /etc/odbcinst.ini:
[ODBC]
Trace = yes
TraceFile = /tmp/odbc_trace.log
Turn off tracing when you finish troubleshooting to avoid unnecessary performance overhead.