Microsoft AI and machine learning products

Microsoft offers several platforms for AI and machine learning. Each platform targets a different combination of the audience, workload, data foundation, and deployment model. This article helps solution architects, machine learning engineers, data scientists, and application developers choose a starting platform, understand how the platforms differ, and learn how to combine them in real-world architectures. Coverage includes classical machine learning, analytics-integrated machine learning, and generative AI workloads across Azure, Microsoft Fabric, and Microsoft Foundry. This article focuses on Microsoft-provided solutions.

Use this article to compare options and design an end-to-end solution.

Primary workload scenarios

Start with the AI or machine learning feature that you need to add to your workload. The following table maps common feature scenarios to a recommended starting platform for custom AI and machine learning capabilities. Use it as an initial orientation before you review platform capabilities and walk through the decision tree.

Compare platform capabilities

The following matrix summarizes where each platform is the primary fit, where it offers complementary support, and where the capability isn't a focus. The matrix uses these icons:

• ✅ Primary capability
• ⚠️ Supported or complementary capability
• ❌ Not a focus

The following notes clarify specific ratings in the matrix:

• Foundry Tools is rated as complementary for generative AI and agents because services like Azure AI Search, Content Safety in Foundry Control Plane, and Azure Document Intelligence in Foundry Tools are common building blocks for RAG and agent solutions. The primary platform for generative AI development is Microsoft Foundry.

• Microsoft Foundry governance comes primarily from Foundry Control Plane, which provides observability, runtime controls, security integration, and fleet management for agents and applications.

• Azure Machine Learning supports generative AI through access to Foundry Models, managed endpoints, evaluation capabilities, and fine-tuning for supported foundation models. It isn't the primary platform for generative AI development.

• Azure Databricks earns a primary rating for governance because Unity Catalog provides unified data and model governance with role-based access control (RBAC) and OpenSharing.

• Microsoft Fabric is rated as complementary for governance because OneLake, Microsoft Purview integration, sensitivity labels, and domain-based controls cover data assets well. Microsoft Fabric includes a built-in MLflow-based model registry for storing and tracking models. Microsoft Fabric also has a built-in lineage view for machine learning models and experiments, including relationships to lakehouses and code items. But Microsoft Fabric doesn't include a responsible AI dashboard. For broader model lifecycle governance, pair Microsoft Fabric with Azure Machine Learning.

• Microsoft Fabric is rated as complementary for real-time inference because Microsoft Fabric Real-Time Intelligence covers streaming analytics, and Microsoft Fabric machine learning model endpoints provide managed online REST endpoints for real-time predictions. Model endpoints are in preview and support a limited set of model flavors. For broad production online serving, you can also pair Microsoft Fabric with Azure Machine Learning managed online endpoints or Microsoft Foundry.

• Foundry Local extends Microsoft Foundry to on-device inference for edge and offline scenarios.

• Azure Arc extends Azure Machine Learning to on-premises Kubernetes clusters for hybrid training or scoring.

• Microsoft Fabric Data Science includes a low-code AutoML experience. Code-first AutoML is available through Fast and Lightweight AutoML (FLAML) APIs in Microsoft Fabric notebooks. For the current status, check the Microsoft Fabric documentation.

Narrow your platform choice

Use the following decision tree to narrow your choice of starting platform. The tree helps you select a primary platform for a workload. Most production solutions combine more than one platform, so use the product-specific sections in this article to refine your choice. Also consider using a compound architecture pattern. For more information, see Combine platforms.

Download a Visio file of this decision tree.

Step-by-step walkthrough

Use the following steps to walk through the decision tree. When you reach a platform choice, skip the remaining steps.

1. Identify the feature scenario.

   Make platform choices at the feature level within your workload. Each feature can have unique functional and nonfunctional requirements that one or more technologies can serve. When you use this flowchart, consider the requirements of the specific use case that you want to solve.

   • If prebuilt AI features such as vision, speech, language, search, document understanding, or content safety meet your workload's needs, use Foundry Tools. Many services in Foundry Tools support customization, such as custom text classification in Azure Language or custom models in Azure Document Intelligence. Consider these services before you commit to full custom training or a broader approach like using a language model to simulate a specialized service. For service-specific documentation, see the following articles:
     • Azure Vision in Foundry Tools
     • Azure Speech in Foundry Tools
     • Azure Language in Foundry Tools
     • Azure AI Search
     • Azure Document Intelligence in Foundry Tools
     • Azure Content Understanding in Foundry Tools
     • Content Safety in Foundry Control Plane
   • If your workload requires generative AI or custom AI agents to add nondeterministic behavior, continue to step 2.
   • If the feature requires custom machine learning or deep learning capabilities, such as classical machine learning, computer vision, forecasting, or fine-tuning your own models, continue to step 3.
   • If you need to integrate machine learning into a broader data and BI workflow, continue to step 4.

2. For generative AI and agents, choose the Microsoft platform that matches your developer profile.

   • Direct large language model (LLM) API access only: Use Azure OpenAI in Microsoft Foundry.

   • Code-first agent or multi-agent development: Use the Microsoft Agent Framework SDK paired with the Foundry Agent Service for hosting and orchestration.

   • Full generative AI platform (Foundry Models, fine-tuning, evaluation, safety, observability, and governance): Use Microsoft Foundry.

3. For custom machine learning and deep learning, base your decision on where training and inference must run.

   • On a Windows app or device: Use Microsoft Foundry on Windows and Windows ML for local model inference with hardware acceleration. Use ML.NET when a .NET application needs local classical machine learning, AutoML, or Open Neural Network Exchange (ONNX) model integration.

   • On an IoT gateway or edge appliance: Use Azure IoT Edge, Foundry Local, Azure Local, or Azure Stack Edge for local or hardware-accelerated inference.

   • Inside a SQL database: Use SQL intelligent applications and AI for in-database prediction, vector retrieval, and AI endpoint integration. Use SQL Server Machine Learning Services or Azure SQL Managed Instance Machine Learning Services when you need Python or R execution in the database.

   • In an on-premises or multicloud Kubernetes cluster: Attach the cluster to Azure Machine Learning through Azure Arc so that data stays in place while training and inference are orchestrated through Azure.

   • On Spark-based or distributed compute: Continue to step 5.

   • On cloud-managed compute with full MLOps: Use Azure Machine Learning.

4. For analytics-integrated machine learning, choose the solution that matches your data and BI pattern.

   • Unified SaaS with Power BI on OneLake: Use the Microsoft Fabric Data Science workload.

   • Existing Azure Synapse Analytics investment: Use Machine learning in Azure Synapse Analytics. For new investments, prefer Microsoft Fabric.

   • Lakehouse-scale Spark and machine learning: Use Azure Databricks.

5. For Spark-based or distributed compute, base your decision on whether you use lakehouse or managed Spark clusters.

   • Lakehouse with MLflow and Unity Catalog governance: Use Azure Databricks. You can still publish models through Azure Machine Learning managed endpoints for serving and governance.

   • Managed Spark clusters on Azure with native storage and security integration: Use Apache Spark in Azure HDInsight.

Azure Machine Learning

Azure Machine Learning is a fully managed cloud service for training, deploying, and managing machine learning models at scale. It supports tens of thousands of open-source Python packages, including TensorFlow, PyTorch, and scikit-learn. It provides tooling for the full machine learning lifecycle, from data preparation through deployment, MLOps, and responsible AI.

Use Azure Machine Learning when you need the following capabilities:

• Custom training. You use custom model training for classical machine learning, deep learning, or fine-tuning of open-source LLMs.

• Production MLOps. You use a production MLOps lifecycle that includes experiment tracking, model versioning, governed registries, CI/CD pipelines, and production monitoring.

• Managed endpoints. You use managed online or batch inferencing endpoints that are hosted in an environment that provides autoscaling, managed identity, and monitoring.

• Hybrid compute. You use hybrid training or inferencing on Azure Arc–enabled Kubernetes clusters.

The following table summarizes the platform capabilities that Azure Machine Learning provides across the machine learning lifecycle.

The following table summarizes core adoption details for Azure Machine Learning.

Azure Machine Learning isn't the best fit in the following situations:

• You only need to consume pretrained APIs for vision, speech, or language. In this case, use Foundry Tools.

• Machine learning is a small extension of an analytics workload. In this case, Microsoft Fabric or Azure Databricks might be a better starting point.

• Your workload is end-to-end generative AI (model selection, agent orchestration, evaluation, deployment, and monitoring). Also, you don't need classical machine learning training or the Azure Machine Learning asset, run, and MLflow model. In this case, Microsoft Foundry alone can be sufficient. An Azure Machine Learning workspace is required only when you need Azure Machine Learning training compute, MLOps lifecycle, or managed online or batch endpoints for custom models.

Microsoft Foundry

Microsoft Foundry is the unified Azure platform for building, optimizing, and governing generative AI applications and AI agents. Microsoft Foundry combines the Foundry model catalog, agent and orchestration services, evaluation and safety tooling, and a control plane for runtime governance. For code-first agent development, Microsoft Foundry pairs with the Microsoft Agent Framework, the unified Microsoft SDK that supersedes the prior split between Semantic Kernel and AutoGen.

Use Microsoft Foundry when you need the following capabilities:

• Generative AI or agents. You build generative AI or agent-based solutions, such as multimodel chatbots or orchestrated LLM workflows.

• Model discovery and evaluation. You need to be able to discover, compare, and evaluate foundation models from multiple providers.

• Model customization and governance. You want to fine-tune, evaluate, and deploy foundation models in a managed environment that provides built-in observability, content safety, and runtime controls.

The following table summarizes the Microsoft Foundry capabilities that support generative AI applications and agent development.

Foundry Toolkit for Visual Studio Code

Foundry Toolkit for Visual Studio Code is the primary developer experience for Microsoft Foundry. Use it when you want a local-first workflow for generative AI applications and agents. The toolkit supports model discovery, local model execution, prompt and agent development, tool integration, debugging, tracing, evaluation, fine-tuning, and deployment to Microsoft Foundry from Visual Studio Code.

Start in Foundry Toolkit when the solution needs code, source control, local iteration, agent inspection, MCP tool integration, or model testing before deployment. Use the Foundry portal for proof-of-concept exploration, project setup, resource review, evaluation review, and operational visibility after applications and agents are running.

Foundry portal and Azure Machine Learning studio

The Foundry portal and Azure Machine Learning studio are complementary web experiences. Use them for prototyping, project and resource management, evaluation review, and operational visibility.

The following table summarizes core adoption details for Microsoft Foundry.

Microsoft Foundry isn't the best fit in the following situations:

• You consume prebuilt AI capabilities such as optical character recognition (OCR), speech-to-text transcription, translation, or content moderation through an API, and you don't require model selection, orchestration, or agents. In this case, use Foundry Tools.

• Your solution requires custom model training, for classical machine learning or deep learning, and full MLOps capabilities. In this case, use Azure Machine Learning. Microsoft Foundry doesn't offer broad provisioned training compute or AutoML for tabular data.

• Your solution requires Spark-scale data engineering. In this case, use Azure Databricks or Microsoft Fabric.

Foundry Tools

Foundry Tools, formerly Azure AI services and Cognitive Services, is a family of pretrained AI capabilities that you consume through REST APIs and SDKs. Each tool targets a specific capability, so you can add intelligent features to applications without building or training a custom model. For help deciding which service to use, see Choose an AI services technology.

Use Foundry Tools in the following situations:

• A pretrained API covers the scenario, so you don't need to train a custom model.

• You need standard, managed APIs and API-based integration with existing applications.

The following table groups commonly used services by capability area. Generative AI models, agents, and orchestration are part of Microsoft Foundry and are listed separately for reference.

To compare the services within a capability area and choose one, see the following detailed selection guides:

• Image and video processing
• Targeted language processing
• Speech recognition and generation
• Vector search
• AI services technology

The following table summarizes core adoption details for Foundry Tools.

Foundry Tools isn't the best fit in the following situation: Your scenario requires custom model architectures or training data beyond the supported customization options. In these cases, use Azure Machine Learning for classical or deep learning models, or use Microsoft Foundry for generative AI fine-tuning.

Microsoft Fabric

Microsoft Fabric is a unified, SaaS-based analytics platform. It integrates data engineering, data science, data warehousing, real-time intelligence, and Power BI on a shared OneLake data foundation. Data engineers, data scientists, and business analysts who use Microsoft Fabric collaborate in the same managed environment.

The Microsoft Fabric Data Science workload is the part that's most relevant to machine learning. It provides Spark notebooks, AutoML, a built-in MLflow experience for experiment tracking, and a model registry. For detailed information, see What is Data Science in Microsoft Fabric?.

Use Microsoft Fabric when you need the following capabilities:

• Analytics-integrated machine learning. Machine learning is part of a broader analytics workflow in your solution, such as building a model on data in a lakehouse and serving predictions to Power BI.

• Fully managed SaaS. You want a fully managed SaaS experience with minimal infrastructure to manage.

• Shared data foundation. Your team works across data engineering, data science, and BI on the same OneLake foundation.

The following table summarizes Microsoft Fabric capabilities that are most relevant to analytics-integrated machine learning.

The following table summarizes core adoption details for Microsoft Fabric.

Microsoft Fabric isn't the best fit in the following situation: You need deep learning at scale, comprehensive MLOps with governed registries, or fine-grained compute control. In these cases, complement Microsoft Fabric with Azure Machine Learning or Azure Databricks. Microsoft Fabric covers analytics-integrated machine learning well, but it isn't a replacement for a dedicated machine learning platform.

Azure Databricks

Azure Databricks is a unified, open analytics platform for building, deploying, sharing, and maintaining data, analytics, and AI solutions. It combines data warehousing and data lake capabilities in a lakehouse architecture and integrates with Azure storage and security.

Use Azure Databricks when you need the following capabilities:

• Spark-scale machine learning. You need to run machine learning on very large datasets or distributed Spark clusters.

• Unified lakehouse workflow. You combine data engineering, feature engineering, experimentation, and model development in one lakehouse workflow.

• Production machine learning operations. You want to deploy, manage, and monitor machine learning solutions in production by using managed serving endpoints, automated workflows, and MLOps pipelines.

• Unity Catalog governance. You want to govern data, features, models, and functions in Unity Catalog, which provides centralized access control and lineage.

• Generative AI and agents. You build generative AI and AI agent workloads in Azure Databricks, including custom and foundation model scenarios.

The following table lists notable Azure Databricks capabilities:

The following table summarizes core adoption details for Azure Databricks.

Azure Databricks isn't the best fit in the following situations:

• Your team doesn't need Spark-scale processing. In this case, the cluster overhead might not be justified.

• Your scenario uses purely API-driven AI. In this case, Foundry Tools or Microsoft Foundry offers a more direct path.

• You need Azure Machine Learning–specific governance controls, endpoint capabilities, or a standardized endpoint layer across teams. In these cases, use Azure Machine Learning managed endpoints in a hybrid design.

SQL intelligent applications and machine learning

SQL in-database AI keeps governed relational data where it already lives and adds prediction, vector retrieval, and AI endpoint integration directly in Transact-SQL (T-SQL).

Use SQL in-database AI in the following situations:

• Data must remain inside your relational database for security, privacy, latency, or regulatory reasons.

• You need low-latency scoring close to transactions, such as fraud checks or next-best-action decisions.

• You want vector search and retrieval directly in the database for RAG and semantic matching patterns.

• You need database-native invocation of external AI endpoints from T-SQL for orchestration or agent-callable workflows.

The following table lists current platforms and tools:

The following table summarizes core adoption details for Azure SQL machine learning.

Combine Azure SQL machine learning with another platform when you want to take the following approach:

• Train externally, score in SQL. For large-scale or deep learning training, use Azure Machine Learning, Azure Databricks, or Microsoft Fabric for training. Then bring inference artifacts back to SQL for in-database scoring.

• Cloud-train, database-deploy, SQL-predict. Train externally, export a scoring artifact supported by the target SQL platform, and then use PREDICT to execute predictions in SQL close to transactional data.

• Generate embeddings in SQL, call endpoints for generation. For RAG and agent scenarios, generate embeddings and vector retrieval in SQL. Then, if you need response generation, use sp_invoke_external_rest_endpoint to call external model endpoints from SQL.

Edge, local, and on-premises machine learning

For machine learning that runs outside the cloud, on local devices, in offline environments, embedded in applications, or inside on-premises infrastructure, Microsoft provides several options.

Use edge, local, and on-premises machine learning when you need the following capabilities:

• Local inference. To meet latency, offline operation, or data privacy requirements, your scenario needs inference to run locally. For example, predictive maintenance solutions in environments with intermittent connectivity require local inference.

• The ability to embed machine learning in applications. You build desktop or mobile applications that need embedded machine learning capabilities.

• On-premises execution. Your scenario needs machine learning execution to stay inside an on-premises environment, such as a local SQL Server deployment.

Microsoft provides these options:

• Microsoft Foundry on Windows includes Windows ML for local inference on Windows devices with hardware acceleration across CPUs, GPUs, and NPUs. Use Windows ML for new Windows app and device inference workloads.

• ML.NET is an open-source, cross-platform machine learning framework for .NET developers. It supports data loading, training (including AutoML for .NET), and local inference. Use ML.NET when the application needs .NET-native model training or inference, including ONNX model integration.

• SQL Server Machine Learning Services supports training and scoring inside on-premises SQL Server when data must remain in the local data tier.

• Foundry Local is an on-device AI inference solution that extends Microsoft Foundry to edge and offline scenarios.

• Azure Local provides Azure-managed infrastructure for local virtual machines (VMs) and containers. Use it to host edge AI components close to data, such as Foundry Local workloads, RAG retrieval components, or ONNX models exported from Azure Machine Learning.

• Kubernetes compute for Azure Machine Learning attaches Azure Arc–enabled Kubernetes clusters to an Azure Machine Learning workspace so that on-premises or multicloud clusters run training and inference while orchestration stays in Azure.

• Azure IoT Edge deploys containerized machine learning models to edge devices so that inference runs locally.

• Azure Stack Edge provides Microsoft-managed edge appliances that you can use for hardware-accelerated machine learning and VM or containerized workloads. Devices are available only for validated partner workloads or deployments of at least 100 nodes.

A typical pattern trains an exportable model in the cloud by using Azure Machine Learning or Azure Databricks. The model is then converted to ONNX when the model and target runtime support that format. Finally, the model is deployed for edge inference through Windows ML, ML.NET, an Azure IoT Edge module, Foundry Local, or Azure Local infrastructure.

Combine platforms

Production solutions can combine more than one platform. The flowchart earlier in this article helps you pick a primary starting platform. Use the following table to plan compound architectures and identify the supporting Microsoft components that each scenario typically requires.

Development platforms and tools

Most teams use cross-cutting development tools and supporting services alongside the platform they pick as the primary one. These tools influence where AI and machine learning work is developed, tested, and packaged before it runs. The supporting services influence where inference is hosted or where compute attaches to data that can't move to a managed cloud platform.

This section separates the primary developer tools, which are the day-to-day development and operational platforms, from the adjacent supporting services that host self-managed inference, accelerate Spark-based machine learning, or extend training and scoring to on-premises infrastructure.

The following tools support development across the cloud and on-premises platforms.

Adjacent supporting services help with experimentation, self-managed inference hosting, Spark-based machine learning, or hybrid operation, but they aren't the primary platform choices in the decision flow.

Contributors

Microsoft maintains this article. The following contributors wrote this article.

Principal author:

• Mahdi Setayesh | Principal Software Engineer

To see nonpublic LinkedIn profiles, sign in to LinkedIn.

Next steps

• What is Azure Machine Learning?
• Microsoft Foundry documentation
• What is Microsoft Fabric?
• What are Foundry Tools?
• How Azure Machine Learning works
• Microsoft AI
• Microsoft Learn training for AI engineers and data scientists

Related resources

• Choose an AI services technology
• Choose an Azure service for vector search
• AI technology overview