Manage data in Microsoft Sentinel and Defender

The Microsoft Defender portal provides a centralized experience for managing Microsoft Sentinel and Microsoft Defender XDR data. The Data & connectors experience consolidates connector management, table management, transformations, and ingestion health monitoring into a single view, so you can understand and optimize your security data without navigating between separate pages.

This article introduces the Data & connectors experience and explains how to use it to manage data in Microsoft Sentinel and Microsoft Defender XDR. For related tasks, see:

• Configure data connectors to send data to Microsoft Sentinel or Microsoft Defender XDR.
• Manage your existing tables and data.

Data & connectors in the Microsoft Defender portal (preview)

The Data & connectors experience brings connector management, table management, transformations, and ingestion health monitoring together into a single view in the Microsoft Defender portal. Use it to understand and manage your Microsoft Sentinel data without navigating between separate pages.

To open the experience, in the Microsoft Defender portal, select Setup & configuration > Data & connectors. The page includes three tabs:

• Overview tab — Workspace-level ingestion health dashboard.
• Connectors tab — Centralized inventory of data connectors, including classic Sentinel connectors and unified connectors.
• Tables tab — Table management, schema exploration, transformations, and retention.

Prerequisites

To use the Data & connectors experience in preview, you need:

• Unified role-based access control (URBAC) enabled in the Microsoft Defender portal.
• Data (manage) permission in the Data Operations permission group.
• Workspace permissions equivalent to Log Analytics Contributor, including:
  • Microsoft.OperationalInsights/workspaces/write
  • Microsoft.OperationalInsights/workspaces/tables/write

Known limitations

The following limitations apply to the preview:

• The Data Sources tab on the table details panel appears only for federated tables. Support for other table types is planned.
• The Data & connectors experience is available in the commercial cloud only. Sovereign clouds, including US Government and China, aren't supported.

Overview tab

The Overview tab is a workspace-level dashboard that surfaces ingestion health signals so you can act on issues before they impact detections. Filter by time range and workspace to scope the dashboard.

The Overview tab includes:

• Ingestion volume — Daily ingestion split by Analytics tier and Data Lake tier for the selected time range.
• Connectors that need attention — Connectors with credential errors, ingestion failures, or outages, along with the error type, failure count in the last 24 hours, and last successful ingestion.
• Table ingestion fluctuations — Tables with unexpected volume changes compared to the same day the previous week, grouped as spikes and drops. You can adjust the fluctuation thresholds.
• Data ingestion volume by table — Top ingesting tables ranked by daily volume, so you can identify high-volume tables driving ingestion costs.
• Ingestion latency by table — The time between data generation and availability in your environment, so you can identify connector throttling or source-side delays.

Connectors tab

The Connectors tab brings unified data connectors and Sentinel classic data connectors together in a single inventory. Use it to review connector health, add new connectors, or open a connector's details view.

The Connectors tab includes:

• Unified connector inventory — All onboarded connectors, with status, provider, content source, and available updates.
• Toggle between unified and classic connectors — Use the classic connectors selector to switch between the unified connector catalog and the Sentinel classic data connectors inventory, which includes connectors built with the Codeless Connector Framework (CCF).
• Filter by provider, data type, and status — Narrow the list to find specific connectors quickly.
• Data onboarding assistant — Guided flow to add new connectors.
• Create custom connector — Start a new custom connector from the Connectors tab.

Tables tab

Data you collect into Microsoft Sentinel and Microsoft Defender XDR is stored in tables. The Tables tab is where you view your tables, review their ingestion health, inspect their schema and transformations, and manage the retention period and storage costs associated with your data.

Which tables can you manage in the Microsoft Defender portal?

This section describes the table types you can manage in the Microsoft Defender portal.

How data tiers and retention work

You can retain data in Microsoft Sentinel in one of two tiers:

• Analytics tier: This tier makes data available for alerting, hunting, workbooks, and all Microsoft Sentinel features. It retains data in two states:

  • Analytics retention: In this "hot" state, data is fully available for real-time analytics, including high-performance queries, analytics rules, and threat hunting. By default, Microsoft Sentinel and Microsoft Defender XDR retain data in this tier for 30 days. You can extend the retention period of all tables to up to two years at a prorated monthly long-term retention charge. You can extend the retention period of Microsoft Sentinel solution tables to 90 days for free.
  • Total retention: By default, all data in the analytics tier is mirrored to the data lake for the same retention period. You can extend the retention of your data in the lake beyond the analytics retention, for up to 12 years of total retention at a low cost.

• Data Lake tier: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the data lake only. Data in the Data Lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through KQL jobs, analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules.

• XDR data: By default, Microsoft Defender XDR threat hunting data is always available in the Analytics tier for 30 days. You can extend retention of this data in the Analytics tier up to 90 days, which incurs ingestion costs, while storage remains free. Extending retention beyond 90 days in the Analytics tier also incurs storage costs. You can also ingest exclusively into the Data Lake tier, but the data remains available in the Analytics tier for 30 days. Ingesting XDR data directly into the Data Lake tier is more cost-effective, but it involves ingestion, storage, and processing costs. In this option, customers still get 30 days' worth of data in the Analytics tier at no additional cost.

For more information about the differences between the Analytics tier and the Data Lake tier, see Compare the analytics and data lake tiers.

The following diagram illustrates the retention components of the Analytics, Data Lake, and XDR default tiers, and the table types that apply to each tier:

For more information about the Microsoft Sentinel data lake, see What is Microsoft Sentinel data lake.

Use Table insights for table-level observability

Table insights on the Tables tab helps you understand ingestion health at the table level for the selected Microsoft Sentinel workspace. It gives you quick observability signals so you can identify ingestion and cost patterns before you change tier or retention settings.

Table insights signals appear in two places on the Tables tab:

• Summary tiles at the top of the page show workspace-level counts, including the number of tables and how they're distributed across the Analytics tier, Data Lake tier, and XDR default tier.
• Columns in the tables grid show per-table health signals for each table, including:
  • Last data received
  • Average daily ingestion volume
  • Estimated daily ingestion cost
  • Volume anomaly (day-over-week ingestion fluctuation)

Use the grid columns to sort, filter, and compare tables side by side. Combine these signals to detect connector failures, investigate cost spikes, validate new connector onboarding, and identify candidates for right-sizing retention and tier.

For setup, access, and interpretation guidance, see Configure table settings in Microsoft Sentinel.

Table details panel

Select a table on the Tables tab to open the table details panel. The panel consolidates schema, transformations, and retention information for the selected table. It includes the following tabs:

Compare the analytics and data lake tiers

This table compares the Analytics and Data Lake tiers and their key characteristics:

What happens when you modify table settings

You can switch a table's tier and retention settings at any time.

When you change a table's tier from Analytics to Data Lake, all real-time analytics and hunting queries stop working.

When you shorten a table's total retention, Microsoft waits 30 days before removing the data, so you can revert the change and avoid data loss if you made a configuration error.

When you increase total retention, the new retention period applies to all data that was already ingested into the table and wasn't yet removed.

When you change the analytics retention settings of a table with existing data, the change takes effect immediately.

Example:

• You have a table in the Analytics tier with 180 days of analytics retention. By default, total retention is also set to 180 days.
• You change analytics retention to 90 days without changing total retention, which remains 180 days.
• Microsoft Sentinel automatically removes the last 90 days of data from analytics retention, but continues to store data that's 90-180 days old in the Data Lake tier.

Manage XDR data in Microsoft Sentinel

By default, Microsoft Defender XDR retains threat hunting data in the XDR default tier for 30 days. This data isn't ingested into the Analytics or Data Lake tiers by default. If you extend the retention period of supported XDR tables beyond 30 days and up to 90 days, Sentinel ingestion costs apply, but there are no additional storage costs. The tables are created in your Microsoft Sentinel workspace in the Analytics tier and mirrored to the Data Lake tier.

If you enable the Microsoft Sentinel XDR connector in the Azure portal, the tables you select during setup are automatically ingested into the Analytics tier and mirrored to the Data Lake tier. The default retention is 30 days, and you can extend it up to 12 years. For a list of tables, see Microsoft Defender XDR integration with Microsoft Sentinel. You can ingest supported XDR tables that you didn't select during connector deployment into the Analytics tier and mirror them to the Data Lake tier by setting retention to more than 30 days.

If you don't enable the Microsoft Sentinel XDR connector, XDR tables aren't automatically ingested, but you can still ingest them by setting Analytics or Data Lake tier retention to more than 30 days in the Microsoft Defender portal.

You can choose to ingest supported XDR tables exclusively into the Data Lake tier by selecting the Data Lake tier option when configuring retention settings. For more information, see Configure data retention and tiering.

To stop ingesting data into the Analytics tier, reset analytics retention and total retention to the default 30 days.

For more information about managing your tables and data, see Manage your existing tables and data.

XDR data retention and costs

The following tables summarize the free retention periods and cost implications for the different tiers in Microsoft Sentinel:

For more information about billing and costs, see Understand the full billing model for Microsoft Sentinel.

In the following examples, XDR data is available through advanced hunting for at least 30 days, regardless of the retention settings in the analytics or data lake tiers.

Next steps

Learn more about:

• Configuring log table tiers and retention settings
• Creating a search job to retrieve data matching particular criteria