Get values for environment variables

The Microsoft 365 Copilot Agent Evaluations CLI (@microsoft/m365-copilot-eval) reads connection and authentication settings from environment variables. This article shows you where to obtain each value.

For the list of variables and where to put them, see Agent Evaluations CLI overview and Quickstart: Use the Agent Evaluations CLI.

Before you begin

Gather the following items before you collect values:

• Access to the Azure subscription that hosts your Azure OpenAI model in Microsoft Foundry project.
• Sign-in access to the Microsoft 365 tenant where your Copilot agent is deployed.
• The Azure AI Developer role on the Microsoft Foundry project. If you can't deploy models or access the project, ask your subscription administrator.

Get your tenant ID (TENANT_ID)

TENANT_ID is the Microsoft Entra tenant ID where your agent is deployed. It's a GUID in the form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

Use any of the following methods.

Option 1: Microsoft Entra admin center

1. Sign in to the Microsoft Entra admin center with an account in the target tenant.
2. Select Identity > Overview.
3. Copy the Tenant ID value.

Option 2: Azure CLI

If you have the Azure CLI installed:

az account show --query tenantId --output tsv

If you belong to multiple tenants, sign in to the correct one first:

az login --tenant <tenant-domain-or-id>

Option 3: Azure portal

1. Sign in to the Azure portal.
2. Select Microsoft Entra ID from the portal menu.
3. Copy the Tenant ID from the Overview page.

Deploy a GPT-5 mini model in Microsoft Foundry

The evaluation tool uses a Microsoft Foundry project with a GPT-5 model to score responses with cloud evaluation. GPT-5 mini is recommended, but you can use any GPT-5 model. You need the project endpoint (AZURE_AI_PROJECT_ENDPOINT) and the model deployment name (AZURE_AI_MODEL_NAME). The Foundry path authenticates with Microsoft Entra, so you don't need an API key.

If you don't already have a Microsoft Foundry project with a deployed model, follow these steps to create one:

1. Sign in to the Azure portal.

2. In the search bar, enter OpenAI and select Azure OpenAI from the results.

   

3. Select Create to create a Microsoft Foundry resource.

   

4. On the Create AI Foundry resource form, fill in the details, and then select Review + create.

   

5. After the resource is deployed, open the Microsoft Foundry portal.

   

6. In the left navigation, select Models + endpoints.

   

7. Select Deploy model > Deploy base model. Select the gpt-5-mini model. GPT-5 mini is supported for the Foundry cloud-evaluation path.

   

8. Select Confirm, and then select Customize.

   

9. Adjust the capacity as needed for your evaluation workload.

   

   

10. Select Deploy and wait for the deployment to report Succeeded.

11. From the Foundry project home page, copy the Project endpoint. This value is for AZURE_AI_PROJECT_ENDPOINT.

12. Under Recent work, select the GPT-5 mini deployment. On the deployment's Details tab, copy the deployment Name. This value is for AZURE_AI_MODEL_NAME.

13. Add these values to your env file. For details, see Step 3 of the Quickstart.

Sign in with Microsoft Entra

The Foundry cloud evaluation path uses Microsoft Entra authentication instead of an API key. Sign in with the Azure CLI before you run evaluations:

az login

If the Foundry project is in a different tenant, sign in to that tenant:

az login --tenant <tenant-id>

You need the Azure AI Developer role on the Foundry project.

Get your agent ID (M365_AGENT_ID)

M365_AGENT_ID identifies the Copilot agent that runevals sends test prompts to. The CLI accepts two formats:

• User-scoped: U_xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
• Tenant-scoped: T_<agent-name>.declarativeAgent

Microsoft 365 Agents Toolkit projects

If you built your agent by using the Microsoft 365 Agents Toolkit, you don't need to set M365_AGENT_ID directly. The CLI auto-detects the agent ID from M365_TITLE_ID, which the Agents Toolkit writes to .env.local during provisioning.

To confirm, open .env.local in your project root and look for a line similar to:

M365_TITLE_ID="U_0dc4a8a2-b95f-edac-91c8-d802023ec2d4"

Non-Microsoft 365 Agents Toolkit projects

Set M365_AGENT_ID explicitly in env/.env.<environment>. Get the value from the location where your agent was published - for example, the Microsoft Teams admin center, the Microsoft 365 admin center, or the deployment output from your publishing pipeline.

Override the agent at run time

To evaluate a different agent without editing env files, pass --m365-agent-id on the command line:

runevals --m365-agent-id "U_0dc4a8a2-b95f-edac-91c8-d802023ec2d4"

For details, see --m365-agent-id in the CLI reference.

Set the model deployment

Two variables control which Microsoft Foundry project and model deployment the CLI uses for scoring. AZURE_AI_MODEL_NAME has a working default, so you can accept the default or set both values explicitly in your env file:

AZURE_AI_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
AZURE_AI_MODEL_NAME="gpt-5-mini"   # deployment name in Foundry

AZURE_AI_MODEL_NAME must match the deployment name exactly as it appears in your Microsoft Foundry project.

Get AZURE_AI_MODEL_NAME from Foundry

1. Sign in to the Microsoft Foundry portal and open your project.
2. Select Models + endpoints.
3. Copy the Deployment name of the model you want to use for scoring.

Validate your values

After you gather the values and add them to the appropriate env file (see Step 3 of the Quickstart), run:

runevals --init-only --log-level debug

This command loads your environment, initializes the Python runtime, and surfaces configuration errors without running evaluations. If it completes without error, your values are loaded correctly.

Troubleshooting

Use the following table to troubleshoot issues you might encounter.

Related content

• Agent Evaluations CLI overview
• Quickstart: Use the Agent Evaluations CLI
• Agent Evaluations CLI reference
• Troubleshooting and support