Use connected models in cloud evaluations

Connected models were previously called admin-connected models.

Connected models are models hosted behind an enterprise AI gateway, such as Azure API Management or a non-Azure AI model gateway, that an administrator connects to your Foundry project. You can use a connected model for cloud evaluation scenarios that accept a model deployment.

Foundry resolves the connection endpoint and authentication, including API key, managed identity, or OAuth 2.0 authentication. Your evaluation request simply references the connection and deployment and the service manages resolving the gateway endpoint resolution and authentication.

Prerequisites

• A Foundry project.
• Foundry User role on the Foundry project.
• For all evaluation role requirements, see Set up permissions for evaluation workflows.
• An administrator has created an Azure API Management or non-Azure AI model gateway connection on your Foundry resource and added the model under Manage > Resource details > Connections. For setup instructions, see Bring your own model to Foundry Agent Service.
• The connection name and deployment name for a model that supports the OpenAI Chat Completions API.

Reference a connected model

Use the following format anywhere a supported evaluation scenario accepts a model deployment:

<connection-name>/<deployment-name>

The following table shows common evaluation surfaces and the field that accepts the reference:

Use a connected model as an evaluator judge model

Set initialization_parameters.model when you configure an AI-assisted evaluator. This example uses the connected model as the judge for the coherence evaluator:

from azure.ai.projects.models import TestingCriterionAzureAIEvaluator

connected_model = "my-apim-connection/my-model-name"

testing_criteria = [
    TestingCriterionAzureAIEvaluator(
        type="azure_ai_evaluator",
        name="coherence",
        evaluator_name="builtin.coherence",
        initialization_parameters={"model": connected_model},
        data_mapping={
            "query": "{{item.query}}",
            "response": "{{item.response}}",
        },
    ),
]

string connectedModel = "my-apim-connection/gpt-4o";

object[] testingCriteria =
[
    new
    {
        type = "azure_ai_evaluator",
        name = "coherence",
        evaluator_name = "builtin.coherence",
        initialization_parameters = new { model = connectedModel },
        data_mapping = new
        {
            query = "{{item.query}}",
            response = "{{item.response}}"
        }
    }
];

const connectedModel = "my-apim-connection/gpt-4o";

const testingCriteria = [
    {
        type: "azure_ai_evaluator",
        name: "coherence",
        evaluator_name: "builtin.coherence",
        initialization_parameters: { model: connectedModel },
        data_mapping: {
            query: "{{item.query}}",
            response: "{{item.response}}",
        },
    },
];

• Reference: TestingCriterionAzureAIEvaluator (Python)
• Reference: OpenAI Evals API (testing_criteria, all languages)

Use a connected model as a target

Set target.model to send each evaluation input to the connected model:

connected_model = "my-apim-connection/my-model-name"

target = {
    "type": "azure_ai_model",
    "model": connected_model,
    "sampling_params": {
        "top_p": 1.0,
        "max_completion_tokens": 2048,
    },
}

string connectedModel = "my-apim-connection/gpt-4o";

object target = new
{
    type = "azure_ai_model",
    model = connectedModel,
    sampling_params = new
    {
        top_p = 1.0f,
        max_completion_tokens = 2048
    }
};

const connectedModel = "my-apim-connection/gpt-4o";

const target = {
    type: "azure_ai_model",
    model: connectedModel,
    sampling_params: {
        top_p: 1.0,
        max_completion_tokens: 2048,
    },
};

• Reference: OpenAI Evals API (data_source.target, all languages)

Use this target with the model target evaluation flow described in Run evaluations in the cloud.

Use other evaluation scenarios

Other model-based scenarios in Run evaluations in the cloud by using the Microsoft Foundry SDK work similarly with connected models. Wherever the scenario accepts a supported model deployment, replace the deployment name with <connection-name>/<deployment-name>. For example, conversation simulation accepts this reference in model_configuration.model.

Keep the Foundry project endpoint unchanged, and don't add the gateway endpoint or credentials to the evaluation request. Foundry resolves those values from the connected model connection.

Related content

• Run evaluations in the cloud by using the Microsoft Foundry SDK
• Bring your own model to Foundry Agent Service