Usare lo strumento di generazione di immagini (anteprima)

Creare un agente con lo strumento di generazione di immagini

Questo esempio crea un agente con lo strumento di generazione di immagini, genera un'immagine e la salva in un file. Selezionare Prompt Agents per usare Azure AI Projects SDK per creare un agente prompt sul lato server o Hosted Agents per usare Agent Framework FoundryChatClient per creare un agente temporaneo in-process.

Usare Python 3.10 o versione successiva per l'esempio di prompt-agent. Installa le sue dipendenze:

python -m pip install azure-ai-projects azure-identity

Agenti rapidi

import base64
import os

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition, ImageGenTool

# Format: "https://resource_name.ai.azure.com/api/projects/project_name"
PROJECT_ENDPOINT = "your_project_endpoint"
IMAGE_MODEL = "gpt-image-1"

# Create clients to call Foundry API
project = AIProjectClient(
    endpoint=PROJECT_ENDPOINT,
    credential=DefaultAzureCredential(),
)
openai = project.get_openai_client()

# Create an agent with the image generation tool
agent = project.agents.create_version(
    agent_name="agent-image-generation",
    definition=PromptAgentDefinition(
        model="gpt-5",
        instructions="Generate images based on user prompts.",
        tools=[ImageGenTool(model=IMAGE_MODEL, quality="low", size="1024x1024")],
    ),
    description="Agent for image generation.",
)
print(f"Agent created (id: {agent.id}, name: {agent.name}, version: {agent.version})")

# Generate an image using the agent
response = openai.responses.create(
    input="Generate an image of the Microsoft logo.",
    extra_headers={
        "x-ms-oai-image-generation-deployment": IMAGE_MODEL,
    },
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)

# Clean up the agent
project.agents.delete_version(agent_name=agent.name, agent_version=agent.version)

# Extract and save the generated image
image_data = [output.result for output in response.output if output.type == "image_generation_call"]
if image_data and image_data[0]:
    file_path = os.path.abspath("microsoft.png")
    with open(file_path, "wb") as f:
        f.write(base64.b64decode(image_data[0]))
    print(f"Image saved to: {file_path}")

Agenti ospitati

Questo esempio usa FoundryChatClient da Microsoft Agent Framework e chiama get_image_generation_tool() per collegare lo strumento di generazione di immagini. Installare il pacchetto con pip install agent-framework-foundry aiohttp, impostare le FOUNDRY_PROJECT_ENDPOINT variabili di ambiente e FOUNDRY_MODEL e accedere con az login.

import asyncio
import base64
import os

from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient
from azure.identity import AzureCliCredential

IMAGE_MODEL = "gpt-image-1"


async def main() -> None:
    agent = Agent(
        client=FoundryChatClient(credential=AzureCliCredential()),
        instructions="Generate images based on user prompts.",
        tools=[
            FoundryChatClient.get_image_generation_tool(
                model=IMAGE_MODEL,
                quality="low",
                size="1024x1024",
            )
        ],
    )

    result = await agent.run("Generate an image of the Microsoft logo.")

    # Extract and save the generated image from the raw response.
    for output in result.raw_representation.output:
        if output.type == "image_generation_call":
            file_path = os.path.abspath("microsoft.png")
            with open(file_path, "wb") as f:
                f.write(base64.b64decode(output.result))
            print(f"Image saved to: {file_path}")

    print(f"Agent: {result.text}")


if __name__ == "__main__":
    asyncio.run(main())

Output previsto

Lo strumento restituisce byte di immagine con codifica Base64, che l'esempio salva su disco; viene stampata anche la risposta di testo del modello:

Image saved to: /path/to/microsoft.png
Agent: Here is the generated Microsoft logo image.

Per ulteriori informazioni sulle "tool factories" di Agent Framework Foundry, consultare gli esempi del provider Foundry.

Esempio per la generazione di immagini in Azure. Intelligenza artificiale. Extensions.OpenAI

In questo esempio viene generata un'immagine in base a un prompt semplice. Il codice in questo esempio è sincrono. Per un esempio asincrono, vedere l'esempio di codice sample nell'Azure SDK per .NET repository in GitHub.

Usare l'SDK .NET 8 o versione successiva. Aggiungere i pacchetti necessari al progetto:

dotnet add package Azure.AI.Projects
dotnet add package Azure.AI.Extensions.OpenAI
dotnet add package Azure.Identity

using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using Azure.AI.Projects;
using Azure.AI.Extensions.OpenAI;
using Azure.Core;
using Azure.Core.Pipeline;
using Azure.Identity;

// Format: "https://resource_name.ai.azure.com/api/projects/project_name"
var projectEndpoint = "your_project_endpoint";
var imageModel = "gpt-image-1";

// Create the AI Project client with custom header policy
AIProjectClientOptions projectOptions = new();
projectOptions.AddPolicy(new HeaderPolicy(imageModel), PipelinePosition.PerCall);

// Create the AI Project client
AIProjectClient projectClient = new(
    endpoint: new Uri(projectEndpoint),
    tokenProvider: new DefaultAzureCredential(),
    options: projectOptions
);

// Use the client to create the versioned agent object.
// To generate images, we need to provide agent with the ImageGenerationTool
// when creating this tool. The ImageGenerationTool parameters include
// the image generation model, image quality and resolution.
// Supported image generation models include gpt-image-1.
DeclarativeAgentDefinition agentDefinition = new(model: "gpt-5")
{
Instructions = "Generate images based on user prompts.",
Tools = {
        ResponseTool.CreateImageGenerationTool(
            model: imageModel,
            quality: ImageGenerationToolQuality.Low,
            size:ImageGenerationToolSize.W1024xH1024
        )
    }
};
AgentVersion agentVersion = projectClient.AgentAdministrationClient.CreateAgentVersion(
    agentName: "myAgent",
    options: new(agentDefinition));

ProjectOpenAIClient openAIClient = projectClient.GetProjectOpenAIClient();
ProjectResponsesClient responseClient = openAIClient.GetProjectResponsesClientForAgent(new AgentReference(name: agentVersion.Name));

ResponseResult response = responseClient.CreateResponse("Generate parody of Newton with apple.");

// Parse the ResponseResult object and save the generated image.
foreach (ResponseItem item in response.OutputItems)
{
    if (item is ImageGenerationCallResponseItem imageItem)
    {
        File.WriteAllBytes("newton.png", imageItem.ImageResultBytes.ToArray());
        Console.WriteLine($"Image downloaded and saved to: {Path.GetFullPath("newton.png")}");
    }
}

// Clean up resources by deleting the Agent.
projectClient.AgentAdministrationClient.DeleteAgentVersion(agentName: agentVersion.Name, agentVersion: agentVersion.Version);

// To use image generation, provide the custom header to web requests,
// which contain the model deployment name, for example:
// `x-ms-oai-image-generation-deployment: gpt-image-1`.
// To implement it, create a custom header policy.
internal class HeaderPolicy(string image_deployment) : PipelinePolicy
{
    private const string image_deployment_header = "x-ms-oai-image-generation-deployment";

    public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
    {
        message.Request.Headers.Add(image_deployment_header, image_deployment);
        ProcessNext(message, pipeline, currentIndex);
    }

    public override async ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
    {
        // Add your desired header name and value
        message.Request.Headers.Add(image_deployment_header, image_deployment);
        await ProcessNextAsync(message, pipeline, currentIndex);
    }
}

Output previsto

Quando si esegue l'esempio, viene visualizzato l'output seguente:

Agent created (id: <agent-id>, name: myAgent, version: 1)
Image downloaded and saved to: /path/to/newton.png
Agent deleted

Creare un agente con lo strumento di generazione di immagini

Usa una shell compatibile con Bash con interfaccia della riga di comando di Azure, curl, jq e un comando base64 che supporti --decode. Impostare FOUNDRY_PROJECT_ENDPOINT prima di eseguire le richieste.

Ottenere un token di accesso:

export AGENT_TOKEN=$(az account get-access-token --scope "https://ai.azure.com/.default" --query accessToken -o tsv)

Nell'esempio seguente viene creato un agente che usa lo strumento di generazione di immagini.

curl -X POST "$FOUNDRY_PROJECT_ENDPOINT/agents?api-version=v1" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AGENT_TOKEN" \
  -d '{
    "name": "image-gen-agent",
    "description": "Test agent for image generation capabilities",
    "definition": {
      "kind": "prompt",
      "model": "gpt-5",
      "tools": [
        {
          "type": "image_generation"
        }
      ],
      "instructions": "You are a creative assistant that generates images when requested. Please respond to image generation requests clearly and concisely."
    }
  }'

Creare una risposta

curl -X POST "$FOUNDRY_PROJECT_ENDPOINT/openai/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AGENT_TOKEN" \
  -H "x-ms-oai-image-generation-deployment: gpt-image-1" \
  -d '{
    "agent_reference": {
      "type": "agent_reference",
      "name": "image-gen-agent"
    },
    "input": [{
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Please generate small image of a sunset over a mountain lake."
        }
      ]
    }],
    "stream": false
  }'

Output previsto

Il codice JSON della risposta include un image_generation_call elemento di output con un result campo contenente i dati dell'immagine con codifica Base64:

{
  "id": "resp_<id>",
  "status": "completed",
  "output": [
    {
      "type": "image_generation_call",
      "result": "<base64-encoded-image-data>",
      "status": "completed"
    },
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Here is the image of a sunset over a mountain lake."
        }
      ]
    }
  ]
}

Per estrarre e salvare l'immagine, pipe la risposta tramite jq e base64:

RESPONSE=$(curl -s -X POST "$FOUNDRY_PROJECT_ENDPOINT/openai/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AGENT_TOKEN" \
  -H "x-ms-oai-image-generation-deployment: gpt-image-1" \
  -d '{ ... }')

echo "$RESPONSE" | jq -r '.output[] | select(.type=="image_generation_call") | .result' \
  | base64 --decode > generated_image.png

Pulire l'agente REST

Eliminare l'agente dopo aver salvato l'immagine generata:

curl --request DELETE \
  --url "$FOUNDRY_PROJECT_ENDPOINT/agents/image-gen-agent?api-version=v1" \
  -H "Authorization: Bearer $AGENT_TOKEN"

Creare un agente con lo strumento di generazione di immagini

Questo esempio illustra come creare un agente di intelligenza artificiale con funzionalità di generazione di immagini usando il client Azure ai projects. L'agente genera immagini in base alle richieste di testo e le salva nei file. Per un esempio javaScript, vedere il codice sample nel repository Azure SDK per JavaScript in GitHub.

Usare Node.js 22 o versione successiva. Installare i pacchetti necessari:

npm install @azure/ai-projects @azure/identity

import { DefaultAzureCredential } from "@azure/identity";
import { AIProjectClient } from "@azure/ai-projects";
import * as fs from "fs";
import * as path from "path";
import { fileURLToPath } from "url";

// Format: "https://resource_name.ai.azure.com/api/projects/project_name"
const PROJECT_ENDPOINT = "your_project_endpoint";
const IMAGE_MODEL = "gpt-image-1";

export async function main(): Promise<void> {
  // Create clients to call Foundry API
  const project = new AIProjectClient(PROJECT_ENDPOINT, new DefaultAzureCredential());
  const openai = project.getOpenAIClient();

  // Create Agent with image generation tool
  const agent = await project.agents.createVersion("agent-image-generation", {
    kind: "prompt",
    model: "gpt-5",
    instructions: "Generate images based on user prompts",
    tools: [
      {
        type: "image_generation",
        quality: "low",
        size: "1024x1024",
      },
    ],
  });
  console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

  // Generate image using the agent
  const response = await openai.responses.create(
    {
      input: "Generate an image of Microsoft logo.",
    },
    {
      body: { agent_reference: { name: agent.name, type: "agent_reference" } },
      headers: { "x-ms-oai-image-generation-deployment": IMAGE_MODEL },
    },
  );

  // Extract and save the generated image
  const imageData = response.output?.filter((output) => output.type === "image_generation_call");

  if (imageData && imageData.length > 0 && imageData[0].result) {
    const __filename = fileURLToPath(import.meta.url);
    const __dirname = path.dirname(__filename);
    const filename = "microsoft.png";
    const filePath = path.join(__dirname, filename);

    // Decode base64 and save to file
    const imageBuffer = Buffer.from(imageData[0].result, "base64");
    fs.writeFileSync(filePath, imageBuffer);

    console.log(`Image downloaded and saved to: ${path.resolve(filePath)}`);
  } else {
    console.log("No image data found in the response.");
  }

  // Clean up resources
  await project.agents.deleteVersion(agent.name, agent.version);
}

main().catch((err) => {
  console.error("The sample encountered an error:", err);
});

Output previsto

Quando si esegue l'esempio, viene visualizzato l'output seguente:

Agent created (id: <agent-id>, name: agent-image-generation, version: 1)
Image downloaded and saved to: /path/to/microsoft.png

Usare la generazione di immagini in un agente Java

Usare JDK 17 o versione successiva e Maven 3.8 o versione successiva. Aggiungere le dipendenze a pom.xml:

Il client Java non espone attualmente l'intestazione richiesta x-ms-oai-image-generation-deployment alla creazione della risposta. Usare Java per creare la definizione dell'agente e usare la procedura REST in questo articolo per richiamare l'agente e recuperare l'immagine generata.

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-agents</artifactId>
    <version>2.4.0</version>
</dependency>
  <dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-identity</artifactId>
    <version>1.18.4</version>
  </dependency>

Creare un agente con generazione di immagini

import com.azure.ai.agents.AgentsClient;
import com.azure.ai.agents.AgentsClientBuilder;
import com.azure.ai.agents.models.*;
import com.azure.identity.DefaultAzureCredentialBuilder;

import java.util.Collections;

public class ImageGenerationExample {
  public static void main(String[] args) throws Exception {
        // Format: "https://resource_name.ai.azure.com/api/projects/project_name"
        String projectEndpoint = "your_project_endpoint";
        String imageModel = "gpt-image-1";

        AgentsClientBuilder builder = new AgentsClientBuilder()
            .credential(new DefaultAzureCredentialBuilder().build())
            .endpoint(projectEndpoint);

        AgentsClient agentsClient = builder.buildAgentsClient();
        // Create image generation tool with model, quality, and size
        ImageGenTool imageGenTool = new ImageGenTool()
            .setModel(ImageGenToolModel.fromString(imageModel))
            .setQuality(ImageGenToolQuality.LOW)
            .setSize(ImageGenToolSize.fromString("1024x1024"));

        // Create agent with image generation tool
        PromptAgentDefinition agentDefinition = new PromptAgentDefinition("gpt-5")
            .setInstructions("You are a creative assistant that can generate images based on descriptions.")
            .setTools(Collections.singletonList(imageGenTool));

        AgentVersionDetails agent = agentsClient.createAgentVersion("image-gen-agent", agentDefinition);
        System.out.printf("Agent created: %s (version %s)%n", agent.getName(), agent.getVersion());

        // Clean up
        agentsClient.deleteAgentVersion(agent.getName(), agent.getVersion());
    }
}

Output previsto

Agent created: image-gen-agent (version 1)