Use a ferramenta de geração de imagens (pré-visualização)

Crie um agente com a ferramenta de geração de imagens

Este exemplo cria um agente com a ferramenta de geração de imagem, gera uma imagem e guarda-a num ficheiro. Selecione Prompt Agents para usar o SDK Azure AI Projects para criar um agente de prompt do lado do servidor, ou Hosted Agents para usar o Agent Framework FoundryChatClient para construir um agente efémero em processo.

Utilize Python 3.10 ou posterior para o exemplo de prompt-agent. Instale as suas dependências:

python -m pip install azure-ai-projects azure-identity

Agentes de comando

import base64
import os

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition, ImageGenTool

# Format: "https://resource_name.ai.azure.com/api/projects/project_name"
PROJECT_ENDPOINT = "your_project_endpoint"
IMAGE_MODEL = "gpt-image-1"

# Create clients to call Foundry API
project = AIProjectClient(
    endpoint=PROJECT_ENDPOINT,
    credential=DefaultAzureCredential(),
)
openai = project.get_openai_client()

# Create an agent with the image generation tool
agent = project.agents.create_version(
    agent_name="agent-image-generation",
    definition=PromptAgentDefinition(
        model="gpt-5",
        instructions="Generate images based on user prompts.",
        tools=[ImageGenTool(model=IMAGE_MODEL, quality="low", size="1024x1024")],
    ),
    description="Agent for image generation.",
)
print(f"Agent created (id: {agent.id}, name: {agent.name}, version: {agent.version})")

# Generate an image using the agent
response = openai.responses.create(
    input="Generate an image of the Microsoft logo.",
    extra_headers={
        "x-ms-oai-image-generation-deployment": IMAGE_MODEL,
    },
    extra_body={"agent_reference": {"name": agent.name, "type": "agent_reference"}},
)

# Clean up the agent
project.agents.delete_version(agent_name=agent.name, agent_version=agent.version)

# Extract and save the generated image
image_data = [output.result for output in response.output if output.type == "image_generation_call"]
if image_data and image_data[0]:
    file_path = os.path.abspath("microsoft.png")
    with open(file_path, "wb") as f:
        f.write(base64.b64decode(image_data[0]))
    print(f"Image saved to: {file_path}")

Agentes alojados

Este exemplo utiliza FoundryChatClient do Microsoft Agent Framework e chama get_image_generation_tool() para anexar a ferramenta de geração de imagens. Instale o pacote com pip install agent-framework-foundry aiohttp, defina as variáveis de ambiente FOUNDRY_PROJECT_ENDPOINT e FOUNDRY_MODEL e inicie sessão com az login.

import asyncio
import base64
import os

from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient
from azure.identity import AzureCliCredential

IMAGE_MODEL = "gpt-image-1"


async def main() -> None:
    agent = Agent(
        client=FoundryChatClient(credential=AzureCliCredential()),
        instructions="Generate images based on user prompts.",
        tools=[
            FoundryChatClient.get_image_generation_tool(
                model=IMAGE_MODEL,
                quality="low",
                size="1024x1024",
            )
        ],
    )

    result = await agent.run("Generate an image of the Microsoft logo.")

    # Extract and save the generated image from the raw response.
    for output in result.raw_representation.output:
        if output.type == "image_generation_call":
            file_path = os.path.abspath("microsoft.png")
            with open(file_path, "wb") as f:
                f.write(base64.b64decode(output.result))
            print(f"Image saved to: {file_path}")

    print(f"Agent: {result.text}")


if __name__ == "__main__":
    asyncio.run(main())

Produção esperada

A ferramenta devolve bytes de imagem codificados em base64, que a amostra guarda no disco; A resposta em texto do modelo também é impressa:

Image saved to: /path/to/microsoft.png
Agent: Here is the generated Microsoft logo image.

Para saber mais sobre as fábricas de ferramentas do Agent Framework Foundry, consulte os exemplos do fornecedor Foundry.

Exemplo para geração de imagens no Azure. IA. Extensões.OpenAI

Neste exemplo, geras uma imagem com base num prompt simples. O código neste exemplo é síncrono. Para um exemplo assíncrono, veja o exemplo de código de amostra no repositório do SDK do Azure para .NET no GitHub.

Usa o SDK .NET 8 ou posterior. Adicione os pacotes necessários ao seu projeto:

dotnet add package Azure.AI.Projects
dotnet add package Azure.AI.Extensions.OpenAI
dotnet add package Azure.Identity

using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using Azure.AI.Projects;
using Azure.AI.Extensions.OpenAI;
using Azure.Core;
using Azure.Core.Pipeline;
using Azure.Identity;

// Format: "https://resource_name.ai.azure.com/api/projects/project_name"
var projectEndpoint = "your_project_endpoint";
var imageModel = "gpt-image-1";

// Create the AI Project client with custom header policy
AIProjectClientOptions projectOptions = new();
projectOptions.AddPolicy(new HeaderPolicy(imageModel), PipelinePosition.PerCall);

// Create the AI Project client
AIProjectClient projectClient = new(
    endpoint: new Uri(projectEndpoint),
    tokenProvider: new DefaultAzureCredential(),
    options: projectOptions
);

// Use the client to create the versioned agent object.
// To generate images, we need to provide agent with the ImageGenerationTool
// when creating this tool. The ImageGenerationTool parameters include
// the image generation model, image quality and resolution.
// Supported image generation models include gpt-image-1.
DeclarativeAgentDefinition agentDefinition = new(model: "gpt-5")
{
Instructions = "Generate images based on user prompts.",
Tools = {
        ResponseTool.CreateImageGenerationTool(
            model: imageModel,
            quality: ImageGenerationToolQuality.Low,
            size:ImageGenerationToolSize.W1024xH1024
        )
    }
};
AgentVersion agentVersion = projectClient.AgentAdministrationClient.CreateAgentVersion(
    agentName: "myAgent",
    options: new(agentDefinition));

ProjectOpenAIClient openAIClient = projectClient.GetProjectOpenAIClient();
ProjectResponsesClient responseClient = openAIClient.GetProjectResponsesClientForAgent(new AgentReference(name: agentVersion.Name));

ResponseResult response = responseClient.CreateResponse("Generate parody of Newton with apple.");

// Parse the ResponseResult object and save the generated image.
foreach (ResponseItem item in response.OutputItems)
{
    if (item is ImageGenerationCallResponseItem imageItem)
    {
        File.WriteAllBytes("newton.png", imageItem.ImageResultBytes.ToArray());
        Console.WriteLine($"Image downloaded and saved to: {Path.GetFullPath("newton.png")}");
    }
}

// Clean up resources by deleting the Agent.
projectClient.AgentAdministrationClient.DeleteAgentVersion(agentName: agentVersion.Name, agentVersion: agentVersion.Version);

// To use image generation, provide the custom header to web requests,
// which contain the model deployment name, for example:
// `x-ms-oai-image-generation-deployment: gpt-image-1`.
// To implement it, create a custom header policy.
internal class HeaderPolicy(string image_deployment) : PipelinePolicy
{
    private const string image_deployment_header = "x-ms-oai-image-generation-deployment";

    public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
    {
        message.Request.Headers.Add(image_deployment_header, image_deployment);
        ProcessNext(message, pipeline, currentIndex);
    }

    public override async ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
    {
        // Add your desired header name and value
        message.Request.Headers.Add(image_deployment_header, image_deployment);
        await ProcessNextAsync(message, pipeline, currentIndex);
    }
}

Produção esperada

Quando executa a amostra, vê o seguinte resultado:

Agent created (id: <agent-id>, name: myAgent, version: 1)
Image downloaded and saved to: /path/to/newton.png
Agent deleted

Crie um agente com a ferramenta de geração de imagens

Use um shell compatível com Bash com CLI do Azure, curl, jq, e um base64 comando que suporte --decode. Define FOUNDRY_PROJECT_ENDPOINT antes de executares os pedidos.

Obtenha um token de acesso:

export AGENT_TOKEN=$(az account get-access-token --scope "https://ai.azure.com/.default" --query accessToken -o tsv)

O exemplo seguinte cria um agente que utiliza a ferramenta de geração de imagens.

curl -X POST "$FOUNDRY_PROJECT_ENDPOINT/agents?api-version=v1" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AGENT_TOKEN" \
  -d '{
    "name": "image-gen-agent",
    "description": "Test agent for image generation capabilities",
    "definition": {
      "kind": "prompt",
      "model": "gpt-5",
      "tools": [
        {
          "type": "image_generation"
        }
      ],
      "instructions": "You are a creative assistant that generates images when requested. Please respond to image generation requests clearly and concisely."
    }
  }'

Crie uma resposta

curl -X POST "$FOUNDRY_PROJECT_ENDPOINT/openai/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AGENT_TOKEN" \
  -H "x-ms-oai-image-generation-deployment: gpt-image-1" \
  -d '{
    "agent_reference": {
      "type": "agent_reference",
      "name": "image-gen-agent"
    },
    "input": [{
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Please generate small image of a sunset over a mountain lake."
        }
      ]
    }],
    "stream": false
  }'

Produção esperada

O JSON de resposta inclui um image_generation_call item de saída com um result campo contendo dados de imagem codificados em base64:

{
  "id": "resp_<id>",
  "status": "completed",
  "output": [
    {
      "type": "image_generation_call",
      "result": "<base64-encoded-image-data>",
      "status": "completed"
    },
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Here is the image of a sunset over a mountain lake."
        }
      ]
    }
  ]
}

Para extrair e guardar a imagem, canalize a resposta através de jq e base64.

RESPONSE=$(curl -s -X POST "$FOUNDRY_PROJECT_ENDPOINT/openai/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AGENT_TOKEN" \
  -H "x-ms-oai-image-generation-deployment: gpt-image-1" \
  -d '{ ... }')

echo "$RESPONSE" | jq -r '.output[] | select(.type=="image_generation_call") | .result' \
  | base64 --decode > generated_image.png

Limpar o agente REST

Eliminar o agente depois de guardar a imagem gerada:

curl --request DELETE \
  --url "$FOUNDRY_PROJECT_ENDPOINT/agents/image-gen-agent?api-version=v1" \
  -H "Authorization: Bearer $AGENT_TOKEN"

Crie um agente com uma ferramenta de geração de imagens

Este exemplo demonstra como criar um agente de IA com capacidades de geração de imagens utilizando o cliente Azure AI Projects. O agente gera imagens com base em prompts de texto e guarda-as em ficheiros. Para um exemplo de JavaScript, veja o código exemplo no repositório SDK do Azure para JavaScript sobre GitHub.

Usa Node.js 22 ou mais tarde. Instale os pacotes necessários:

npm install @azure/ai-projects @azure/identity

import { DefaultAzureCredential } from "@azure/identity";
import { AIProjectClient } from "@azure/ai-projects";
import * as fs from "fs";
import * as path from "path";
import { fileURLToPath } from "url";

// Format: "https://resource_name.ai.azure.com/api/projects/project_name"
const PROJECT_ENDPOINT = "your_project_endpoint";
const IMAGE_MODEL = "gpt-image-1";

export async function main(): Promise<void> {
  // Create clients to call Foundry API
  const project = new AIProjectClient(PROJECT_ENDPOINT, new DefaultAzureCredential());
  const openai = project.getOpenAIClient();

  // Create Agent with image generation tool
  const agent = await project.agents.createVersion("agent-image-generation", {
    kind: "prompt",
    model: "gpt-5",
    instructions: "Generate images based on user prompts",
    tools: [
      {
        type: "image_generation",
        quality: "low",
        size: "1024x1024",
      },
    ],
  });
  console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

  // Generate image using the agent
  const response = await openai.responses.create(
    {
      input: "Generate an image of Microsoft logo.",
    },
    {
      body: { agent_reference: { name: agent.name, type: "agent_reference" } },
      headers: { "x-ms-oai-image-generation-deployment": IMAGE_MODEL },
    },
  );

  // Extract and save the generated image
  const imageData = response.output?.filter((output) => output.type === "image_generation_call");

  if (imageData && imageData.length > 0 && imageData[0].result) {
    const __filename = fileURLToPath(import.meta.url);
    const __dirname = path.dirname(__filename);
    const filename = "microsoft.png";
    const filePath = path.join(__dirname, filename);

    // Decode base64 and save to file
    const imageBuffer = Buffer.from(imageData[0].result, "base64");
    fs.writeFileSync(filePath, imageBuffer);

    console.log(`Image downloaded and saved to: ${path.resolve(filePath)}`);
  } else {
    console.log("No image data found in the response.");
  }

  // Clean up resources
  await project.agents.deleteVersion(agent.name, agent.version);
}

main().catch((err) => {
  console.error("The sample encountered an error:", err);
});

Produção esperada

Quando executa a amostra, vê o seguinte resultado:

Agent created (id: <agent-id>, name: agent-image-generation, version: 1)
Image downloaded and saved to: /path/to/microsoft.png

Usar geração de imagens num agente Java

Usa o JDK 17 ou posterior e o Maven 3.8 ou posterior. Adicione as dependências ao seu pom.xml:

O cliente Java atualmente não expõe o cabeçalho necessário x-ms-oai-image-generation-deployment na criação da resposta. Use Java para criar a definição do agente e utilize o procedimento REST neste artigo para invocar o agente e recuperar a imagem gerada.

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-agents</artifactId>
    <version>2.4.0</version>
</dependency>
  <dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-identity</artifactId>
    <version>1.18.4</version>
  </dependency>

Crie um agente com geração de imagem

import com.azure.ai.agents.AgentsClient;
import com.azure.ai.agents.AgentsClientBuilder;
import com.azure.ai.agents.models.*;
import com.azure.identity.DefaultAzureCredentialBuilder;

import java.util.Collections;

public class ImageGenerationExample {
  public static void main(String[] args) throws Exception {
        // Format: "https://resource_name.ai.azure.com/api/projects/project_name"
        String projectEndpoint = "your_project_endpoint";
        String imageModel = "gpt-image-1";

        AgentsClientBuilder builder = new AgentsClientBuilder()
            .credential(new DefaultAzureCredentialBuilder().build())
            .endpoint(projectEndpoint);

        AgentsClient agentsClient = builder.buildAgentsClient();
        // Create image generation tool with model, quality, and size
        ImageGenTool imageGenTool = new ImageGenTool()
            .setModel(ImageGenToolModel.fromString(imageModel))
            .setQuality(ImageGenToolQuality.LOW)
            .setSize(ImageGenToolSize.fromString("1024x1024"));

        // Create agent with image generation tool
        PromptAgentDefinition agentDefinition = new PromptAgentDefinition("gpt-5")
            .setInstructions("You are a creative assistant that can generate images based on descriptions.")
            .setTools(Collections.singletonList(imageGenTool));

        AgentVersionDetails agent = agentsClient.createAgentVersion("image-gen-agent", agentDefinition);
        System.out.printf("Agent created: %s (version %s)%n", agent.getName(), agent.getVersion());

        // Clean up
        agentsClient.deleteAgentVersion(agent.getName(), agent.getVersion());
    }
}

Produção esperada

Agent created: image-gen-agent (version 1)