打造一個超越對話的 AI 助理——它能呼叫函數來執行動作。 助理決定何時需要某個函式,你執行它,並將結果回饋回來。 所有東西都是本地用 Foundry Local SDK 執行。
在本教學課程中,您將瞭解如何:
- 建立專案並安裝 Foundry 本地 SDK
- 定義助理可以呼叫的工具
- 發送一個觸發工具使用的訊息
- 執行工具並將結果回傳給模型
- 處理完整的工具呼叫迴路
- 清理資源
先決條件
- 一台至少有 8GB 記憶體的 Windows、macOS 或 Linux 電腦。
樣本庫
你可以在 Foundry 範例 GitHub 倉庫找到這篇文章的完整範例程式碼。 要複製儲存庫並導向範例,請使用:
git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/csharp/foundry-local/tutorial-tool-calling
安裝套件
dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI
GitHub 倉庫裡的 C# 範例都是預先設定好的專案。 如果你是從零開始建立,建議閱讀 Foundry Local SDK 參考資料 ,了解更多如何用 Foundry Local 設定 C# 專案的細節。
定義工具
工具呼叫讓模型請求你的程式碼執行函式並回傳結果。 你可以將可用工具定義為一份 JSON 架構清單,描述每個函式的名稱、用途和參數。
打開
Program.cs並新增以下工具定義:// --- Tool definitions and implementations --- var toolDefinitions = new[] { new { Name = "get_weather", Description = "Get the current weather for a location", Schema = """ { "type": "object", "properties": { "location": { "type": "string", "description": "The city or location" }, "unit": { "type": "string", "description": "Temperature unit", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } """ }, new { Name = "calculate", Description = "Perform a math calculation", Schema = """ { "type": "object", "properties": { "expression": { "type": "string", "description": "The math expression to evaluate" } }, "required": ["expression"] } """ } }; string ExecuteTool(string functionName, JsonElement arguments) { switch (functionName) { case "get_weather": var location = arguments.GetProperty("location").GetString() ?? "unknown"; var unit = arguments.TryGetProperty("unit", out var u) ? u.GetString() ?? "celsius" : "celsius"; var temp = unit == "celsius" ? 22 : 72; return JsonSerializer.Serialize(new { location, temperature = temp, unit, condition = "Sunny" }); case "calculate": var expression = arguments.GetProperty("expression").GetString() ?? ""; try { var result = new System.Data.DataTable().Compute(expression, null); return JsonSerializer.Serialize(new { expression, result = result?.ToString() }); } catch (Exception ex) { return JsonSerializer.Serialize(new { error = ex.Message }); } default: return JsonSerializer.Serialize(new { error = $"Unknown function: {functionName}" }); } }每個工具定義包含一個
name、 ,description幫助模型決定何時使用,以及parameters描述預期輸入的結構圖。新增實作每個工具的 C# 方法:
// --- Tool definitions and implementations --- var toolDefinitions = new[] { new { Name = "get_weather", Description = "Get the current weather for a location", Schema = """ { "type": "object", "properties": { "location": { "type": "string", "description": "The city or location" }, "unit": { "type": "string", "description": "Temperature unit", "enum": ["celsius", "fahrenheit"] } }, "required": ["location"] } """ }, new { Name = "calculate", Description = "Perform a math calculation", Schema = """ { "type": "object", "properties": { "expression": { "type": "string", "description": "The math expression to evaluate" } }, "required": ["expression"] } """ } }; string ExecuteTool(string functionName, JsonElement arguments) { switch (functionName) { case "get_weather": var location = arguments.GetProperty("location").GetString() ?? "unknown"; var unit = arguments.TryGetProperty("unit", out var u) ? u.GetString() ?? "celsius" : "celsius"; var temp = unit == "celsius" ? 22 : 72; return JsonSerializer.Serialize(new { location, temperature = temp, unit, condition = "Sunny" }); case "calculate": var expression = arguments.GetProperty("expression").GetString() ?? ""; try { var result = new System.Data.DataTable().Compute(expression, null); return JsonSerializer.Serialize(new { expression, result = result?.ToString() }); } catch (Exception ex) { return JsonSerializer.Serialize(new { error = ex.Message }); } default: return JsonSerializer.Serialize(new { error = $"Unknown function: {functionName}" }); } }模型不會直接執行這些函式。 它會回傳一個包含函式名稱和參數的工具呼叫請求,然後你的程式碼執行該函式。
發送一個觸發工具使用的訊息
初始化 Foundry Local SDK,載入模型,並發送訊息,讓模型透過呼叫工具回應。
// --- Main application ---
var config = new Configuration
{
AppName = "foundry_local_samples",
LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};
using var loggerFactory = LoggerFactory.Create(builder =>
{
builder.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Information);
});
var logger = loggerFactory.CreateLogger<Program>();
await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
if (epName != currentEp)
{
if (currentEp != "") Console.WriteLine();
currentEp = epName;
}
Console.Write($"\r {epName.PadRight(30)} {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();
var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
?? throw new Exception("Model not found");
await model.DownloadAsync(progress =>
{
Console.Write($"\rDownloading model: {progress:F2}%");
if (progress >= 100f) Console.WriteLine();
});
await model.LoadAsync();
Console.WriteLine("Model loaded and ready.");
using var chatSession = new ChatSession(model);
chatSession.SetOptions(new RequestOptions { ToolChoice = ToolChoice.Auto });
foreach (var tool in toolDefinitions)
{
chatSession.AddToolDefinition(tool.Name, tool.Description, tool.Schema);
}
當模型判斷需要工具時,回應中會包含 ToolCalls,而不是一般文字訊息。 下一步將說明如何偵測並處理這些來電。
執行工具並回傳結果
模型回應工具呼叫後,擷取函式名稱和參數,執行函式,然後回傳結果。
Console.WriteLine("\nTool-calling assistant ready! Type 'quit' to exit.\n");
while (true)
{
Console.Write("You: ");
var userInput = Console.ReadLine();
if (string.IsNullOrWhiteSpace(userInput) ||
userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
{
break;
}
using var request = new Request();
if (chatSession.TurnCount == 0)
{
request.AddItem(MessageItem.System(
"You are a helpful assistant with access to tools. Use them when needed."));
}
request.AddItem(MessageItem.User(userInput));
using var response = await chatSession.ProcessRequestAsync(request, ct);
var toolCalls = response.OfType<ToolCallItem>()
.Select(call => (call.CallId, call.Name, call.Arguments))
.ToList();
if (toolCalls.Count > 0)
{
using var toolRequest = new Request();
foreach (var toolCall in toolCalls)
{
using var argumentsDocument = JsonDocument.Parse(toolCall.Arguments);
var arguments = argumentsDocument.RootElement;
Console.WriteLine($" Tool call: {toolCall.Name}({arguments})");
var result = ExecuteTool(toolCall.Name, arguments);
toolRequest.AddItem(new ToolResultItem(toolCall.CallId, result));
}
using var finalResponse = await chatSession.ProcessRequestAsync(toolRequest, ct);
var answer = GetResponseText(finalResponse);
Console.WriteLine($"Assistant: {answer}\n");
}
else
{
Console.WriteLine($"Assistant: {GetResponseText(response)}\n");
}
}
chatSession.Dispose();
await model.UnloadAsync();
Console.WriteLine("Model unloaded. Goodbye!");
工具呼叫迴圈中的關鍵步驟包括:
-
偵測工具呼叫 — 檢查
response.Choices[0].Message.ToolCalls。 - 執行函式 ——解析參數並呼叫本地函式。
-
回傳結果 — 新增一個訊息,包含角色
tool,以及匹配的ToolCallId。 - 取得最終答案 ——模型會利用工具結果產生自然反應。
處理完整的工具呼叫迴路
這是完整的應用程式,將工具定義、SDK 初始化和工具呼叫迴圈合併成一個可執行的檔案。
將 Program.cs 的內容替換為以下完整程式碼:
using System.Text.Json;
using Microsoft.AI.Foundry.Local;
using Microsoft.Extensions.Logging;
CancellationToken ct = CancellationToken.None;
// --- Tool definitions and implementations ---
var toolDefinitions = new[]
{
new
{
Name = "get_weather",
Description = "Get the current weather for a location",
Schema =
"""
{
"type": "object",
"properties": {
"location": { "type": "string", "description": "The city or location" },
"unit": {
"type": "string",
"description": "Temperature unit",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
"""
},
new
{
Name = "calculate",
Description = "Perform a math calculation",
Schema =
"""
{
"type": "object",
"properties": {
"expression": { "type": "string", "description": "The math expression to evaluate" }
},
"required": ["expression"]
}
"""
}
};
string ExecuteTool(string functionName, JsonElement arguments)
{
switch (functionName)
{
case "get_weather":
var location = arguments.GetProperty("location").GetString() ?? "unknown";
var unit = arguments.TryGetProperty("unit", out var u)
? u.GetString() ?? "celsius"
: "celsius";
var temp = unit == "celsius" ? 22 : 72;
return JsonSerializer.Serialize(new
{
location,
temperature = temp,
unit,
condition = "Sunny"
});
case "calculate":
var expression = arguments.GetProperty("expression").GetString() ?? "";
try
{
var result = new System.Data.DataTable().Compute(expression, null);
return JsonSerializer.Serialize(new { expression, result = result?.ToString() });
}
catch (Exception ex)
{
return JsonSerializer.Serialize(new { error = ex.Message });
}
default:
return JsonSerializer.Serialize(new { error = $"Unknown function: {functionName}" });
}
}
// --- Main application ---
var config = new Configuration
{
AppName = "foundry_local_samples",
LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};
using var loggerFactory = LoggerFactory.Create(builder =>
{
builder.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Information);
});
var logger = loggerFactory.CreateLogger<Program>();
await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
if (epName != currentEp)
{
if (currentEp != "") Console.WriteLine();
currentEp = epName;
}
Console.Write($"\r {epName.PadRight(30)} {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();
var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
?? throw new Exception("Model not found");
await model.DownloadAsync(progress =>
{
Console.Write($"\rDownloading model: {progress:F2}%");
if (progress >= 100f) Console.WriteLine();
});
await model.LoadAsync();
Console.WriteLine("Model loaded and ready.");
using var chatSession = new ChatSession(model);
chatSession.SetOptions(new RequestOptions { ToolChoice = ToolChoice.Auto });
foreach (var tool in toolDefinitions)
{
chatSession.AddToolDefinition(tool.Name, tool.Description, tool.Schema);
}
Console.WriteLine("\nTool-calling assistant ready! Type 'quit' to exit.\n");
while (true)
{
Console.Write("You: ");
var userInput = Console.ReadLine();
if (string.IsNullOrWhiteSpace(userInput) ||
userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
{
break;
}
using var request = new Request();
if (chatSession.TurnCount == 0)
{
request.AddItem(MessageItem.System(
"You are a helpful assistant with access to tools. Use them when needed."));
}
request.AddItem(MessageItem.User(userInput));
using var response = await chatSession.ProcessRequestAsync(request, ct);
var toolCalls = response.OfType<ToolCallItem>()
.Select(call => (call.CallId, call.Name, call.Arguments))
.ToList();
if (toolCalls.Count > 0)
{
using var toolRequest = new Request();
foreach (var toolCall in toolCalls)
{
using var argumentsDocument = JsonDocument.Parse(toolCall.Arguments);
var arguments = argumentsDocument.RootElement;
Console.WriteLine($" Tool call: {toolCall.Name}({arguments})");
var result = ExecuteTool(toolCall.Name, arguments);
toolRequest.AddItem(new ToolResultItem(toolCall.CallId, result));
}
using var finalResponse = await chatSession.ProcessRequestAsync(toolRequest, ct);
var answer = GetResponseText(finalResponse);
Console.WriteLine($"Assistant: {answer}\n");
}
else
{
Console.WriteLine($"Assistant: {GetResponseText(response)}\n");
}
}
chatSession.Dispose();
await model.UnloadAsync();
Console.WriteLine("Model unloaded. Goodbye!");
執行工具呼叫助理:
dotnet run
你會看到類似的輸出:
Downloading model: 100.00%
Model loaded and ready.
Tool-calling assistant ready! Type 'quit' to exit.
You: What's the weather like today?
Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.
You: What is 245 * 38?
Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.
You: quit
Model unloaded. Goodbye!
模型會根據使用者的訊息決定何時呼叫工具。 天氣問題會呼叫 get_weather,數學問題會呼叫 calculate,一般問題則直接回應,無需使用工具。
樣本庫
本文的完整範例程式碼可在 Foundry-samples GitHub 倉庫取得。 要複製儲存庫並導向範例,請使用:
git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/javascript/foundry-local/tutorial-tool-calling
安裝套件
npm install foundry-local-sdk openai
定義工具
工具呼叫讓模型請求你的程式碼執行函式並回傳結果。 你可以將可用工具定義為一份 JSON 架構清單,描述每個函式的名稱、用途和參數。
建立一個名為
index.js. 的檔案。補充以下工具定義:
// --- Tool definitions --- const tools = [ { name: 'get_weather', description: 'Get the current weather for a location', jsonSchema: JSON.stringify({ type: 'object', properties: { location: { type: 'string', description: 'The city or location' }, unit: { type: 'string', enum: ['celsius', 'fahrenheit'], description: 'Temperature unit' } }, required: ['location'] }) }, { name: 'calculate', description: 'Perform a math calculation', jsonSchema: JSON.stringify({ type: 'object', properties: { expression: { type: 'string', description: 'The math expression to evaluate' } }, required: ['expression'] }) } ]; // --- Tool implementations --- function getWeather(location, unit = 'celsius') { return { location, temperature: unit === 'celsius' ? 22 : 72, unit, condition: 'Sunny' }; } function calculate(expression) { // Input is validated against a strict allowlist of numeric/math characters, // making this safe from code injection in this tutorial context. const allowed = /^[0-9+\-*/(). ]+$/; if (!allowed.test(expression)) { return { error: 'Invalid expression' }; } try { const result = Function( `"use strict"; return (${expression})` )(); return { expression, result }; } catch (err) { return { error: err.message }; } } const toolFunctions = { get_weather: (args) => getWeather(args.location, args.unit), calculate: (args) => calculate(args.expression) };每個工具定義包含一個
name、 ,description幫助模型決定何時使用,以及parameters描述預期輸入的結構圖。新增實作每個工具的 JavaScript 函式:
// --- Tool definitions --- const tools = [ { name: 'get_weather', description: 'Get the current weather for a location', jsonSchema: JSON.stringify({ type: 'object', properties: { location: { type: 'string', description: 'The city or location' }, unit: { type: 'string', enum: ['celsius', 'fahrenheit'], description: 'Temperature unit' } }, required: ['location'] }) }, { name: 'calculate', description: 'Perform a math calculation', jsonSchema: JSON.stringify({ type: 'object', properties: { expression: { type: 'string', description: 'The math expression to evaluate' } }, required: ['expression'] }) } ]; // --- Tool implementations --- function getWeather(location, unit = 'celsius') { return { location, temperature: unit === 'celsius' ? 22 : 72, unit, condition: 'Sunny' }; } function calculate(expression) { // Input is validated against a strict allowlist of numeric/math characters, // making this safe from code injection in this tutorial context. const allowed = /^[0-9+\-*/(). ]+$/; if (!allowed.test(expression)) { return { error: 'Invalid expression' }; } try { const result = Function( `"use strict"; return (${expression})` )(); return { expression, result }; } catch (err) { return { error: err.message }; } } const toolFunctions = { get_weather: (args) => getWeather(args.location, args.unit), calculate: (args) => calculate(args.expression) };模型不會直接執行這些函式。 它會回傳一個包含函式名稱和參數的工具呼叫請求,然後你的程式碼執行該函式。
發送一個觸發工具使用的訊息
初始化 Foundry Local SDK,載入模型,並發送訊息,讓模型透過呼叫工具回應。
// --- Main application ---
const manager = FoundryLocalManager.create({
appName: 'foundry_local_samples',
logLevel: 'info'
});
// Download and register all execution providers.
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
if (epName !== currentEp) {
if (currentEp !== '') process.stdout.write('\n');
currentEp = epName;
}
process.stdout.write(`\r ${epName.padEnd(30)} ${percent.toFixed(1).padStart(5)}%`);
});
if (currentEp !== '') process.stdout.write('\n');
const model = await manager.catalog.getModel('qwen2.5-0.5b');
await model.download((progress) => {
process.stdout.write(
`\rDownloading model: ${progress.toFixed(2)}%`
);
});
console.log('\nModel downloaded.');
await model.load();
console.log('Model loaded and ready.');
const chatSession = new ChatSession(model);
for (const tool of tools) {
chatSession.addToolDefinition(tool);
}
let firstTurn = true;
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
const askQuestion = (prompt) =>
new Promise((resolve) => {
if (rl.closed) return resolve('quit');
const onClose = () => resolve('quit');
rl.once('close', onClose);
rl.question(prompt, (answer) => {
rl.off('close', onClose);
resolve(answer);
});
});
console.log(
'\nTool-calling assistant ready! Type \'quit\' to exit.\n'
);
while (true) {
const userInput = await askQuestion('You: ');
if (
userInput.trim().toLowerCase() === 'quit' ||
userInput.trim().toLowerCase() === 'exit'
) {
break;
}
const request = new Request().setOptions({ toolChoice: 'auto' });
if (firstTurn) {
request.addItem(Item.systemMessage(
'You are a helpful assistant with access to tools. ' +
'Use them when needed to answer questions accurately.'
));
firstTurn = false;
}
request.addItem(Item.userMessage(userInput));
const response = await chatSession.processRequest(request);
const answer = await processToolCalls(
response, chatSession
);
console.log(`Assistant: ${answer}\n`);
}
chatSession.dispose();
await model.unload();
console.log('Model unloaded. Goodbye!');
rl.close();
當模型判斷需要工具時,回應中會包含 tool_calls,而不是一般文字訊息。 下一步將說明如何偵測並處理這些來電。
執行工具並回傳結果
模型回應工具呼叫後,擷取函式名稱和參數,執行函式,然後回傳結果。
async function processToolCalls(response, chatSession) {
let currentResponse = response;
while (currentResponse.finishReason === 'toolCalls') {
const toolResults = [];
for (const toolCall of currentResponse.output.filter(
(item) => item.type === 'toolCall'
)) {
const functionName = toolCall.name;
const args = JSON.parse(toolCall.arguments);
console.log(
` Tool call: ${functionName}` +
`(${JSON.stringify(args)})`
);
const implementation = toolFunctions[functionName];
const result = implementation
? implementation(args)
: { error: `Unknown tool: ${functionName}` };
toolResults.push(Item.toolResult(
toolCall.callId,
JSON.stringify(result)
));
}
if (toolResults.length === 0) {
throw new Error('The model requested tools but returned no tool calls.');
}
const followUp = new Request()
.setOptions({ toolChoice: 'auto' });
for (const result of toolResults) {
followUp.addItem(result);
}
currentResponse = await chatSession.processRequest(followUp);
}
return currentResponse.output
.map((item) => item.type === 'message'
? item.content ?? item.parts?.map((part) => part.text ?? '').join('') ?? ''
: item.type === 'text' ? item.text : '')
.join('');
}
工具呼叫迴圈中的關鍵步驟包括:
-
偵測工具呼叫 — 檢查
response.choices[0]?.message?.tool_calls。 - 執行函式 ——解析參數並呼叫本地函式。
-
回傳結果 — 新增一個訊息,包含角色
tool,以及匹配的tool_call_id。 - 取得最終答案 ——模型會利用工具結果產生自然反應。
處理完整的工具呼叫迴路
這是完整的應用程式,將工具定義、SDK 初始化和工具呼叫迴圈合併成一個可執行的檔案。
建立一個名為 index.js S 的檔案,並加入以下完整程式碼:
import {
ChatSession,
FoundryLocalManager,
Item,
Request
} from 'foundry-local-sdk';
import * as readline from 'readline';
// --- Tool definitions ---
const tools = [
{
name: 'get_weather',
description: 'Get the current weather for a location',
jsonSchema: JSON.stringify({
type: 'object',
properties: {
location: {
type: 'string',
description: 'The city or location'
},
unit: {
type: 'string',
enum: ['celsius', 'fahrenheit'],
description: 'Temperature unit'
}
},
required: ['location']
})
},
{
name: 'calculate',
description: 'Perform a math calculation',
jsonSchema: JSON.stringify({
type: 'object',
properties: {
expression: {
type: 'string',
description: 'The math expression to evaluate'
}
},
required: ['expression']
})
}
];
// --- Tool implementations ---
function getWeather(location, unit = 'celsius') {
return {
location,
temperature: unit === 'celsius' ? 22 : 72,
unit,
condition: 'Sunny'
};
}
function calculate(expression) {
// Input is validated against a strict allowlist of numeric/math characters,
// making this safe from code injection in this tutorial context.
const allowed = /^[0-9+\-*/(). ]+$/;
if (!allowed.test(expression)) {
return { error: 'Invalid expression' };
}
try {
const result = Function(
`"use strict"; return (${expression})`
)();
return { expression, result };
} catch (err) {
return { error: err.message };
}
}
const toolFunctions = {
get_weather: (args) => getWeather(args.location, args.unit),
calculate: (args) => calculate(args.expression)
};
async function processToolCalls(response, chatSession) {
let currentResponse = response;
while (currentResponse.finishReason === 'toolCalls') {
const toolResults = [];
for (const toolCall of currentResponse.output.filter(
(item) => item.type === 'toolCall'
)) {
const functionName = toolCall.name;
const args = JSON.parse(toolCall.arguments);
console.log(
` Tool call: ${functionName}` +
`(${JSON.stringify(args)})`
);
const implementation = toolFunctions[functionName];
const result = implementation
? implementation(args)
: { error: `Unknown tool: ${functionName}` };
toolResults.push(Item.toolResult(
toolCall.callId,
JSON.stringify(result)
));
}
if (toolResults.length === 0) {
throw new Error('The model requested tools but returned no tool calls.');
}
const followUp = new Request()
.setOptions({ toolChoice: 'auto' });
for (const result of toolResults) {
followUp.addItem(result);
}
currentResponse = await chatSession.processRequest(followUp);
}
return currentResponse.output
.map((item) => item.type === 'message'
? item.content ?? item.parts?.map((part) => part.text ?? '').join('') ?? ''
: item.type === 'text' ? item.text : '')
.join('');
}
// --- Main application ---
const manager = FoundryLocalManager.create({
appName: 'foundry_local_samples',
logLevel: 'info'
});
// Download and register all execution providers.
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
if (epName !== currentEp) {
if (currentEp !== '') process.stdout.write('\n');
currentEp = epName;
}
process.stdout.write(`\r ${epName.padEnd(30)} ${percent.toFixed(1).padStart(5)}%`);
});
if (currentEp !== '') process.stdout.write('\n');
const model = await manager.catalog.getModel('qwen2.5-0.5b');
await model.download((progress) => {
process.stdout.write(
`\rDownloading model: ${progress.toFixed(2)}%`
);
});
console.log('\nModel downloaded.');
await model.load();
console.log('Model loaded and ready.');
const chatSession = new ChatSession(model);
for (const tool of tools) {
chatSession.addToolDefinition(tool);
}
let firstTurn = true;
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
const askQuestion = (prompt) =>
new Promise((resolve) => {
if (rl.closed) return resolve('quit');
const onClose = () => resolve('quit');
rl.once('close', onClose);
rl.question(prompt, (answer) => {
rl.off('close', onClose);
resolve(answer);
});
});
console.log(
'\nTool-calling assistant ready! Type \'quit\' to exit.\n'
);
while (true) {
const userInput = await askQuestion('You: ');
if (
userInput.trim().toLowerCase() === 'quit' ||
userInput.trim().toLowerCase() === 'exit'
) {
break;
}
const request = new Request().setOptions({ toolChoice: 'auto' });
if (firstTurn) {
request.addItem(Item.systemMessage(
'You are a helpful assistant with access to tools. ' +
'Use them when needed to answer questions accurately.'
));
firstTurn = false;
}
request.addItem(Item.userMessage(userInput));
const response = await chatSession.processRequest(request);
const answer = await processToolCalls(
response, chatSession
);
console.log(`Assistant: ${answer}\n`);
}
chatSession.dispose();
await model.unload();
console.log('Model unloaded. Goodbye!');
rl.close();
執行工具呼叫助理:
node index.js
你會看到類似的輸出:
Downloading model: 100.00%
Model downloaded.
Model loaded and ready.
Tool-calling assistant ready! Type 'quit' to exit.
You: What's the weather like today?
Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.
You: What is 245 * 38?
Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.
You: quit
Model unloaded. Goodbye!
模型會根據使用者的訊息決定何時呼叫工具。 天氣問題會呼叫 get_weather,數學問題會呼叫 calculate,一般問題則直接回應,無需使用工具。
樣本庫
本文的完整範例程式碼可在 Foundry-samples GitHub 倉庫取得。 要複製儲存庫並導向範例,請使用:
git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/python/foundry-local/tutorial-tool-calling
安裝套件
pip install foundry-local-sdk openai
定義工具
工具呼叫讓模型請求你的程式碼執行函式並回傳結果。 你可以將可用工具定義為一份 JSON 架構清單,描述每個函式的名稱、用途和參數。
建立一個名為 main.py Tool 的檔案,並加入以下工具定義:
# --- Tool definitions ---
tools = [
{
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city or location",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit",
},
},
"required": ["location"],
},
},
{
"name": "calculate",
"description": "Perform a math calculation",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "The math expression to evaluate",
}
},
"required": ["expression"],
},
},
]
# --- Tool implementations ---
def get_weather(location, unit="celsius"):
"""Simulate a weather lookup."""
return {
"location": location,
"temperature": 22 if unit == "celsius" else 72,
"unit": unit,
"condition": "Sunny",
}
def calculate(expression):
"""Evaluate a math expression safely."""
allowed = set("0123456789+-*/(). ")
if not all(character in allowed for character in expression):
return {"error": "Invalid expression"}
try:
result = eval(expression, {"__builtins__": {}}, {})
return {"expression": expression, "result": result}
except Exception as error:
return {"error": str(error)}
tool_functions = {"get_weather": get_weather, "calculate": calculate}
每個工具定義包含一個 name、 , description 幫助模型決定何時使用,以及 parameters 描述預期輸入的結構圖。
接著,加入實作每個工具的 Python 函式:
# --- Tool definitions ---
tools = [
{
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city or location",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit",
},
},
"required": ["location"],
},
},
{
"name": "calculate",
"description": "Perform a math calculation",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "The math expression to evaluate",
}
},
"required": ["expression"],
},
},
]
# --- Tool implementations ---
def get_weather(location, unit="celsius"):
"""Simulate a weather lookup."""
return {
"location": location,
"temperature": 22 if unit == "celsius" else 72,
"unit": unit,
"condition": "Sunny",
}
def calculate(expression):
"""Evaluate a math expression safely."""
allowed = set("0123456789+-*/(). ")
if not all(character in allowed for character in expression):
return {"error": "Invalid expression"}
try:
result = eval(expression, {"__builtins__": {}}, {})
return {"expression": expression, "result": result}
except Exception as error:
return {"error": str(error)}
tool_functions = {"get_weather": get_weather, "calculate": calculate}
模型不會直接執行這些函式。 它會回傳一個包含函式名稱和參數的工具呼叫請求,然後你的程式碼執行該函式。
發送一個觸發工具使用的訊息
初始化 Foundry Local SDK,載入模型,並發送訊息,讓模型透過呼叫工具回應。
def main():
# Initialize the Foundry Local SDK
config = Configuration(app_name="foundry_local_samples")
FoundryLocalManager.initialize(config)
manager = FoundryLocalManager.instance
# Discover and register execution providers that are not already available.
missing_eps = [ep.name for ep in manager.discover_eps() if not ep.is_registered]
if missing_eps:
manager.download_and_register_eps(missing_eps)
# Select and load a tool-capable model.
model = manager.catalog.get_model("qwen2.5-0.5b")
if model is None:
raise RuntimeError("Model 'qwen2.5-0.5b' was not found in the catalog.")
if not model.is_cached:
model.download(
lambda progress: print(
f"\rDownloading model: {progress:.2f}%", end="", flush=True
)
)
print()
model.load()
print("Model loaded and ready.")
try:
with ChatSession(model) as session:
session.set_options(
RequestOptions(
search=SearchOptions(temperature=0.0, max_output_tokens=512)
)
)
for tool in tools:
session.add_tool_definition(
tool["name"],
tool["description"],
json.dumps(tool["parameters"]),
)
system_prompt = (
"You are a helpful assistant with access to tools. "
"Use them when needed to answer questions accurately."
)
first_turn = True
print("\nTool-calling assistant ready! Type 'quit' to exit.\n")
while True:
user_input = input("You: ")
if user_input.strip().lower() in ("quit", "exit"):
break
with Request() as request:
if first_turn:
request.add_item(MessageItem.system(system_prompt))
first_turn = False
request.add_item(MessageItem.user(user_input))
with session.process_request(request) as response:
answer, pending_calls = response_content(response)
answer = process_tool_calls(session, answer, pending_calls)
print(f"Assistant: {answer}\n")
finally:
# Clean up
if model.is_loaded:
model.unload()
manager.close()
print("Model unloaded. Goodbye!")
當模型判斷需要工具時,回應中會包含 tool_calls,而不是一般文字訊息。 下一步將說明如何偵測並處理這些來電。
執行工具並回傳結果
模型回應工具呼叫後,擷取函式名稱和參數,執行函式,然後回傳結果。
def process_tool_calls(session, answer, pending_calls):
"""Handle tool calls until the stateful session produces a final answer."""
while pending_calls:
with Request() as follow_up:
for call_id, function_name, arguments_json in pending_calls:
arguments = json.loads(arguments_json)
print(f" Tool call: {function_name}({arguments})")
result = tool_functions[function_name](**arguments)
follow_up.add_item(
ToolResultItem(call_id=call_id, result=json.dumps(result))
)
with session.process_request(follow_up) as next_response:
answer, pending_calls = response_content(next_response)
return answer
工具呼叫迴圈中的關鍵步驟包括:
-
偵測工具呼叫 — 檢查
response.choices[0].message.tool_calls。 - 執行函式 ——解析參數並呼叫本地函式。
-
回傳結果 — 新增一個訊息,包含角色
tool,以及匹配的tool_call_id。 - 取得最終答案 ——模型會利用工具結果產生自然反應。
處理完整的工具呼叫迴路
這是完整的應用程式,將工具定義、SDK 初始化和工具呼叫迴圈合併成一個可執行的檔案。
建立一個名為 main.py S 的檔案,並加入以下完整程式碼:
import json
from foundry_local_sdk import (
ChatSession,
Configuration,
FoundryLocalManager,
MessageItem,
Request,
RequestOptions,
SearchOptions,
TextItem,
ToolCallItem,
ToolResultItem,
)
# --- Tool definitions ---
tools = [
{
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city or location",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit",
},
},
"required": ["location"],
},
},
{
"name": "calculate",
"description": "Perform a math calculation",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "The math expression to evaluate",
}
},
"required": ["expression"],
},
},
]
# --- Tool implementations ---
def get_weather(location, unit="celsius"):
"""Simulate a weather lookup."""
return {
"location": location,
"temperature": 22 if unit == "celsius" else 72,
"unit": unit,
"condition": "Sunny",
}
def calculate(expression):
"""Evaluate a math expression safely."""
allowed = set("0123456789+-*/(). ")
if not all(character in allowed for character in expression):
return {"error": "Invalid expression"}
try:
result = eval(expression, {"__builtins__": {}}, {})
return {"expression": expression, "result": result}
except Exception as error:
return {"error": str(error)}
tool_functions = {"get_weather": get_weather, "calculate": calculate}
def response_content(response):
text_parts = []
tool_calls = []
for item in response:
parts = item.parts if isinstance(item, MessageItem) else [item]
for part in parts:
if isinstance(part, TextItem):
text_parts.append(part.text)
elif isinstance(part, ToolCallItem):
tool_calls.append((part.call_id, part.name, part.arguments))
return "".join(text_parts), tool_calls
def process_tool_calls(session, answer, pending_calls):
"""Handle tool calls until the stateful session produces a final answer."""
while pending_calls:
with Request() as follow_up:
for call_id, function_name, arguments_json in pending_calls:
arguments = json.loads(arguments_json)
print(f" Tool call: {function_name}({arguments})")
result = tool_functions[function_name](**arguments)
follow_up.add_item(
ToolResultItem(call_id=call_id, result=json.dumps(result))
)
with session.process_request(follow_up) as next_response:
answer, pending_calls = response_content(next_response)
return answer
def main():
# Initialize the Foundry Local SDK
config = Configuration(app_name="foundry_local_samples")
FoundryLocalManager.initialize(config)
manager = FoundryLocalManager.instance
# Discover and register execution providers that are not already available.
missing_eps = [ep.name for ep in manager.discover_eps() if not ep.is_registered]
if missing_eps:
manager.download_and_register_eps(missing_eps)
# Select and load a tool-capable model.
model = manager.catalog.get_model("qwen2.5-0.5b")
if model is None:
raise RuntimeError("Model 'qwen2.5-0.5b' was not found in the catalog.")
if not model.is_cached:
model.download(
lambda progress: print(
f"\rDownloading model: {progress:.2f}%", end="", flush=True
)
)
print()
model.load()
print("Model loaded and ready.")
try:
with ChatSession(model) as session:
session.set_options(
RequestOptions(
search=SearchOptions(temperature=0.0, max_output_tokens=512)
)
)
for tool in tools:
session.add_tool_definition(
tool["name"],
tool["description"],
json.dumps(tool["parameters"]),
)
system_prompt = (
"You are a helpful assistant with access to tools. "
"Use them when needed to answer questions accurately."
)
first_turn = True
print("\nTool-calling assistant ready! Type 'quit' to exit.\n")
while True:
user_input = input("You: ")
if user_input.strip().lower() in ("quit", "exit"):
break
with Request() as request:
if first_turn:
request.add_item(MessageItem.system(system_prompt))
first_turn = False
request.add_item(MessageItem.user(user_input))
with session.process_request(request) as response:
answer, pending_calls = response_content(response)
answer = process_tool_calls(session, answer, pending_calls)
print(f"Assistant: {answer}\n")
finally:
# Clean up
if model.is_loaded:
model.unload()
manager.close()
print("Model unloaded. Goodbye!")
if __name__ == "__main__":
main()
執行工具呼叫助理:
python main.py
你會看到類似的輸出:
Downloading model: 100.00%
Model loaded and ready.
Tool-calling assistant ready! Type 'quit' to exit.
You: What's the weather like today?
Tool call: get_weather({'location': 'current location'})
Assistant: The weather today is sunny with a temperature of 22°C.
You: What is 245 * 38?
Tool call: calculate({'expression': '245 * 38'})
Assistant: 245 multiplied by 38 equals 9,310.
You: quit
Model unloaded. Goodbye!
模型會根據使用者的訊息決定何時呼叫工具。 天氣問題會呼叫 get_weather,數學問題會呼叫 calculate,一般問題則直接回應,無需使用工具。
樣本庫
本文的完整範例程式碼可在 Foundry-samples GitHub 倉庫取得。 要複製儲存庫並導向範例,請使用:
git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/rust/foundry-local/tutorial-tool-calling
安裝套件
cargo add foundry-local-sdk
cargo add tokio --features full
cargo add tokio-stream anyhow
定義工具
工具呼叫讓模型請求你的程式碼執行函式並回傳結果。 你可以將可用工具定義為一份 JSON 架構清單,描述每個函式的名稱、用途和參數。
請新增
serde_json以處理 JSON 相依性:cargo add serde_json打開
src/main.rs並新增以下工具定義:let weather_schema = json!({ "type": "object", "properties": { "location": {"type": "string", "description": "The city or location"}, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"], "description": "Temperature unit" } }, "required": ["location"] }); let calculator_schema = json!({ "type": "object", "properties": { "expression": { "type": "string", "description": "The math expression to evaluate" } }, "required": ["expression"] });每個工具定義包含一個
name、 ,description幫助模型決定何時使用,以及parameters描述預期輸入的結構圖。新增實作每個工具的 Rust 函式:
let weather_schema = json!({ "type": "object", "properties": { "location": {"type": "string", "description": "The city or location"}, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"], "description": "Temperature unit" } }, "required": ["location"] }); let calculator_schema = json!({ "type": "object", "properties": { "expression": { "type": "string", "description": "The math expression to evaluate" } }, "required": ["expression"] });模型不會直接執行這些函式。 它會回傳一個包含函式名稱和參數的工具呼叫請求,然後你的程式碼執行該函式。
發送一個觸發工具使用的訊息
初始化 Foundry Local SDK,載入模型,並發送訊息,讓模型透過呼叫工具回應。
let manager = FoundryLocalManager::create(FoundryLocalConfig::new("foundry_local_samples"))?;
manager
.download_and_register_eps_with_progress(None, {
let mut current_ep = String::new();
move |ep_name: &str, percent: f64| {
if ep_name != current_ep {
if !current_ep.is_empty() {
println!();
}
current_ep = ep_name.to_string();
}
print!("\r {ep_name:<30} {percent:5.1}%");
io::stdout().flush().ok();
}
})
.await?;
println!();
let model = manager.catalog().get_model("qwen2.5-0.5b").await?;
if !model.is_cached().await? {
println!("Downloading model...");
model
.download(Some(|progress: f64| {
print!("\r {progress:.1}%");
io::stdout().flush().ok();
}))
.await?;
println!();
}
model.load().await?;
let session = ChatSession::new(&model).await?;
session
.set_options(RequestOptions {
search: SearchOptions {
temperature: Some(0.7),
max_output_tokens: Some(512),
..Default::default()
},
tool_choice: Some(ToolChoice::Auto),
..Default::default()
})
.await?;
session
.add_tool_definition(
ToolDefinition::new("get_weather", weather_schema.to_string())
.with_description("Get the current weather for a location."),
)
.await?;
session
.add_tool_definition(
ToolDefinition::new("calculate", calculator_schema.to_string())
.with_description("Perform a math calculation."),
)
.await?;
當模型判斷需要工具時,回應中會包含 tool_calls,而不是一般文字訊息。 下一步將說明如何偵測並處理這些來電。
執行工具並回傳結果
模型回應工具呼叫後,擷取函式名稱和參數,執行函式,然後回傳結果。
println!("\nTool-calling assistant ready! Type 'quit' to exit.\n");
let stdin = io::stdin();
loop {
print!("You: ");
io::stdout().flush()?;
let mut input = String::new();
stdin.lock().read_line(&mut input)?;
let input = input.trim();
if input.eq_ignore_ascii_case("quit") || input.eq_ignore_ascii_case("exit") {
break;
}
if input.is_empty() {
continue;
}
let mut items = Vec::new();
if session.turn_count() == 0 {
items.push(Item::system_message(vec![Item::text(
"You are a helpful assistant with access to tools. Use them when needed.",
)]));
}
items.push(Item::user_message(vec![Item::text(input)]));
loop {
let response = session
.process_request(Request::from_items(items).with_options(RequestOptions {
search: SearchOptions {
temperature: Some(0.7),
max_output_tokens: Some(512),
..Default::default()
},
tool_choice: Some(ToolChoice::Auto),
..Default::default()
}))
.await?;
let mut tool_results = Vec::new();
let mut answer = String::new();
for item in &response.items {
if let Some(call) = item.as_tool_call() {
let arguments: Value =
serde_json::from_str(&call.arguments).unwrap_or_else(|_| json!({}));
println!(" Tool call: {}({arguments})", call.name);
let result = execute_tool(&call.name, &arguments);
tool_results.push(Item::tool_result(call.call_id.clone(), result.to_string()));
} else if let Some(message) = item.as_message() {
answer.push_str(&message.text());
} else if let Some(text) = item.as_text() {
answer.push_str(text);
}
}
if tool_results.is_empty() {
println!("Assistant: {answer}\n");
break;
}
items = tool_results;
}
}
drop(session);
model.unload().await?;
println!("Model unloaded. Goodbye!");
工具呼叫迴圈中的關鍵步驟包括:
-
偵測工具呼叫 — 檢查
response.choices[0].message.tool_calls。 - 執行函式 ——解析參數並呼叫本地函式。
-
回傳結果 — 新增一個包含角色
tool和對應工具呼叫 ID 的訊息。 - 取得最終答案 ——模型會利用工具結果產生自然反應。
處理完整的工具呼叫迴路
這是完整的應用程式,將工具定義、SDK 初始化和工具呼叫迴圈合併成一個可執行的檔案。
將 src/main.rs 的內容替換為以下完整程式碼:
use foundry_local_sdk::{
ChatSession, FoundryLocalConfig, FoundryLocalManager, Item, Request, RequestOptions,
SearchOptions, ToolChoice, ToolDefinition,
};
use serde_json::{json, Value};
use std::io::{self, BufRead, Write};
fn execute_tool(name: &str, arguments: &Value) -> Value {
match name {
"get_weather" => {
let location = arguments["location"].as_str().unwrap_or("unknown");
let unit = arguments["unit"].as_str().unwrap_or("celsius");
let temp = if unit == "celsius" { 22 } else { 72 };
json!({
"location": location,
"temperature": temp,
"unit": unit,
"condition": "Sunny"
})
}
"calculate" => {
let expression = arguments["expression"].as_str().unwrap_or("");
let is_valid = expression
.chars()
.all(|character| "0123456789+-*/(). ".contains(character));
if !is_valid {
return json!({"error": "Invalid expression"});
}
match eval_expression(expression) {
Ok(result) => json!({"expression": expression, "result": result}),
Err(error) => json!({"error": error}),
}
}
_ => json!({"error": format!("Unknown function: {name}")}),
}
}
fn eval_expression(expr: &str) -> Result<f64, String> {
let expr = expr.replace(' ', "");
let chars: Vec<char> = expr.chars().collect();
let mut pos = 0;
let result = parse_add(&chars, &mut pos)?;
if pos < chars.len() {
return Err("Unexpected character".to_string());
}
Ok(result)
}
fn parse_add(chars: &[char], pos: &mut usize) -> Result<f64, String> {
let mut result = parse_mul(chars, pos)?;
while *pos < chars.len() && (chars[*pos] == '+' || chars[*pos] == '-') {
let op = chars[*pos];
*pos += 1;
let right = parse_mul(chars, pos)?;
result = if op == '+' {
result + right
} else {
result - right
};
}
Ok(result)
}
fn parse_mul(chars: &[char], pos: &mut usize) -> Result<f64, String> {
let mut result = parse_atom(chars, pos)?;
while *pos < chars.len() && (chars[*pos] == '*' || chars[*pos] == '/') {
let op = chars[*pos];
*pos += 1;
let right = parse_atom(chars, pos)?;
result = if op == '*' {
result * right
} else {
result / right
};
}
Ok(result)
}
fn parse_atom(chars: &[char], pos: &mut usize) -> Result<f64, String> {
if *pos < chars.len() && chars[*pos] == '(' {
*pos += 1;
let result = parse_add(chars, pos)?;
if *pos < chars.len() && chars[*pos] == ')' {
*pos += 1;
}
return Ok(result);
}
let start = *pos;
while *pos < chars.len() && (chars[*pos].is_ascii_digit() || chars[*pos] == '.') {
*pos += 1;
}
if start == *pos {
return Err("Expected number".to_string());
}
chars[start..*pos]
.iter()
.collect::<String>()
.parse::<f64>()
.map_err(|error| error.to_string())
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let weather_schema = json!({
"type": "object",
"properties": {
"location": {"type": "string", "description": "The city or location"},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit"
}
},
"required": ["location"]
});
let calculator_schema = json!({
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "The math expression to evaluate"
}
},
"required": ["expression"]
});
let manager = FoundryLocalManager::create(FoundryLocalConfig::new("foundry_local_samples"))?;
manager
.download_and_register_eps_with_progress(None, {
let mut current_ep = String::new();
move |ep_name: &str, percent: f64| {
if ep_name != current_ep {
if !current_ep.is_empty() {
println!();
}
current_ep = ep_name.to_string();
}
print!("\r {ep_name:<30} {percent:5.1}%");
io::stdout().flush().ok();
}
})
.await?;
println!();
let model = manager.catalog().get_model("qwen2.5-0.5b").await?;
if !model.is_cached().await? {
println!("Downloading model...");
model
.download(Some(|progress: f64| {
print!("\r {progress:.1}%");
io::stdout().flush().ok();
}))
.await?;
println!();
}
model.load().await?;
let session = ChatSession::new(&model).await?;
session
.set_options(RequestOptions {
search: SearchOptions {
temperature: Some(0.7),
max_output_tokens: Some(512),
..Default::default()
},
tool_choice: Some(ToolChoice::Auto),
..Default::default()
})
.await?;
session
.add_tool_definition(
ToolDefinition::new("get_weather", weather_schema.to_string())
.with_description("Get the current weather for a location."),
)
.await?;
session
.add_tool_definition(
ToolDefinition::new("calculate", calculator_schema.to_string())
.with_description("Perform a math calculation."),
)
.await?;
println!("\nTool-calling assistant ready! Type 'quit' to exit.\n");
let stdin = io::stdin();
loop {
print!("You: ");
io::stdout().flush()?;
let mut input = String::new();
stdin.lock().read_line(&mut input)?;
let input = input.trim();
if input.eq_ignore_ascii_case("quit") || input.eq_ignore_ascii_case("exit") {
break;
}
if input.is_empty() {
continue;
}
let mut items = Vec::new();
if session.turn_count() == 0 {
items.push(Item::system_message(vec![Item::text(
"You are a helpful assistant with access to tools. Use them when needed.",
)]));
}
items.push(Item::user_message(vec![Item::text(input)]));
loop {
let response = session
.process_request(Request::from_items(items).with_options(RequestOptions {
search: SearchOptions {
temperature: Some(0.7),
max_output_tokens: Some(512),
..Default::default()
},
tool_choice: Some(ToolChoice::Auto),
..Default::default()
}))
.await?;
let mut tool_results = Vec::new();
let mut answer = String::new();
for item in &response.items {
if let Some(call) = item.as_tool_call() {
let arguments: Value =
serde_json::from_str(&call.arguments).unwrap_or_else(|_| json!({}));
println!(" Tool call: {}({arguments})", call.name);
let result = execute_tool(&call.name, &arguments);
tool_results.push(Item::tool_result(call.call_id.clone(), result.to_string()));
} else if let Some(message) = item.as_message() {
answer.push_str(&message.text());
} else if let Some(text) = item.as_text() {
answer.push_str(text);
}
}
if tool_results.is_empty() {
println!("Assistant: {answer}\n");
break;
}
items = tool_results;
}
}
drop(session);
model.unload().await?;
println!("Model unloaded. Goodbye!");
Ok(())
}
執行工具呼叫助理:
cargo run
你會看到類似的輸出:
Downloading model: 100.00%
Model loaded and ready.
Tool-calling assistant ready! Type 'quit' to exit.
You: What's the weather like today?
Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.
You: What is 245 * 38?
Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.
You: quit
Model unloaded. Goodbye!
模型會根據使用者的訊息決定何時呼叫工具。 天氣問題會呼叫 get_weather,數學問題會呼叫 calculate,一般問題則直接回應,無需使用工具。
樣本庫
複製 Foundry samples GitHub 存放庫,然後前往 C++ 範例:
git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/cpp/foundry-local/tutorial-tool-calling-assistant
安裝套件
C++ 範例需要:
設定並建置範例:
cmake -S . -B build
cmake --build build --config Release
在設定過程中,樣本會將 Microsoft.AI.Foundry.Local.Runtime NuGet 套件的 2.2.1 版還原至 build/_packages。 該套件包含 Windows、Linux 和 macOS 的 C++ 標頭及原生函式庫。 你不需要另外的 WinML 套件。
CMake 設定會選擇適合作業系統與架構的執行時資產,並將所需的原生函式庫複製到執行檔旁。 在 macOS 和 Linux 上,從 build 執行該可執行檔。 在 Windows 上,從 build 執行 .exe 檔案。
定義工具
定義可用工具及實作它們的本地函式:
constexpr const char* kTimeSchema = R"({
"type": "object",
"properties": {
"timezone": {"type": "string", "description": "Requested timezone label"}
},
"required": ["timezone"]
})";
std::string GetLocalTime() {
const auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
std::tm local_time{};
#ifdef _WIN32
localtime_s(&local_time, &now);
#else
localtime_r(&now, &local_time);
#endif
std::ostringstream output;
output << std::put_time(&local_time, "%Y-%m-%d %H:%M:%S");
return R"({"timezone":"local","time":")" + output.str() + R"("})";
}
發送一個觸發工具使用的訊息
初始化 SDK,載入模型,並發送一個可使用工具的請求:
Configuration config("foundry_local_cpp_tool_assistant");
Manager manager(std::move(config));
auto model = manager.GetCatalog().GetModel("qwen2.5-0.5b");
if (!model) {
throw std::runtime_error("Model 'qwen2.5-0.5b' was not found in the catalog.");
}
if (!model->IsCached()) {
model->Download([](float progress) {
std::cout << "\rDownloading model: " << static_cast<int>(progress) << "%" << std::flush;
return 0;
});
std::cout << "\n";
}
model->Load();
執行工具並回傳結果
偵測工具呼叫,執行本地函式,回傳其結果,並向模型詢問最終答案:
Request request{
SystemMessage("Use the available tool when current time is requested."),
UserMessage("What time is it?"),
};
request.SetOptions({
.search = {.temperature = 0.0F, .max_output_tokens = 256, .seed = 42},
.tool_choice = FOUNDRY_LOCAL_TOOL_CHOICE_REQUIRED,
});
Response response = session.ProcessRequest(request);
for (int iteration = 0; iteration < 8; ++iteration) {
bool called_tool = false;
for (const Item& item : response.GetItems()) {
if (item.GetType() == FOUNDRY_LOCAL_ITEM_MESSAGE) {
std::cout << "Assistant: ";
PrintAssistant(item);
continue;
}
if (item.GetType() != FOUNDRY_LOCAL_ITEM_TOOL_CALL) {
continue;
}
const auto call = item.GetToolCall();
if (call.name != "get_current_time") {
throw std::runtime_error("The model requested an unknown tool.");
}
Request tool_result;
tool_result.AddItem(Item::ToolResult(std::string(call.call_id), GetLocalTime()));
tool_result.SetOptions({
.search = {.temperature = 0.0F, .max_output_tokens = 256, .seed = 42},
.tool_choice = FOUNDRY_LOCAL_TOOL_CHOICE_NONE,
});
response = session.ProcessRequest(tool_result);
called_tool = true;
break;
}
if (!called_tool) {
break;
}
}
完整程式碼
#include <foundry_local/foundry_local_cpp.h>
#include <chrono>
#include <ctime>
#include <iomanip>
#include <iostream>
#include <sstream>
#include <string>
using namespace foundry_local;
constexpr const char* kTimeSchema = R"({
"type": "object",
"properties": {
"timezone": {"type": "string", "description": "Requested timezone label"}
},
"required": ["timezone"]
})";
std::string GetLocalTime() {
const auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
std::tm local_time{};
#ifdef _WIN32
localtime_s(&local_time, &now);
#else
localtime_r(&now, &local_time);
#endif
std::ostringstream output;
output << std::put_time(&local_time, "%Y-%m-%d %H:%M:%S");
return R"({"timezone":"local","time":")" + output.str() + R"("})";
}
void PrintAssistant(const Item& message) {
for (const Item& part : message.GetMessage().parts) {
if (part.GetType() == FOUNDRY_LOCAL_ITEM_TEXT) {
std::cout << part.GetText().text;
}
}
std::cout << "\n";
}
int main() {
try {
Configuration config("foundry_local_cpp_tool_assistant");
Manager manager(std::move(config));
auto model = manager.GetCatalog().GetModel("qwen2.5-0.5b");
if (!model) {
throw std::runtime_error("Model 'qwen2.5-0.5b' was not found in the catalog.");
}
if (!model->IsCached()) {
model->Download([](float progress) {
std::cout << "\rDownloading model: " << static_cast<int>(progress) << "%" << std::flush;
return 0;
});
std::cout << "\n";
}
model->Load();
{
ChatSession session(*model);
session.AddToolDefinition(ToolDefinition{
"get_current_time",
"Get the current local date and time.",
kTimeSchema,
});
Request request{
SystemMessage("Use the available tool when current time is requested."),
UserMessage("What time is it?"),
};
request.SetOptions({
.search = {.temperature = 0.0F, .max_output_tokens = 256, .seed = 42},
.tool_choice = FOUNDRY_LOCAL_TOOL_CHOICE_REQUIRED,
});
Response response = session.ProcessRequest(request);
for (int iteration = 0; iteration < 8; ++iteration) {
bool called_tool = false;
for (const Item& item : response.GetItems()) {
if (item.GetType() == FOUNDRY_LOCAL_ITEM_MESSAGE) {
std::cout << "Assistant: ";
PrintAssistant(item);
continue;
}
if (item.GetType() != FOUNDRY_LOCAL_ITEM_TOOL_CALL) {
continue;
}
const auto call = item.GetToolCall();
if (call.name != "get_current_time") {
throw std::runtime_error("The model requested an unknown tool.");
}
Request tool_result;
tool_result.AddItem(Item::ToolResult(std::string(call.call_id), GetLocalTime()));
tool_result.SetOptions({
.search = {.temperature = 0.0F, .max_output_tokens = 256, .seed = 42},
.tool_choice = FOUNDRY_LOCAL_TOOL_CHOICE_NONE,
});
response = session.ProcessRequest(tool_result);
called_tool = true;
break;
}
if (!called_tool) {
break;
}
}
}
model->Unload();
manager.Shutdown();
} catch (const Error& error) {
std::cerr << "Foundry Local error [" << error.Code() << "]: " << error.what() << "\n";
return 1;
} catch (const std::exception& error) {
std::cerr << "Error: " << error.what() << "\n";
return 1;
}
return 0;
}
執行助理:
./build/tutorial-tool-calling-assistant
清理資源
卸載模型後,模型權重仍保留在本地快取中。 這表示下次執行應用程式時,下載步驟會被跳過,模型載入速度會更快。 除非你想搶回磁碟空間,否則不需要額外清理。