AgentOptimizationEvaluationConfiguration Class

Reusable quality-measurement configuration.

Constructor

AgentOptimizationEvaluationConfiguration(*args: Any, **kwargs: Any)

Variables

Name	Description
training_set	AgentOptimizationEvaluationSet Evaluation set used to guide the optimization search. Inline data supports up to 2,000 test cases when a separate validation set is supplied; otherwise it is also used for validation and is limited to 500. Required.
validation_set	AgentOptimizationEvaluationSet Held-out evaluation set used for full candidate evaluation, limited to 500 inline test cases. The training set is reused and subject to the same 500-test-case validation limit when omitted.
evaluators	list[AgentOptimizationEvaluator] Evaluator references used to score candidate quality. Required.
evaluation_model	EvaluationModelConfiguration Model configuration used by model-based evaluators and conversation simulation. Required.
max_concurrent_agent_runs	int Maximum number of target-agent runs executed concurrently during each single-turn evaluation. If omitted, the service defaults to 1. Conversation evaluation supports only 1.

Methods

as_dict	Return a dict that can be turned into json using json.dump.
clear	Remove all items from the dictionary.
copy
get	Get the value for key if key is in the dictionary, else default. :param str key: The key to look up. :param any default: The value to return if key is not in the dictionary. Defaults to None :returns: The value for key if key is in the dictionary, else default. :rtype: any
items
keys
pop	Removes specified key and return the corresponding value. :param str key: The key to pop. :param any default: The value to return if key is not in the dictionary :returns: The value corresponding to the key. :rtype: any :raises KeyError: If key is not found and default is not given.
popitem	Removes and returns some (key, value) pair :returns: The (key, value) pair. :rtype: tuple :raises KeyError: if the dictionary is empty.
setdefault	Return the value for key if key is in the dictionary; otherwise set the key to default and return default. :param str key: The key to look up. :param any default: The value to set if key is not in the dictionary :returns: The value for key if key is in the dictionary, else default. :rtype: any
update	Update the dictionary from a mapping or an iterable of key-value pairs. :param any args: Either a mapping object or an iterable of key-value pairs.
values

as_dict

Return a dict that can be turned into json using json.dump.

as_dict(*, exclude_readonly: bool = False) -> dict[str, Any]

Keyword-Only Parameters

Name	Description
exclude_readonly	bool Whether to remove the readonly properties. Default value: False

Returns

Type	Description
dict	A dict JSON compatible object

clear

Remove all items from the dictionary.

clear() -> None

copy

copy() -> Model

get

Get the value for key if key is in the dictionary, else default. :param str key: The key to look up. :param any default: The value to return if key is not in the dictionary. Defaults to None :returns: The value for key if key is in the dictionary, else default. :rtype: any

get(key: str, default: Any = None) -> Any

Parameters

Name	Description
key Required
default	Default value: None

items

items() -> ItemsView[str, Any]

Returns

Type	Description
ItemsView	a set-like object providing a view on the mapping's items

keys

keys() -> KeysView[str]

Returns

Type	Description
KeysView	a set-like object providing a view on the mapping's keys

pop

Removes specified key and return the corresponding value. :param str key: The key to pop. :param any default: The value to return if key is not in the dictionary :returns: The value corresponding to the key. :rtype: any :raises KeyError: If key is not found and default is not given.

pop(key: str, default: ~typing.Any = <object object>) -> Any

Parameters

Name	Description
key Required
default

popitem

Removes and returns some (key, value) pair :returns: The (key, value) pair. :rtype: tuple :raises KeyError: if the dictionary is empty.

popitem() -> tuple[str, Any]

setdefault

Return the value for key if key is in the dictionary; otherwise set the key to default and return default. :param str key: The key to look up. :param any default: The value to set if key is not in the dictionary :returns: The value for key if key is in the dictionary, else default. :rtype: any

setdefault(key: str, default: ~typing.Any = <object object>) -> Any

Parameters

Name	Description
key Required
default

update

Update the dictionary from a mapping or an iterable of key-value pairs. :param any args: Either a mapping object or an iterable of key-value pairs.

update(*args: Any, **kwargs: Any) -> None

values

values() -> ValuesView[Any]

Returns

Type	Description
ValuesView	an object providing a view on the mapping's values

Attributes

evaluation_model

Model configuration used by model-based evaluators and conversation simulation. Required.

evaluation_model: _models.EvaluationModelConfiguration

evaluators

Evaluator references used to score candidate quality. Required.

evaluators: list['_models.AgentOptimizationEvaluator']

max_concurrent_agent_runs

Maximum number of target-agent runs executed concurrently during each single-turn evaluation. If omitted, the service defaults to 1. Conversation evaluation supports only 1.

max_concurrent_agent_runs: int | None

training_set

Evaluation set used to guide the optimization search. Inline data supports up to 2,000 test cases when a separate validation set is supplied; otherwise it is also used for validation and is limited to 500. Required.

training_set: _models.AgentOptimizationEvaluationSet

validation_set

Held-out evaluation set used for full candidate evaluation, limited to 500 inline test cases. The training set is reused and subject to the same 500-test-case validation limit when omitted.

validation_set: _models.AgentOptimizationEvaluationSet | None

Comentários

Esta página foi útil?