RubricBasedEvaluatorDefinition Class

Rubric-based evaluator definition — stores dimensions produced by the generate API. Used for both quality and safety evaluators.

Constructor

RubricBasedEvaluatorDefinition(*args: Any, **kwargs: Any)

Variables

Name	Description
init_parameters	dict[str, any] The JSON schema (Draft 2020-12) for the evaluator's input parameters. This includes parameters like type, properties, required.
data_schema	dict[str, any] The JSON schema (Draft 2020-12) for the evaluator's input data. This includes parameters like type, properties, required.
metrics	dict[str, EvaluatorMetric] List of output metrics produced by this evaluator.
type	str or <xref:azure.ai.projects.models.RUBRIC> Required. Rubric-based evaluator definition. Stores dimensions (the scoring blueprint) for both quality and safety evaluators. Can be created via the generate API or manually via createVersion.
dimensions	list[Dimension] The set of dimensions — the scoring blueprint used by the LLM judge. Quality evaluators include a non-editable residual dimension with id 'general_quality' (always_applicable: true); safety evaluators include 'general_policy_compliance'. Both use the same Dimension structure. Required.
pass_threshold	float Pass/fail threshold for the aggregate rubric score, on the same normalized 0.0-1.0 scale as the emitted `score`. When the runtime weighted average meets or exceeds this value, the result is `pass`. Defaults to 0.5 (equivalent to a raw 1-5 weighted average of 3.0). The 'any dimension scored 1 → fail' rule still applies regardless of this threshold.

Methods

as_dict	Return a dict that can be turned into json using json.dump.
clear	Remove all items from the dictionary.
copy
get	Get the value for key if key is in the dictionary, else default. :param str key: The key to look up. :param any default: The value to return if key is not in the dictionary. Defaults to None :returns: The value for key if key is in the dictionary, else default. :rtype: any
items
keys
pop	Removes specified key and return the corresponding value. :param str key: The key to pop. :param any default: The value to return if key is not in the dictionary :returns: The value corresponding to the key. :rtype: any :raises KeyError: If key is not found and default is not given.
popitem	Removes and returns some (key, value) pair :returns: The (key, value) pair. :rtype: tuple :raises KeyError: if the dictionary is empty.
setdefault	Return the value for key if key is in the dictionary; otherwise set the key to default and return default. :param str key: The key to look up. :param any default: The value to set if key is not in the dictionary :returns: The value for key if key is in the dictionary, else default. :rtype: any
update	Update the dictionary from a mapping or an iterable of key-value pairs. :param any args: Either a mapping object or an iterable of key-value pairs.
values

as_dict

Return a dict that can be turned into json using json.dump.

as_dict(*, exclude_readonly: bool = False) -> dict[str, Any]

Keyword-Only Parameters

Name	Description
exclude_readonly	bool Whether to remove the readonly properties. Default value: False

Returns

Type	Description
dict	A dict JSON compatible object

clear

Remove all items from the dictionary.

clear() -> None

copy

copy() -> Model

get

Get the value for key if key is in the dictionary, else default. :param str key: The key to look up. :param any default: The value to return if key is not in the dictionary. Defaults to None :returns: The value for key if key is in the dictionary, else default. :rtype: any

get(key: str, default: Any = None) -> Any

Parameters

Name	Description
key Required
default	Default value: None

items

items() -> ItemsView[str, Any]

Returns

Type	Description
ItemsView	a set-like object providing a view on the mapping's items

keys

keys() -> KeysView[str]

Returns

Type	Description
KeysView	a set-like object providing a view on the mapping's keys

pop

Removes specified key and return the corresponding value. :param str key: The key to pop. :param any default: The value to return if key is not in the dictionary :returns: The value corresponding to the key. :rtype: any :raises KeyError: If key is not found and default is not given.

pop(key: str, default: ~typing.Any = <object object>) -> Any

Parameters

Name	Description
key Required
default

popitem

Removes and returns some (key, value) pair :returns: The (key, value) pair. :rtype: tuple :raises KeyError: if the dictionary is empty.

popitem() -> tuple[str, Any]

setdefault

Return the value for key if key is in the dictionary; otherwise set the key to default and return default. :param str key: The key to look up. :param any default: The value to set if key is not in the dictionary :returns: The value for key if key is in the dictionary, else default. :rtype: any

setdefault(key: str, default: ~typing.Any = <object object>) -> Any

Parameters

Name	Description
key Required
default

update

Update the dictionary from a mapping or an iterable of key-value pairs. :param any args: Either a mapping object or an iterable of key-value pairs.

update(*args: Any, **kwargs: Any) -> None

values

values() -> ValuesView[Any]

Returns

Type	Description
ValuesView	an object providing a view on the mapping's values

Attributes

data_schema

The JSON schema (Draft 2020-12) for the evaluator's input data. This includes parameters like type, properties, required.

data_schema: dict[str, Any] | None

dimensions

The set of dimensions — the scoring blueprint used by the LLM judge. Quality evaluators include a non-editable residual dimension with id 'general_quality' (always_applicable: true); safety evaluators include 'general_policy_compliance'. Both use the same Dimension structure. Required.

dimensions: list['_models.Dimension']

init_parameters

The JSON schema (Draft 2020-12) for the evaluator's input parameters. This includes parameters like type, properties, required.

init_parameters: dict[str, Any] | None

metrics

List of output metrics produced by this evaluator.

metrics: dict[str, '_models.EvaluatorMetric'] | None

pass_threshold

Pass/fail threshold for the aggregate rubric score, on the same normalized 0.0-1.0 scale as the emitted score. When the runtime weighted average meets or exceeds this value, the result is pass. Defaults to 0.5 (equivalent to a raw 1-5 weighted average of 3.0). The 'any dimension scored 1 → fail' rule still applies regardless of this threshold.

pass_threshold: float | None

type

Required. Rubric-based evaluator definition. Stores dimensions (the scoring blueprint) for both quality and safety evaluators. Can be created via the generate API or manually via createVersion.

type: RUBRIC: 'rubric'>]

Opinia

Czy ta strona była pomocna?