langtest.utils.custom_types.sample.NERSample#

class NERSample(*, original: str = None, test_type: str = None, test_case: str = None, expected_results: Result = None, actual_results: Result = None, transformations: List[Transformation] = None, category: str = None, state: str = None, threshold: float = None, dataset_name: str = None, gender: str = None, task: str = 'ner')#

Bases: BaseSample

__init__(**data)#

Constructor method

Methods

__init__(**data)	Constructor method
construct([_fields_set])	Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
copy(*[, include, exclude, update, deep])	Duplicate a model, optionally choose which fields to include, exclude and change.
dict(*[, include, exclude, by_alias, ...])	Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
from_orm(obj)
get_aligned_span_pairs()	Realigns the original text with the perturbed by using the Transformations
is_pass()	Checks if the sample passes based on the maximum score.
json(*[, include, exclude, by_alias, ...])	Generate a JSON representation of the model, include and exclude arguments as per dict().
parse_file(path, *[, content_type, ...])
parse_obj(obj)
parse_raw(b, *[, content_type, encoding, ...])
schema([by_alias, ref_template])
schema_json(*[, by_alias, ref_template])
sort_transformations(v)	Validator ensuring that transformations are in correct order
to_dict()	Returns the dict version of sample.
update_forward_refs(**localns)	Try to update ForwardRefs on fields based on this Model, globalns and localns.
validate(value)

Attributes

ignored_predictions	List of predictions that should be ignored because of the perturbations applied
irrelevant_transformations	Retrieves the transformations that do not need to be taken into
realigned_spans	Shifting the actual_results spans according to the perturbations that were applied to the text.
relevant_transformations	Retrieves the transformations that need to be taken into account to realign original and test_case.
gender	Helper object for named entity recognition tasks
task

classmethod construct(_fields_set: SetStr | None = None, **values: Any) → Model#

Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

copy(*, include: AbstractSetIntStr | MappingIntStrAny | None = None, exclude: AbstractSetIntStr | MappingIntStrAny | None = None, update: DictStrAny | None = None, deep: bool = False) → Model#

Duplicate a model, optionally choose which fields to include, exclude and change.

Parameters:

• include – fields to include in new model

• exclude – fields to exclude from new model, as with values this takes precedence over include

• update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

• deep – set to True to make a deep copy of the model

Returns:

new model instance

dict(*, include: AbstractSetIntStr | MappingIntStrAny | None = None, exclude: AbstractSetIntStr | MappingIntStrAny | None = None, by_alias: bool = False, skip_defaults: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

gender: str#

Helper object for named entity recognition tasks

get_aligned_span_pairs() → List[Tuple[NERPrediction | None, NERPrediction | None]]#

Realigns the original text with the perturbed by using the Transformations

Returns:

List of aligned predicted spans from the original sentence to the perturbed one. The tuples are of the form: (perturbed span, original span). The alignment is achieved by using the transformations apply to the original text. If a Span couldn’t be aligned with any other the tuple is of the form (Span, None) (or (None, Span)).

Return type:

List[Tuple[Optional[NERPrediction], Optional[NERPrediction]]]

property ignored_predictions: List[NERPrediction]#

List of predictions that should be ignored because of the perturbations applied

Returns:

list of predictions which should be ignored

Return type:

List[NERPrediction]

property irrelevant_transformations: List[Transformation] | None#

Retrieves the transformations that do not need to be taken into

account to realign original and test_case.

Returns:

list of transformations which should be ignored

Return type:

Optional[List[Transformation]]

is_pass() → bool#

Checks if the sample passes based on the maximum score.

json(*, include: AbstractSetIntStr | MappingIntStrAny | None = None, exclude: AbstractSetIntStr | MappingIntStrAny | None = None, by_alias: bool = False, skip_defaults: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode#

Generate a JSON representation of the model, include and exclude arguments as per dict().

encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

property realigned_spans: NEROutput#

Shifting the actual_results spans according to the perturbations that were applied to the text.

Note: we ignore predicted spans that were added during a perturbation

Returns:

realigned NER predictions

Return type:

NEROutput

property relevant_transformations: List[Transformation] | None#

Retrieves the transformations that need to be taken into account to realign original and test_case.

Returns:

list of transformations which shouldn’t be ignored

Return type:

Optional[List[Transformation]]

classmethod sort_transformations(v)#

Validator ensuring that transformations are in correct order

to_dict() → Dict[str, Any]#

Returns the dict version of sample.

classmethod update_forward_refs(**localns: Any) → None#

Try to update ForwardRefs on fields based on this Model, globalns and localns.