langtest.utils.custom_types.sample.FactualitySample#

class FactualitySample(*, article_sent: str, incorrect_sent: str, correct_sent: str, state: str = None, dataset_name: str = None, task: str = None, category: str = None, test_type: str = None, result: str = None, swapped_result: str = None, ran_pass: bool = None)#

Bases: BaseModel

A class representing a sample for the Factuality task.

article_sent#

The original article sentence.

Type:

str

incorrect_sent#

The incorrect version of the sentence.

Type:

str

correct_sent#

The correct version of the sentence.

Type:

str

state#

The state of the sample (e.g., ‘draft’, ‘final’).

Type:

str, optional

dataset_name#

The name of the dataset.

Type:

str, optional

task#

The task related to the sample.

Type:

str, optional

category#

The category of the sample.

Type:

str, optional

test_type#

The type of test conducted on the sample.

Type:

str, optional

result#

Stores the output when the correct summary is presented first.

Type:

str, optional

swapped_result#

Stores the output when the incorrect summary is presented first.

Type:

str, optional

to_dict()#

Convert the sample to a dictionary.

is_pass()#

Check if the sample passes the evaluation.

remove_punctuation(input_string)#

Remove punctuation from the input string.

_is_eval()#

Internal method to evaluate the sample.

run(model, **kwargs)#

Run the sample through a specified model.

__init__(**data)#

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Methods

__init__(**data)	Create a new model by parsing and validating input data from keyword arguments.
construct([_fields_set])	Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
copy(*[, include, exclude, update, deep])	Duplicate a model, optionally choose which fields to include, exclude and change.
dict(*[, include, exclude, by_alias, ...])	Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
from_orm(obj)
is_pass()	Check if the sample passes the evaluation.
json(*[, include, exclude, by_alias, ...])	Generate a JSON representation of the model, include and exclude arguments as per dict().
parse_file(path, *[, content_type, ...])
parse_obj(obj)
parse_raw(b, *[, content_type, encoding, ...])
remove_punctuation(input_string)	Remove punctuation from the input string.
run(model, **kwargs)	Run the sample through a specified model.
schema([by_alias, ref_template])
schema_json(*[, by_alias, ref_template])
to_dict()	Convert the sample to a dictionary.
update_forward_refs(**localns)	Try to update ForwardRefs on fields based on this Model, globalns and localns.
validate(value)

Attributes

article_sent
incorrect_sent
correct_sent
state
dataset_name
task
category
test_type
result
swapped_result
ran_pass

classmethod construct(_fields_set: SetStr | None = None, **values: Any) → Model#

Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values

copy(*, include: AbstractSetIntStr | MappingIntStrAny | None = None, exclude: AbstractSetIntStr | MappingIntStrAny | None = None, update: DictStrAny | None = None, deep: bool = False) → Model#

Duplicate a model, optionally choose which fields to include, exclude and change.

Parameters:

• include – fields to include in new model

• exclude – fields to exclude from new model, as with values this takes precedence over include

• update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data

• deep – set to True to make a deep copy of the model

Returns:

new model instance

dict(*, include: AbstractSetIntStr | MappingIntStrAny | None = None, exclude: AbstractSetIntStr | MappingIntStrAny | None = None, by_alias: bool = False, skip_defaults: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

is_pass()#

Check if the sample passes the evaluation.

Returns:

True if the sample passes, False otherwise.

Return type:

bool

json(*, include: AbstractSetIntStr | MappingIntStrAny | None = None, exclude: AbstractSetIntStr | MappingIntStrAny | None = None, by_alias: bool = False, skip_defaults: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode#

Generate a JSON representation of the model, include and exclude arguments as per dict().

encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

remove_punctuation(input_string)#

Remove punctuation from the input string.

Parameters:

input_string (str) – The input string with punctuation.

Returns:

The input string with punctuation removed.

Return type:

str

run(model, **kwargs)#

Run the sample through a specified model.

Parameters:

• model – The machine learning model to run the sample through.

• **kwargs – Additional keyword arguments.

Returns:

True if the operation was successful.

Return type:

bool

to_dict() → Dict[str, Any]#

Convert the sample to a dictionary.

Returns:

A dictionary representation of the sample.

Return type:

dict

classmethod update_forward_refs(**localns: Any) → None#

Try to update ForwardRefs on fields based on this Model, globalns and localns.