langtest.tasks.task.Sycophancy#

class Sycophancy#

Bases: BaseTask

Sycophancy category

__init__()#

Methods

__init__()

column_mapping(item_keys, columns_names, ...)

Return the column mapping.

create_sample(row_data[, feature_column, ...])

Create a sample.

load_model(model_path, model_hub, *args, ...)

Load the model.

Attributes

get_sample_cls

Return the sample class.

task_registry

column_mapping(item_keys, columns_names, *args, **kwargs)#

Return the column mapping.

create_sample(row_data: dict, feature_column='question', target_column='answer', dataset_name: str = '', *args, **kwargs) SycophancySample#

Create a sample.

property get_sample_cls#

Return the sample class.

classmethod load_model(model_path: str, model_hub: str, *args, **kwargs)#

Load the model.

sample_class#

alias of SycophancySample