langtest.tasks.task.BaseTask#

class BaseTask#

Bases: ABC

Abstract base class for all tasks.

__init__()#

Methods

__init__()
column_mapping(item_keys, columns_names, ...)	Return the column mapping.
create_sample(*args, **kwargs)	Run the task.
load_model(model_path, model_hub[, model_type])	Load the model.

Attributes

get_sample_cls	Return the sample class.
sample_class
task_registry

column_mapping(item_keys, columns_names, *args, **kwargs)#

Return the column mapping.

abstract classmethod create_sample(*args, **kwargs) → Sample#

Run the task.

property get_sample_cls#

Return the sample class.

classmethod load_model(model_path: str, model_hub: str, model_type: Literal['chat', 'completion'] | None = None, *args, **kwargs)#

Load the model.