medcat2.stats.kfold

Attributes

MedCATTrainerExportProjectInfo

The project name, project ID, CUIs str, and TUIs str

IntValuedMetric

FloatValuedMetric

Classes

CAT

This is a collection of serialisable model parts.

MedCATTrainerExport

dict() -> new empty dictionary

MedCATTrainerExportProject

dict() -> new empty dictionary

MedCATTrainerExportDocument

dict() -> new empty dictionary

MedCATTrainerExportAnnotation

dict() -> new empty dictionary

SplitType

The split type.

FoldCreator

The FoldCreator based on a MCT export.

SimpleFoldCreator

The FoldCreator based on a MCT export.

PerDocsFoldCreator

The FoldCreator based on a MCT export.

PerAnnsFoldCreator

The FoldCreator based on a MCT export.

WeightedDocumentsCreator

The FoldCreator based on a MCT export.

PerCUIMetrics

Usage docs: https://docs.pydantic.dev/2.9/concepts/models/

Functions

captured_state_cdb(cdb[, save_state_to_disk])

A context manager that captures and re-applies the initial CDB state.

get_stats(cat, data[, epoch, use_project_filters, ...])

TODO: Refactor and make nice

count_all_annotations(export)

Count the number of annotations in a trainer export.

count_all_docs(export)

Count the number of documents in a trainer export.

get_nr_of_annotations(doc)

Get the number of annotations for a tariner export document.

iter_anns(export)

Iterate over all the annotations in a trainer export.

iter_docs(export)

Iterate over all the docs in a trainer export.

get_fold_creator(mct_export, nr_of_folds, split_type)

Get the appropriate fold creator.

get_per_fold_metrics(cat, folds, *args, **kwargs)

Get per fold metrics for a given set of folds.

_merge_examples(all_examples, cur_examples)

_add_helper(joined, single)

_add_weighted_helper(joined, single, cui2count)

get_metrics_mean(metrics, include_std)

The the mean of the provided metrics.

get_k_fold_stats(cat, mct_export_data[, k, ...])

Get the k-fold stats for the model with the specified data.

Module Contents

class medcat2.stats.kfold.CAT(cdb, vocab=None, config=None, model_load_path=None)

Bases: medcat2.storage.serialisables.AbstractSerialisable

This is a collection of serialisable model parts.

Parameters:
__init__(cdb, vocab=None, config=None, model_load_path=None)
Parameters:
Return type:

None

cdb
vocab = None
config = None
_trainer: medcat2.trainer.Trainer | None = None
_pipeline
_recrate_pipe(model_load_path=None)
Parameters:

model_load_path (Optional[str])

Return type:

medcat2.pipeline.pipeline.Pipeline

classmethod get_init_attrs()
Return type:

list[str]

classmethod ignore_attrs()
Return type:

list[str]

__call__(text)
Parameters:

text (str)

Return type:

Optional[medcat2.tokenizing.tokens.MutableDocument]

_ensure_not_training()

Method to ensure config is not set to train.

config.components.linking.train should only be True while training and not during inference. This aalso corrects the setting if necessary.

Return type:

None

get_entities(text, only_cui=False)

Get the entities recognised and linked within the provided text.

This will run the text through the pipeline and annotated the recognised and linked entities.

Parameters:

  • text (str) – The text to use.

  • only_cui (bool, optional) – Whether to only output the CUIs rather than the entire context. Defaults to False.

Returns:

Union[dict, Entities, OnlyCUIEntities] – The entities found and linked within the text.

Return type:

Union[dict, medcat2.data.entities.Entities, medcat2.data.entities.OnlyCUIEntities]

_get_entity(ent, doc_tokens, cui)
Parameters:
Return type:

medcat2.data.entities.Entity

_doc_to_out_entity(ent, doc_tokens, only_cui)
Parameters:
Return type:

tuple[int, Union[medcat2.data.entities.Entity, str]]

_doc_to_out(doc, only_cui, out_with_text=False)
Parameters:
Return type:

Union[medcat2.data.entities.Entities, medcat2.data.entities.OnlyCUIEntities]

property trainer

The trainer object.

save_model_pack(target_folder, pack_name=DEFAULT_PACK_NAME, serialiser_type='dill', make_archive=True)

Save model pack.

Parameters:

  • target_folder (str) – The folder to save the pack in.

  • pack_name (str, optional) – The model pack name. Defaults to DEFAULT_PACK_NAME.

  • serialiser_type (Union[str, AvailableSerialisers], optional) – The serialiser type. Defaults to ‘dill’.

  • make_archive (bool) – Whether to make the arhive /.zip file. Defaults to True.

Returns:

str – The final model pack path.

Return type:

str

classmethod load_model_pack(model_pack_path)

Load the model pack from file.

Parameters:

model_pack_path (str) – The model pack path.

Raises:

ValueError – If the saved data does not represent a model pack.

Returns:

CAT – The loaded model pack.

Return type:

CAT

__eq__(other)
Parameters:

other (Any)

Return type:

bool

add_addon(addon)
Parameters:

addon (medcat2.components.addons.addons.AddonComponent)

Return type:

None

get_strategy()
Return type:

SerialisingStrategy

classmethod include_properties()
Return type:

list[str]

medcat2.stats.kfold.captured_state_cdb(cdb, save_state_to_disk=False)

A context manager that captures and re-applies the initial CDB state.

The context manager captures/copies the initial state of the CDB when entering. It then allows the user to modify the state (i.e training). Upon exit re-applies the initial CDB state.

If RAM is an issue, it is recommended to use save_state_to_disk. Otherwise the copy of the original state will be held in memory. If saved on disk, a temporary file is used and removed afterwards.

Parameters:

  • cdb – The CDB to use.

  • save_state_to_disk (bool) – Whether to save state on disk or hold in memory. Defaults to False.

Yields:

None

medcat2.stats.kfold.get_stats(cat, data, epoch=0, use_project_filters=False, use_overlaps=False, extra_cui_filter=None, do_print=True)

TODO: Refactor and make nice Print metrics on a dataset (F1, P, R), it will also print the concepts that have the most FP,FN,TP.

Parameters:

  • cat (medcat2.cat.CAT) – (CAT): The model pack.

  • data (dict) – The json object that we get from MedCATtrainer on export.

  • epoch (int) – Used during training, so we know what epoch is it.

  • use_project_filters (bool) – Each project in MedCATtrainer can have filters, do we want to respect those filters when calculating metrics.

  • use_overlaps (bool) – Allow overlapping entities, nearly always False as it is very difficult to annotate overlapping entities.

  • use_cui_doc_limit (bool) – If True the metrics for a CUI will be only calculated if that CUI appears in a document, in other words if the document was annotated for that CUI. Useful in very specific situations when during the annotation process the set of CUIs changed.

  • use_groups (bool) – If True concepts that have groups will be combined and stats will be reported on groups.

  • extra_cui_filter (Optional[set]) – This filter will be intersected with all other filters, or if all others are not set then only this one will be used.

  • do_print (bool) – Whether to print stats out. Defaults to True.

Returns:

  • fps (dict) – False positives for each CUI.

  • fns (dict) – False negatives for each CUI.

  • tps (dict) – True positives for each CUI.

  • cui_prec (dict) – Precision for each CUI.

  • cui_rec (dict) – Recall for each CUI.

  • cui_f1 (dict) – F1 for each CUI.

  • cui_counts (dict) – Number of occurrence for each CUI.

  • examples (dict) – Examples for each of the fp, fn, tp. Format will be examples[‘fp’][‘cui’][<list_of_examples>].

Return type:

tuple[dict[str, int], dict[str, int], dict[str, int], dict[str, float], dict[str, float], dict[str, float], dict[str, int], dict]

class medcat2.stats.kfold.MedCATTrainerExport

Bases: typing_extensions.TypedDict

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

projects: list[MedCATTrainerExportProject]
__contains__()

True if the dictionary has the specified key, else False.

__delattr__()

Implement delattr(self, name).

__delitem__()

Delete self[key].

__dir__()

Default dir() implementation.

__eq__()

Return self==value.

__format__()

Default object formatter.

__ge__()

Return self>=value.

__getattribute__()

Return getattr(self, name).

__getitem__()

x.__getitem__(y) <==> x[y]

__gt__()

Return self>value.

__init__()

Initialize self. See help(type(self)) for accurate signature.

__ior__()

Return self|=value.

__iter__()

Implement iter(self).

__le__()

Return self<=value.

__len__()

Return len(self).

__lt__()

Return self<value.

__ne__()

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__or__()

Return self|value.

__reduce__()

Helper for pickle.

__reduce_ex__()

Helper for pickle.

__repr__()

Return repr(self).

__reversed__()

Return a reverse iterator over the dict keys.

__ror__()

Return value|self.

__setattr__()

Implement setattr(self, name, value).

__setitem__()

Set self[key] to value.

__sizeof__()

D.__sizeof__() -> size of D in memory, in bytes

__str__()

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

clear()

D.clear() -> None. Remove all items from D.

copy()

D.copy() -> a shallow copy of D

get()

Return the value for key if key is in the dictionary, else default.

items()

D.items() -> a set-like object providing a view on D’s items

keys()

D.keys() -> a set-like object providing a view on D’s keys

pop()

D.pop(k[,d]) -> v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault()

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update()

D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values()

D.values() -> an object providing a view on D’s values

class medcat2.stats.kfold.MedCATTrainerExportProject

Bases: typing_extensions.TypedDict

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

name: str
id: Any
cuis: str
tuis: str | None
documents: list[MedCATTrainerExportDocument]
__contains__()

True if the dictionary has the specified key, else False.

__delattr__()

Implement delattr(self, name).

__delitem__()

Delete self[key].

__dir__()

Default dir() implementation.

__eq__()

Return self==value.

__format__()

Default object formatter.

__ge__()

Return self>=value.

__getattribute__()

Return getattr(self, name).

__getitem__()

x.__getitem__(y) <==> x[y]

__gt__()

Return self>value.

__init__()

Initialize self. See help(type(self)) for accurate signature.

__ior__()

Return self|=value.

__iter__()

Implement iter(self).

__le__()

Return self<=value.

__len__()

Return len(self).

__lt__()

Return self<value.

__ne__()

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__or__()

Return self|value.

__reduce__()

Helper for pickle.

__reduce_ex__()

Helper for pickle.

__repr__()

Return repr(self).

__reversed__()

Return a reverse iterator over the dict keys.

__ror__()

Return value|self.

__setattr__()

Implement setattr(self, name, value).

__setitem__()

Set self[key] to value.

__sizeof__()

D.__sizeof__() -> size of D in memory, in bytes

__str__()

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

clear()

D.clear() -> None. Remove all items from D.

copy()

D.copy() -> a shallow copy of D

get()

Return the value for key if key is in the dictionary, else default.

items()

D.items() -> a set-like object providing a view on D’s items

keys()

D.keys() -> a set-like object providing a view on D’s keys

pop()

D.pop(k[,d]) -> v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault()

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update()

D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values()

D.values() -> an object providing a view on D’s values

class medcat2.stats.kfold.MedCATTrainerExportDocument

Bases: typing_extensions.TypedDict

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

name: str
id: Any
last_modified: str
text: str
annotations: list[MedCATTrainerExportAnnotation]
__contains__()

True if the dictionary has the specified key, else False.

__delattr__()

Implement delattr(self, name).

__delitem__()

Delete self[key].

__dir__()

Default dir() implementation.

__eq__()

Return self==value.

__format__()

Default object formatter.

__ge__()

Return self>=value.

__getattribute__()

Return getattr(self, name).

__getitem__()

x.__getitem__(y) <==> x[y]

__gt__()

Return self>value.

__init__()

Initialize self. See help(type(self)) for accurate signature.

__ior__()

Return self|=value.

__iter__()

Implement iter(self).

__le__()

Return self<=value.

__len__()

Return len(self).

__lt__()

Return self<value.

__ne__()

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__or__()

Return self|value.

__reduce__()

Helper for pickle.

__reduce_ex__()

Helper for pickle.

__repr__()

Return repr(self).

__reversed__()

Return a reverse iterator over the dict keys.

__ror__()

Return value|self.

__setattr__()

Implement setattr(self, name, value).

__setitem__()

Set self[key] to value.

__sizeof__()

D.__sizeof__() -> size of D in memory, in bytes

__str__()

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

clear()

D.clear() -> None. Remove all items from D.

copy()

D.copy() -> a shallow copy of D

get()

Return the value for key if key is in the dictionary, else default.

items()

D.items() -> a set-like object providing a view on D’s items

keys()

D.keys() -> a set-like object providing a view on D’s keys

pop()

D.pop(k[,d]) -> v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault()

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update()

D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values()

D.values() -> an object providing a view on D’s values

class medcat2.stats.kfold.MedCATTrainerExportAnnotation

Bases: typing_extensions.TypedDict

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

start: int
end: int
cui: str
value: str
__contains__()

True if the dictionary has the specified key, else False.

__delattr__()

Implement delattr(self, name).

__delitem__()

Delete self[key].

__dir__()

Default dir() implementation.

__eq__()

Return self==value.

__format__()

Default object formatter.

__ge__()

Return self>=value.

__getattribute__()

Return getattr(self, name).

__getitem__()

x.__getitem__(y) <==> x[y]

__gt__()

Return self>value.

__init__()

Initialize self. See help(type(self)) for accurate signature.

__ior__()

Return self|=value.

__iter__()

Implement iter(self).

__le__()

Return self<=value.

__len__()

Return len(self).

__lt__()

Return self<value.

__ne__()

Return self!=value.

__new__()

Create and return a new object. See help(type) for accurate signature.

__or__()

Return self|value.

__reduce__()

Helper for pickle.

__reduce_ex__()

Helper for pickle.

__repr__()

Return repr(self).

__reversed__()

Return a reverse iterator over the dict keys.

__ror__()

Return value|self.

__setattr__()

Implement setattr(self, name, value).

__setitem__()

Set self[key] to value.

__sizeof__()

D.__sizeof__() -> size of D in memory, in bytes

__str__()

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

clear()

D.clear() -> None. Remove all items from D.

copy()

D.copy() -> a shallow copy of D

get()

Return the value for key if key is in the dictionary, else default.

items()

D.items() -> a set-like object providing a view on D’s items

keys()

D.keys() -> a set-like object providing a view on D’s keys

pop()

D.pop(k[,d]) -> v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault()

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update()

D.update([E, ]**F) -> None. Update D from dict/iterable E and F. If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values()

D.values() -> an object providing a view on D’s values

medcat2.stats.kfold.count_all_annotations(export)

Count the number of annotations in a trainer export.

Parameters:

export (MedCATTrainerExport) – The trainer export.

Returns:

int – The total number of annotations.

Return type:

int

medcat2.stats.kfold.count_all_docs(export)

Count the number of documents in a trainer export.

Parameters:

export (MedCATTrainerExport) – The trainer export.

Returns:

int – The total number of documents.

Return type:

int

medcat2.stats.kfold.get_nr_of_annotations(doc)

Get the number of annotations for a tariner export document.

Parameters:

doc (MedCATTrainerExportDocument) – The trainer export document.

Returns:

int – The number of annotations within the document.

Return type:

int

medcat2.stats.kfold.iter_anns(export)

Iterate over all the annotations in a trainer export.

Parameters:

export (MedCATTrainerExport) – The trainer export.

Yields:
Iterator[tuple[MedCATTrainerExportProjectInfo,

MedCATTrainerExportDocument, MedCATTrainerExportAnnotation]]:

The project info, the document, and the annotation.

Return type:

Iterator[tuple[MedCATTrainerExportProjectInfo, MedCATTrainerExportDocument, MedCATTrainerExportAnnotation]]

medcat2.stats.kfold.iter_docs(export)

Iterate over all the docs in a trainer export.

Parameters:

export (MedCATTrainerExport) – The trainer export.

Yields:
Iterator[tuple[MedCATTrainerExportProjectInfo,

MedCATTrainerExportDocument]]:

The project info and the document.

Return type:

Iterator[tuple[MedCATTrainerExportProjectInfo, MedCATTrainerExportDocument]]

medcat2.stats.kfold.MedCATTrainerExportProjectInfo

The project name, project ID, CUIs str, and TUIs str

class medcat2.stats.kfold.SplitType

Bases: enum.Enum

The split type.

DOCUMENTS

Split over number of documents.

ANNOTATIONS

Split over number of annotations.

DOCUMENTS_WEIGHTED

Split over number of documents based on the number of annotations. So essentially this ensures that the same document isn’t in 2 folds while trying to more equally distribute documents with different number of annotations. For example:

If we have 6 documents that we want to split into 3 folds. The number of annotations per document are as follows:

[40, 40, 20, 10, 5, 5]

If we were to split this trivially over documents, we’d end up with the 3 folds with number of annotations that are far from even:

[80, 30, 10]

However, if we use the annotations as weights, we would be able to create folds that have more evenly distributed annotations, e.g:

[[D1,], [D2], [D3, D4, D5, D6]]

where D# denotes the number of the documents, with the number of annotations being equal:

[ 40, 40, 20 + 10 + 5 + 5 = 40]

__new__(value)
_generate_next_value_(start, count, last_values)

Generate the next value when not given.

name: the name of the member start: the initial start value or None count: the number of existing members last_value: the last value assigned or None

classmethod _missing_(value)
__repr__()
__str__()
__dir__()

Returns all members and all public methods

__format__(format_spec)

Returns format using actual value type unless __str__ has been overridden.

__hash__()
__reduce_ex__(proto)
name()

The name of the Enum member.

value()

The value of the Enum member.

class medcat2.stats.kfold.FoldCreator(mct_export, nr_of_folds)

Bases: abc.ABC

The FoldCreator based on a MCT export.

Parameters:

  • mct_export (MedCATTrainerExport) – The MCT export dict.

  • nr_of_folds (int) – Number of folds to create.

  • use_annotations (bool) – Whether to fold on number of annotations or documents.

__init__(mct_export, nr_of_folds)
Parameters:
Return type:

None

mct_export
nr_of_folds
_find_or_add_doc(project, orig_doc)
Parameters:
Return type:

medcat2.data.mctexport.MedCATTrainerExportDocument

_create_new_project(proj_info)
Parameters:

proj_info (medcat2.data.mctexport.MedCATTrainerExportProjectInfo)

Return type:

medcat2.data.mctexport.MedCATTrainerExportProject

_create_export_with_documents(relevant_docs)
Parameters:

relevant_docs (Iterable[tuple[medcat2.data.mctexport.MedCATTrainerExportProjectInfo, medcat2.data.mctexport.MedCATTrainerExportDocument]])

Return type:

medcat2.data.mctexport.MedCATTrainerExport

abstract create_folds()

Create folds.

Raises:

ValueError – If something went wrong.

Returns:

list[MedCATTrainerExport] – The created folds.

Return type:

list[medcat2.data.mctexport.MedCATTrainerExport]

__slots__ = ()
class medcat2.stats.kfold.SimpleFoldCreator(mct_export, nr_of_folds, counter)

Bases: FoldCreator

The FoldCreator based on a MCT export.

Parameters:
__init__(mct_export, nr_of_folds, counter)
Parameters:
Return type:

None

_counter
total
per_fold
_init_per_fold()
Return type:

list[int]

abstract _create_fold(fold_nr)
Parameters:

fold_nr (int)

Return type:

medcat2.data.mctexport.MedCATTrainerExport

create_folds()

Create folds.

Raises:

ValueError – If something went wrong.

Returns:

list[MedCATTrainerExport] – The created folds.

Return type:

list[medcat2.data.mctexport.MedCATTrainerExport]

mct_export
nr_of_folds
_find_or_add_doc(project, orig_doc)
Parameters:
Return type:

medcat2.data.mctexport.MedCATTrainerExportDocument

_create_new_project(proj_info)
Parameters:

proj_info (medcat2.data.mctexport.MedCATTrainerExportProjectInfo)

Return type:

medcat2.data.mctexport.MedCATTrainerExportProject

_create_export_with_documents(relevant_docs)
Parameters:

relevant_docs (Iterable[tuple[medcat2.data.mctexport.MedCATTrainerExportProjectInfo, medcat2.data.mctexport.MedCATTrainerExportDocument]])

Return type:

medcat2.data.mctexport.MedCATTrainerExport

__slots__ = ()
class medcat2.stats.kfold.PerDocsFoldCreator(mct_export, nr_of_folds)

Bases: FoldCreator

The FoldCreator based on a MCT export.

Parameters:

  • mct_export (MedCATTrainerExport) – The MCT export dict.

  • nr_of_folds (int) – Number of folds to create.

  • use_annotations (bool) – Whether to fold on number of annotations or documents.

__init__(mct_export, nr_of_folds)
Parameters:
Return type:

None

nr_of_docs
per_doc_simple
_all_docs
_create_fold(fold_nr)
Parameters:

fold_nr (int)

Return type:

medcat2.data.mctexport.MedCATTrainerExport

create_folds()

Create folds.

Raises:

ValueError – If something went wrong.

Returns:

list[MedCATTrainerExport] – The created folds.

Return type:

list[medcat2.data.mctexport.MedCATTrainerExport]

mct_export
nr_of_folds
_find_or_add_doc(project, orig_doc)
Parameters:
Return type:

medcat2.data.mctexport.MedCATTrainerExportDocument

_create_new_project(proj_info)
Parameters:

proj_info (medcat2.data.mctexport.MedCATTrainerExportProjectInfo)

Return type:

medcat2.data.mctexport.MedCATTrainerExportProject

_create_export_with_documents(relevant_docs)
Parameters:

relevant_docs (Iterable[tuple[medcat2.data.mctexport.MedCATTrainerExportProjectInfo, medcat2.data.mctexport.MedCATTrainerExportDocument]])

Return type:

medcat2.data.mctexport.MedCATTrainerExport

__slots__ = ()
class medcat2.stats.kfold.PerAnnsFoldCreator(mct_export, nr_of_folds)

Bases: SimpleFoldCreator

The FoldCreator based on a MCT export.

Parameters:

  • mct_export (MedCATTrainerExport) – The MCT export dict.

  • nr_of_folds (int) – Number of folds to create.

  • use_annotations (bool) – Whether to fold on number of annotations or documents.

__init__(mct_export, nr_of_folds)
Parameters:
Return type:

None

_add_target_ann(project, orig_doc, ann)
Parameters:
Return type:

None

_targets(start_at)
Parameters:

start_at (int)

Return type:

Iterable[tuple[medcat2.data.mctexport.MedCATTrainerExportProjectInfo, medcat2.data.mctexport.MedCATTrainerExportDocument, medcat2.data.mctexport.MedCATTrainerExportAnnotation]]

_create_fold(fold_nr)
Parameters:

fold_nr (int)

Return type:

medcat2.data.mctexport.MedCATTrainerExport

_counter
total
per_fold
_init_per_fold()
Return type:

list[int]

create_folds()

Create folds.

Raises:

ValueError – If something went wrong.

Returns:

list[MedCATTrainerExport] – The created folds.

Return type:

list[medcat2.data.mctexport.MedCATTrainerExport]

mct_export
nr_of_folds
_find_or_add_doc(project, orig_doc)
Parameters:
Return type:

medcat2.data.mctexport.MedCATTrainerExportDocument

_create_new_project(proj_info)
Parameters:

proj_info (medcat2.data.mctexport.MedCATTrainerExportProjectInfo)

Return type:

medcat2.data.mctexport.MedCATTrainerExportProject

_create_export_with_documents(relevant_docs)
Parameters:

relevant_docs (Iterable[tuple[medcat2.data.mctexport.MedCATTrainerExportProjectInfo, medcat2.data.mctexport.MedCATTrainerExportDocument]])

Return type:

medcat2.data.mctexport.MedCATTrainerExport

__slots__ = ()
class medcat2.stats.kfold.WeightedDocumentsCreator(mct_export, nr_of_folds, weight_calculator)

Bases: FoldCreator

The FoldCreator based on a MCT export.

Parameters:
__init__(mct_export, nr_of_folds, weight_calculator)
Parameters:
Return type:

None

_weight_calculator
_weighted_docs
create_folds()

Create folds.

Raises:

ValueError – If something went wrong.

Returns:

list[MedCATTrainerExport] – The created folds.

Return type:

list[medcat2.data.mctexport.MedCATTrainerExport]

mct_export
nr_of_folds
_find_or_add_doc(project, orig_doc)
Parameters:
Return type:

medcat2.data.mctexport.MedCATTrainerExportDocument

_create_new_project(proj_info)
Parameters:

proj_info (medcat2.data.mctexport.MedCATTrainerExportProjectInfo)

Return type:

medcat2.data.mctexport.MedCATTrainerExportProject

_create_export_with_documents(relevant_docs)
Parameters:

relevant_docs (Iterable[tuple[medcat2.data.mctexport.MedCATTrainerExportProjectInfo, medcat2.data.mctexport.MedCATTrainerExportDocument]])

Return type:

medcat2.data.mctexport.MedCATTrainerExport

__slots__ = ()
medcat2.stats.kfold.get_fold_creator(mct_export, nr_of_folds, split_type)

Get the appropriate fold creator.

Parameters:

  • mct_export (MedCATTrainerExport) – The MCT export.

  • nr_of_folds (int) – Number of folds to use.

  • split_type (SplitType) – The type of split to use.

Raises:

ValueError – In case of an unknown split type.

Returns:

FoldCreator – The corresponding fold creator.

Return type:

FoldCreator

medcat2.stats.kfold.get_per_fold_metrics(cat, folds, *args, **kwargs)

Get per fold metrics for a given set of folds.

This method captures the state of the before processing each fold. For each fold, it trains on all other folds, and runs metrics on the fold itself.

Parameters:
Returns:

list[tuple] – The metrics for each fold.

Return type:

list[tuple]

medcat2.stats.kfold._merge_examples(all_examples, cur_examples)
Parameters:

  • all_examples (dict)

  • cur_examples (dict)

Return type:

None

medcat2.stats.kfold.IntValuedMetric
medcat2.stats.kfold.FloatValuedMetric
class medcat2.stats.kfold.PerCUIMetrics(/, **data)

Bases: pydantic.BaseModel

Usage docs: https://docs.pydantic.dev/2.9/concepts/models/

A base class for creating Pydantic models.

Parameters:

data (Any)

__class_vars__

The names of the class variables defined on the model.

__private_attributes__

Metadata about the private attributes of the model.

__signature__

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__

The core schema of the model.

__pydantic_custom_init__

Whether the model has a custom __init__ function.

__pydantic_decorators__

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__

Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__

The name of the post-init method for the model, if defined.

__pydantic_root_model__

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_extra__

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__

The names of fields explicitly set during instantiation.

__pydantic_private__

Values of private attributes set on the model instance.

weights: list[int | float] = []
vals: list[int | float] = []
add(val, weight=1)
Parameters:

weight (int)

get_mean()
get_std()
model_config: ClassVar[pydantic.config.ConfigDict]

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[Dict[str, pydantic.fields.FieldInfo]]

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo] objects.

This replaces Model.__fields__ from Pydantic V1.

model_computed_fields: ClassVar[Dict[str, pydantic.fields.ComputedFieldInfo]]

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

__class_vars__: ClassVar[set[str]]

The names of the class variables defined on the model.

__private_attributes__: ClassVar[Dict[str, pydantic.fields.ModelPrivateAttr]]

Metadata about the private attributes of the model.

__signature__: ClassVar[inspect.Signature]

The synthesized __init__ [Signature][inspect.Signature] of the model.

__pydantic_complete__: ClassVar[bool] = False

Whether model building is completed, or if there are still undefined fields.

__pydantic_core_schema__: ClassVar[pydantic_core.CoreSchema]

The core schema of the model.

__pydantic_custom_init__: ClassVar[bool]

Whether the model has a custom __init__ method.

__pydantic_decorators__: ClassVar[pydantic._internal._decorators.DecoratorInfos]

Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.

__pydantic_generic_metadata__: ClassVar[pydantic._internal._generics.PydanticGenericMetadata]

Metadata for generic models; contains data used for a similar purpose to __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

__pydantic_parent_namespace__: ClassVar[Dict[str, Any] | None] = None

Parent namespace of the model, used for automatic rebuilding of models.

__pydantic_post_init__: ClassVar[None | Literal['model_post_init']]

The name of the post-init method for the model, if defined.

__pydantic_root_model__: ClassVar[bool] = False

Whether the model is a [RootModel][pydantic.root_model.RootModel].

__pydantic_serializer__: ClassVar[pydantic_core.SchemaSerializer]

The pydantic-core SchemaSerializer used to dump instances of the model.

__pydantic_validator__: ClassVar[pydantic_core.SchemaValidator | pydantic.plugin._schema_validator.PluggableSchemaValidator]

The pydantic-core SchemaValidator used to validate instances of the model.

__pydantic_extra__: dict[str, Any] | None

A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.

__pydantic_fields_set__: set[str]

The names of fields explicitly set during instantiation.

__pydantic_private__: dict[str, Any] | None

Values of private attributes set on the model instance.

__slots__ = ('__dict__', '__pydantic_fields_set__', '__pydantic_extra__', '__pydantic_private__')
__init__(/, **data)

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Parameters:

data (Any)

Return type:

None

property model_extra: dict[str, Any] | None

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or `None` if `config.extra` is not set to `”allow”`.

Return type:

dict[str, Any] | None

property model_fields_set: set[str]

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set, – i.e. that were not filled from defaults.

Return type:

set[str]

classmethod model_construct(_fields_set=None, **values)

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (set[str] | None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Returns:

A new instance of the `Model` class with validated data.

Return type:

typing_extensions.Self

model_copy(*, update=None, deep=False)

Usage docs: https://docs.pydantic.dev/2.9/concepts/serialization/#model_copy

Returns a copy of the model.

Parameters:

  • update (dict[str, Any] | None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool) – Set to True to make a deep copy of the model.

Returns:

New model instance.

Return type:

typing_extensions.Self

model_dump(*, mode='python', include=None, exclude=None, context=None, by_alias=False, exclude_unset=False, exclude_defaults=False, exclude_none=False, round_trip=False, warnings=True, serialize_as_any=False)

Usage docs: https://docs.pydantic.dev/2.9/concepts/serialization/#modelmodel_dump

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Parameters:

  • mode (Literal['json', 'python'] | str) – The mode in which to_python should run. If mode is ‘json’, the output will only contain JSON serializable types. If mode is ‘python’, the output may contain non-JSON-serializable Python objects.

  • include (IncEx | None) – A set of fields to include in the output.

  • exclude (IncEx | None) – A set of fields to exclude from the output.

  • context (Any | None) – Additional context to pass to the serializer.

  • by_alias (bool) – Whether to use the field’s alias in the dictionary key if defined.

  • exclude_unset (bool) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool) – Whether to exclude fields that have a value of None.

  • round_trip (bool) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (bool | Literal['none', 'warn', 'error']) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • serialize_as_any (bool) – Whether to serialize fields with duck-typing serialization behavior.

Returns:

A dictionary representation of the model.

Return type:

dict[str, Any]

model_dump_json(*, indent=None, include=None, exclude=None, context=None, by_alias=False, exclude_unset=False, exclude_defaults=False, exclude_none=False, round_trip=False, warnings=True, serialize_as_any=False)

Usage docs: https://docs.pydantic.dev/2.9/concepts/serialization/#modelmodel_dump_json

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (int | None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • include (IncEx | None) – Field(s) to include in the JSON output.

  • exclude (IncEx | None) – Field(s) to exclude from the JSON output.

  • context (Any | None) – Additional context to pass to the serializer.

  • by_alias (bool) – Whether to serialize using field aliases.

  • exclude_unset (bool) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool) – Whether to exclude fields that have a value of None.

  • round_trip (bool) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (bool | Literal['none', 'warn', 'error']) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • serialize_as_any (bool) – Whether to serialize fields with duck-typing serialization behavior.

Returns:

A JSON string representation of the model.

Return type:

str

classmethod model_json_schema(by_alias=True, ref_template=DEFAULT_REF_TEMPLATE, schema_generator=GenerateJsonSchema, mode='validation')

Generates a JSON schema for a model class.

Parameters:

  • by_alias (bool) – Whether to use attribute aliases or not.

  • ref_template (str) – The reference template.

  • schema_generator (type[pydantic.json_schema.GenerateJsonSchema]) – To override the logic used to generate the JSON schema, as a subclass of GenerateJsonSchema with your desired modifications

  • mode (pydantic.json_schema.JsonSchemaMode) – The mode in which to generate the schema.

Returns:

The JSON schema for the given model class.

Return type:

dict[str, Any]

classmethod model_parametrized_name(params)

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], Ellipsis]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Returns:

String representing the new class where `params` are passed to `cls` as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

Return type:

str

model_post_init(__context)

Override this method to perform additional initialization after __init__ and model_construct. This is useful if you want to do some validation that requires the entire model to be initialized.

Parameters:

__context (Any)

Return type:

None

classmethod model_rebuild(*, force=False, raise_errors=True, _parent_namespace_depth=2, _types_namespace=None)

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (dict[str, Any] | None) – The types namespace, defaults to None.

Returns:

  • Returns `None` if the schema is already “complete” and rebuilding was not required.

  • If rebuilding _was_ required, returns `True` if rebuilding was successful, otherwise `False`.

Return type:

bool | None

classmethod model_validate(obj, *, strict=None, from_attributes=None, context=None)

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (bool | None) – Whether to enforce types strictly.

  • from_attributes (bool | None) – Whether to extract data from object attributes.

  • context (Any | None) – Additional context to pass to the validator.

Raises:

ValidationError – If the object could not be validated.

Returns:

The validated model instance.

Return type:

typing_extensions.Self

classmethod model_validate_json(json_data, *, strict=None, context=None)

Usage docs: https://docs.pydantic.dev/2.9/concepts/json/#json-parsing

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (bool | None) – Whether to enforce types strictly.

  • context (Any | None) – Extra variables to pass to the validator.

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

Return type:

typing_extensions.Self

classmethod model_validate_strings(obj, *, strict=None, context=None)

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (bool | None) – Whether to enforce types strictly.

  • context (Any | None) – Extra variables to pass to the validator.

Returns:

The validated Pydantic model.

Return type:

typing_extensions.Self

classmethod __get_pydantic_core_schema__(source, handler, /)

Hook into generating the model’s CoreSchema.

Parameters:

  • source (type[BaseModel]) – The class we are generating a schema for. This will generally be the same as the cls argument if this is a classmethod.

  • handler (pydantic.annotated_handlers.GetCoreSchemaHandler) – A callable that calls into Pydantic’s internal CoreSchema generation logic.

Returns:

A `pydantic-core` `CoreSchema`.

Return type:

pydantic_core.CoreSchema

classmethod __get_pydantic_json_schema__(core_schema, handler, /)

Hook into generating the model’s JSON schema.

Parameters:

  • core_schema (pydantic_core.CoreSchema) – A pydantic-core CoreSchema. You can ignore this argument and call the handler with a new CoreSchema, wrap this CoreSchema ({‘type’: ‘nullable’, ‘schema’: current_schema}), or just call the handler with the original schema.

  • handler (pydantic.annotated_handlers.GetJsonSchemaHandler) – Call into Pydantic’s internal JSON schema generation. This will raise a pydantic.errors.PydanticInvalidForJsonSchema if JSON schema generation fails. Since this gets called by BaseModel.model_json_schema you can override the schema_generator argument to that function to change JSON schema generation globally for a type.

Returns:

A JSON schema, as a Python object.

Return type:

pydantic.json_schema.JsonSchemaValue

classmethod __pydantic_init_subclass__(**kwargs)

This is intended to behave just like __init_subclass__, but is called by ModelMetaclass only after the class is actually fully initialized. In particular, attributes like model_fields will be present when this is called.

This is necessary because __init_subclass__ will always be called by type.__new__, and it would require a prohibitively large refactor to the ModelMetaclass to ensure that type.__new__ was called in such a manner that the class would already be sufficiently initialized.

This will receive the same kwargs that would be passed to the standard __init_subclass__, namely, any kwargs passed to the class definition that aren’t used internally by pydantic.

Parameters:

**kwargs (Any) – Any keyword arguments passed to the class definition that aren’t used internally by pydantic.

Return type:

None

classmethod __class_getitem__(typevar_values)
Parameters:

typevar_values (type[Any] | tuple[type[Any], Ellipsis])

Return type:

type[BaseModel] | pydantic._internal._forward_ref.PydanticRecursiveRef

__copy__()

Returns a shallow copy of the model.

Return type:

typing_extensions.Self

__deepcopy__(memo=None)

Returns a deep copy of the model.

Parameters:

memo (dict[int, Any] | None)

Return type:

typing_extensions.Self

__getattr__(item)
Parameters:

item (str)

Return type:

Any

_check_frozen(name, value)
Parameters:

  • name (str)

  • value (Any)

Return type:

None

__getstate__()
Return type:

dict[Any, Any]

__setstate__(state)
Parameters:

state (dict[Any, Any])

Return type:

None

__eq__(other)
Parameters:

other (Any)

Return type:

bool

classmethod __init_subclass__(**kwargs)

This signature is included purely to help type-checkers check arguments to class declaration, which provides a way to conveniently set model_config key/value pairs.

```py from pydantic import BaseModel

class MyModel(BaseModel, extra=’allow’): … ```

However, this may be deceiving, since the _actual_ calls to __init_subclass__ will not receive any of the config arguments, and will only receive any keyword arguments passed during class initialization that are _not_ expected keys in ConfigDict. (This is due to the way ModelMetaclass.__new__ works.)

Parameters:

**kwargs (typing_extensions.Unpack[pydantic.config.ConfigDict]) – Keyword arguments passed to the class definition, which set model_config

Note

You may want to override __pydantic_init_subclass__ instead, which behaves similarly but is called after the class is fully initialized.

__iter__()

So dict(model) works.

Return type:

TupleGenerator

__repr__()
Return type:

str

__repr_args__()
Return type:

pydantic._internal._repr.ReprArgs

__repr_name__
__repr_str__
__pretty__
__rich_repr__
__str__()
Return type:

str

property __fields__: dict[str, pydantic.fields.FieldInfo]
Return type:

dict[str, pydantic.fields.FieldInfo]

property __fields_set__: set[str]
Return type:

set[str]

dict(*, include=None, exclude=None, by_alias=False, exclude_unset=False, exclude_defaults=False, exclude_none=False)
Parameters:

  • include (IncEx | None)

  • exclude (IncEx | None)

  • by_alias (bool)

  • exclude_unset (bool)

  • exclude_defaults (bool)

  • exclude_none (bool)

Return type:

Dict[str, Any]

json(*, include=None, exclude=None, by_alias=False, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=PydanticUndefined, models_as_dict=PydanticUndefined, **dumps_kwargs)
Parameters:

  • include (IncEx | None)

  • exclude (IncEx | None)

  • by_alias (bool)

  • exclude_unset (bool)

  • exclude_defaults (bool)

  • exclude_none (bool)

  • encoder (Callable[[Any], Any] | None)

  • models_as_dict (bool)

  • dumps_kwargs (Any)

Return type:

str

classmethod parse_obj(obj)
Parameters:

obj (Any)

Return type:

typing_extensions.Self

classmethod parse_raw(b, *, content_type=None, encoding='utf8', proto=None, allow_pickle=False)
Parameters:

  • b (str | bytes)

  • content_type (str | None)

  • encoding (str)

  • proto (pydantic.deprecated.parse.Protocol | None)

  • allow_pickle (bool)

Return type:

typing_extensions.Self

classmethod parse_file(path, *, content_type=None, encoding='utf8', proto=None, allow_pickle=False)
Parameters:

  • path (str | pathlib.Path)

  • content_type (str | None)

  • encoding (str)

  • proto (pydantic.deprecated.parse.Protocol | None)

  • allow_pickle (bool)

Return type:

typing_extensions.Self

classmethod from_orm(obj)
Parameters:

obj (Any)

Return type:

typing_extensions.Self

classmethod construct(_fields_set=None, **values)
Parameters:

  • _fields_set (set[str] | None)

  • values (Any)

Return type:

typing_extensions.Self

copy(*, include=None, exclude=None, update=None, deep=False)

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`py data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (pydantic._internal._utils.AbstractSetIntStr | pydantic._internal._utils.MappingIntStrAny | None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (pydantic._internal._utils.AbstractSetIntStr | pydantic._internal._utils.MappingIntStrAny | None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Dict[str, Any] | None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool) – If True, the values of fields that are Pydantic models will be deep-copied.

Returns:

A copy of the model with included, excluded and updated fields as specified.

Return type:

typing_extensions.Self

classmethod schema(by_alias=True, ref_template=DEFAULT_REF_TEMPLATE)
Parameters:

  • by_alias (bool)

  • ref_template (str)

Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias=True, ref_template=DEFAULT_REF_TEMPLATE, **dumps_kwargs)
Parameters:

  • by_alias (bool)

  • ref_template (str)

  • dumps_kwargs (Any)

Return type:

str

classmethod validate(value)
Parameters:

value (Any)

Return type:

typing_extensions.Self

classmethod update_forward_refs(**localns)
Parameters:

localns (Any)

Return type:

None

_iter(*args, **kwargs)
Parameters:

  • args (Any)

  • kwargs (Any)

Return type:

Any

_copy_and_set_values(*args, **kwargs)
Parameters:

  • args (Any)

  • kwargs (Any)

Return type:

Any

classmethod _get_value(*args, **kwargs)
Parameters:

  • args (Any)

  • kwargs (Any)

Return type:

Any

_calculate_keys(*args, **kwargs)
Parameters:

  • args (Any)

  • kwargs (Any)

Return type:

Any

medcat2.stats.kfold._add_helper(joined, single)
Parameters:

  • joined (list[dict[str, PerCUIMetrics]])

  • single (list[dict[str, int]])

Return type:

None

medcat2.stats.kfold._add_weighted_helper(joined, single, cui2count)
Parameters:

  • joined (list[dict[str, PerCUIMetrics]])

  • single (list[dict[str, float]])

  • cui2count (dict[str, int])

Return type:

None

medcat2.stats.kfold.get_metrics_mean(metrics, include_std)

The the mean of the provided metrics.

Parameters:

  • metrics (list[tuple[dict, dict, dict, dict, dict, dict, dict, dict]) – The metrics.

  • include_std (bool) – Whether to include the standard deviation.

Returns:

  • fps (dict) – False positives for each CUI.

  • fns (dict) – False negatives for each CUI.

  • tps (dict) – True positives for each CUI.

  • cui_prec (dict) – Precision for each CUI.

  • cui_rec (dict) – Recall for each CUI.

  • cui_f1 (dict) – F1 for each CUI.

  • cui_counts (dict) – Number of occurrence for each CUI.

  • examples (dict) – Examples for each of the fp, fn, tp. Format will be examples[‘fp’][‘cui’][<list_of_examples>].

Return type:

tuple[dict, dict, dict, dict, dict, dict, dict, dict]

medcat2.stats.kfold.get_k_fold_stats(cat, mct_export_data, k=3, split_type=SplitType.DOCUMENTS_WEIGHTED, include_std=False, *args, **kwargs)

Get the k-fold stats for the model with the specified data.

First this will split the MCT export into k folds. You can do this either per document or per-annotation.

For each of the k folds, it will start from the base model, train it with with the other k-1 folds and record the metrics. After that the base model state is restored before doing the next fold. After all the folds have been done, the metrics are averaged.

Parameters:

  • cat (CAT) – The model pack.

  • mct_export_data (MedCATTrainerExport) – The MCT export.

  • k (int) – The number of folds. Defaults to 3.

  • split_type (SplitType) – Whether to use annodations or docs. Defaults to DOCUMENTS_WEIGHTED.

  • include_std (bool) – Whether to include stanrdard deviation. Defaults to False.

  • *args – Arguments passed to the CAT.train_supervised_raw method.

  • **kwargs – Keyword arguments passed to the CAT.train_supervised_raw method.

Returns:

tuple – The averaged metrics. Potentially with their corresponding standard deviations.

Return type:

tuple