medcat.utils.ner.metrics
========================

.. py:module:: medcat.utils.ner.metrics


Attributes
----------

.. autoapisummary::

   medcat.utils.ner.metrics.logger


Functions
---------

.. autoapisummary::

   medcat.utils.ner.metrics.metrics
   medcat.utils.ner.metrics._anno_within_pred_list
   medcat.utils.ner.metrics.evaluate_predictions


Module Contents
---------------

.. py:data:: logger

.. py:function:: metrics(p, return_df=False, plus_recall=0, tokenizer=None, dataset=None, merged_negative={0, 1, -100}, padding_label=-100, csize=15, subword_label=1, verbose=False)

   Calculate metrics for a model's predictions, based off the tokenized
   output of a MedCATTrainer project.

   :param p: The model's predictions.
   :param return_df: Whether to return a DataFrame of metrics.
   :param plus_recall: The recall to add to the model's predictions.
   :param tokenizer: The tokenizer used to tokenize the texts.
   :param dataset: The dataset used to train the model.
   :param merged_negative: The negative labels to merge.
   :param padding_label: The padding label.
   :param csize: The size of the context window.
   :param subword_label: The subword label.
   :param verbose: Whether to print the metrics.

   :Returns: **Dict** -- A dictionary of metrics.


.. py:function:: _anno_within_pred_list(label, preds)

   Check if a label is within a list of predictions,

   :param label: an annotation likely from a MedCATTrainer project
   :type label: Dict
   :param preds: a list of predictions likely from a cat.__call__
   :type preds: List[Dict]

   :Returns: **bool** -- True if the label is within the list of predictions,
             False otherwise


.. py:function:: evaluate_predictions(true_annotations, all_preds, texts, cui2preferred_name)

   Evaluate predictions against sets of collected labels as collected and
   utput from a MedCATTrainer project.
   Counts predictions as correct if the prediction fully encloses the label.

   :param true_annotations: Ground truth predictions by text
   :type true_annotations: list[list[dict]]
   :param all_preds: Model predictions by text
   :type all_preds: list[list[dict]]
   :param texts: Original list of texts
   :type texts: list[str]
   :param cui2preferred_name: Dictionary of CUI to preferred
                              name, likely to be cat.cdb.cui2preferred_name.
   :type cui2preferred_name: dict[str, str]

   :Returns: **tuple[pd.DataFrame, Dict]** -- A tuple containing a DataFrame of
             evaluation metrics and a dictionary of missed annotations per CUI.