spateo.segmentation.benchmark#

Functions to help segmentation benchmarking, specifically to compare two sets of segmentation labels.

Module Contents#

Functions#

adjusted_rand_score(→ float)

Compute the Adjusted Rand Score (ARS).

iou(→ scipy.sparse.csr_matrix)

Compute intersection-over-union (IOU).

average_precision(→ float)

Compute average precision (AP).

classification_stats(→ Tuple[float, float, float, ...)

Calculate pixel classification statistics by considering labeled pixels

labeling_stats(→ Tuple[float, float, float, float])

Calculate labeling (cluster) statistics.

compare(), seed)

Compute segmentation statistics.
spateo.segmentation.benchmark.adjusted_rand_score(y_true: numpy.ndarray, y_pred: numpy.ndarray) → float[source]#

Compute the Adjusted Rand Score (ARS).

Re-implementation to deal with over/underflow that is common with large datasets.

Parameters
y_true

True labels

y_pred

Predicted labels

Returns

Adjusted Rand Score

spateo.segmentation.benchmark.iou(labels1: numpy.ndarray, labels2: numpy.ndarray) → scipy.sparse.csr_matrix[source]#

Compute intersection-over-union (IOU).

Parameters
labels1

First set of labels

labels2

Second set of labels

Returns

Sparse matrix where the first axis corresponds to the first set of

labels and vice-versa.

spateo.segmentation.benchmark.average_precision(iou: scipy.sparse.csr_matrix, tau: float = 0.5) → float[source]#

Compute average precision (AP).

Parameters
iou

IOU of true and predicted labels

tau

IOU threshold to determine whether a prediction is correct

Returns

Average precision

spateo.segmentation.benchmark.classification_stats(y_true: numpy.ndarray, y_pred: numpy.ndarray) → Tuple[float, float, float, float, float, float, float][source]#

Calculate pixel classification statistics by considering labeled pixels as occupied (1) and unlabled pixels as unoccupied (0).

Parameters
y_true

True labels

y_pred

Predicted labels

Returns

  • true negative rate

  • false positive rate

  • false negative rate

  • true positive rate (a.k.a. recall)

  • precision

  • accuracy

  • F1 score

Return type

A 7-element tuple containing the following values

spateo.segmentation.benchmark.labeling_stats(y_true: numpy.ndarray, y_pred: numpy.ndarray) → Tuple[float, float, float, float][source]#

Calculate labeling (cluster) statistics.

Parameters
y_true

True labels

y_pred

Predicted labels

Returns

  • adjusted rand score

  • homogeneity

  • completeness

  • v score

Return type

A 4-element tuple containing the following values

spateo.segmentation.benchmark.compare(adata: anndata.AnnData, true_layer: str, pred_layer: str, data_layer: str = SKM.X_LAYER, umi_pixels_only: bool = True, random_background: bool = True, ap_taus: Tuple[int, Ellipsis] = tuple(np.arange(0.5, 1, 0.05)), seed: Optional[int] = None) → pandas.DataFrame[source]#

Compute segmentation statistics.

Parameters
adata

Input Anndata

true_layer

Layer containing true labels

pred_layer

Layer containing predicted labels

data_layer

Layer containing UMIs

umi_pixels_only

Whether or not to only consider pixels that have at least one UMI captured (as determined by data_layer).

random_background

Simulate random background by randomly permuting the pred_layer labels and computing the same statistics against true_layer. The returned DataFrame will have an additional column for these statistics.

ap_taus

Tau thresholds to calculate average precision. Defaults to 0.05 increments starting at 0.5 and ending at (and including) 0.95.

seed

Random seed.

Returns

Pandas DataFrame containing classification and labeling statistics