Skip to content

Commit

Permalink
📖 Fix CalibrationError docstring
Browse files Browse the repository at this point in the history
  • Loading branch information
alafage committed Feb 21, 2025
1 parent b07b278 commit 9f98e04
Showing 1 changed file with 24 additions and 24 deletions.
48 changes: 24 additions & 24 deletions torch_uncertainty/metrics/classification/calibration_error.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,39 +237,39 @@ def __new__( # type: ignore[misc]
.. math::
\text{ECE} = \sum_{i=1}^N b_i \lvert p_i - c_i \rvert
\text{ECE} = \sum_{i=1}^N b_i \lvert p_i - c_i \rvert
**Maximum Calibration Error (MCE):**
.. math::
\text{MCE} = \max_{i} \lvert p_i - c_i \rvert
\text{MCE} = \max_{i} \lvert p_i - c_i \rvert
**Root Mean Square Calibration Error (RMSCE):**
.. math::
\text{RMSCE} = \sqrt{\sum_{i=1}^N b_i (p_i - c_i)^2}
\text{RMSCE} = \sqrt{\sum_{i=1}^N b_i (p_i - c_i)^2}
Here:
- :math:`p_i` is the accuracy of bin :math:`i` (fraction of correct predictions).
- :math:`c_i` is the mean predicted confidence in bin :math:`i`.
- :math:`b_i` is the fraction of total samples falling into bin :math:`i`.
- :math:`p_i` is the accuracy of bin :math:`i` (fraction of correct predictions).
- :math:`c_i` is the mean predicted confidence in bin :math:`i`.
- :math:`b_i` is the fraction of total samples falling into bin :math:`i`.
Bins are constructed either uniformly in the range :math:`[0, 1]` or adaptively
(if `adaptive=True`).
Args:
task (str): Specifies the task type, either ``"binary"`` or ``"multiclass"``.
adaptive (bool, optional): Whether to use adaptive binning. Defaults to ``False``.
num_bins (int, optional): Number of bins to divide the probability space. Defaults to ``10``.
norm (str, optional): Specifies the type of norm to use: ``"l1"``, ``"l2"``, or ``"max"``.
Defaults to ``"l1"``.
num_classes (int, optional): Number of classes for ``"multiclass"`` tasks. Required when task
is ``"multiclass"``.
ignore_index (int, optional): Index to ignore during calculations. Defaults to ``None``.
validate_args (bool, optional): Whether to validate input arguments. Defaults to ``True``.
**kwargs: Additional keyword arguments for the metric.
task (str): Specifies the task type, either ``"binary"`` or ``"multiclass"``.
adaptive (bool, optional): Whether to use adaptive binning. Defaults to ``False``.
num_bins (int, optional): Number of bins to divide the probability space. Defaults to ``10``.
norm (str, optional): Specifies the type of norm to use: ``"l1"``, ``"l2"``, or ``"max"``.
Defaults to ``"l1"``.
num_classes (int, optional): Number of classes for ``"multiclass"`` tasks. Required when task
is ``"multiclass"``.
ignore_index (int, optional): Index to ignore during calculations. Defaults to ``None``.
validate_args (bool, optional): Whether to validate input arguments. Defaults to ``True``.
**kwargs: Additional keyword arguments for the metric.
Example:
Expand All @@ -292,20 +292,20 @@ def __new__( # type: ignore[misc]
# Output: Calibration Error: 0.199
Note:
- Bins are either uniformly distributed in :math:`[0, 1]` or adaptively sized
(if `adaptive=True`).
Bins are either uniformly distributed in :math:`[0, 1]` or adaptively sized
(if `adaptive=True`).
Warning:
If `task="multiclass"`, `num_classes` must be an integer; otherwise, a :class:`TypeError`
is raised.
If `task="multiclass"`, `num_classes` must be an integer; otherwise, a :class:`TypeError`
is raised.
References:
[1] `Naeini et al. Obtaining well calibrated probabilities using Bayesian binning. In AAAI, 2015
<https://ojs.aaai.org/index.php/AAAI/article/view/9602>`_.
[1] `Naeini et al. Obtaining well calibrated probabilities using Bayesian binning. In AAAI, 2015
<https://ojs.aaai.org/index.php/AAAI/article/view/9602>`_.
.. seealso::
- See `CalibrationError <https://torchmetrics.readthedocs.io/en/stable/classification/calibration_error.html>`_
for details. Our version of the metric is a wrapper around the original metric providing a plotting functionality.
See `CalibrationError <https://torchmetrics.readthedocs.io/en/stable/classification/calibration_error.html>`_
for details. Our version of the metric is a wrapper around the original metric providing a plotting functionality.
"""
if kwargs.get("n_bins") is not None:
raise ValueError("`n_bins` does not exist, use `num_bins`.")
Expand Down

0 comments on commit 9f98e04

Please sign in to comment.