TOOD - Task-Aware Out-of-Distribution Score Calibration for Continual Learners

Publications  · 
Download the paper Check the code

Continual Learning (CL), the task of learning new tasks over time while retaining knowledge of previous ones, is important for deep learning systems that need to adapt after deployment. A robot, for example, should be able to learn new objects without forgetting those it already recognizes. For reliable deployment, it also needs to identify unfamiliar inputs that fall outside its learned classes. This ability, known as Out-of-Distribution (OOD) detection, allows the system to flag inputs it is not equipped to classify.

In this research, we address an often overlooked aspect of continual learning: a model can retain useful classification accuracy on familiar, in-distribution (ID) data while losing its ability to distinguish that data from unfamiliar inputs. We study this deterioration, which we call OOD Forgetting (OODF), and propose TOOD (Task-Aware OOD Score Calibration) to improve detection by calibrating each task’s scores using a small set of familiar examples.

I’m excited to share that our paper was accepted to CoLLAs 2026, the Conference on Lifelong Learning Agents, for an oral presentation! You can also see the conference’s post on X.

Our Core Intuition

Our approach is built on two observations about how a model’s output scores change during continual learning.

First, classification depends on the relative ordering of the model’s output scores, or logits. As long as the correct class receives the highest score, the prediction remains correct. Second, many OOD detectors depend on the absolute values of those scores. A change in their scale can therefore weaken OOD detection even when the predicted class stays the same.

As the model learns new classes, we observe that logits associated with older tasks tend to shrink relative to those of newer tasks. We call this the confidence gap. An example from an old task may still be classified correctly, but its detection score can now resemble that of an OOD input.

This led us to ask: How strongly does an input match each task, compared with familiar examples from that task? We reasoned that measuring each response relative to its own task would make older and newer tasks easier to compare. TOOD does this by computing a separate energy score for each task, aligning those scores using ID examples, and taking the strongest calibrated response.

Animation illustrating how familiar and unfamiliar input scores overlap as new tasks are learned, and how TOOD calibration improves their separation
An animated overview of the confidence gap and TOOD's calibration. The score distributions are illustrative, not measured experimental data.

What is OOD Detection in Continual Learning?

Imagine a robot that first learns to recognize cups and plates, then learns books and pens. After the second task, it should recognize objects from both tasks. If it encounters a shoe, which belongs to neither task, it should be able to flag it as unfamiliar instead of confidently assigning it to one of the four known classes.

Continual learning makes this problem more challenging because the boundary between ID and OOD changes over time. A book is unfamiliar before the second task but becomes a known class afterward. At the same time, learning books and pens changes the model’s parameters and can alter the scores it produces for cups and plates.

We focus on class-incremental learning, where new classes arrive in successive tasks and the model must classify an input without being told which task it belongs to. For a consistent measure of OOD forgetting, we evaluate against OOD datasets whose classes remain outside the learned class set throughout the sequence of tasks.

Classification forgetting measures whether the robot can still distinguish cups from plates after learning new objects. OOD forgetting measures whether it can still separate those familiar objects from something it has never learned. Our experiments show that preserving the first ability does not guarantee preserving the second.

Two Mechanisms Behind OOD Forgetting

Our experiments identify two recurring mechanisms that help explain why OOD detection deteriorates.

  1. The Confidence Gap: Logits for older tasks tend to shrink relative to those of newer tasks. Detectors that combine all class logits into a single energy score can then struggle to separate familiar examples from OOD inputs, even when those examples are still classified correctly.
  2. Manifold Crowding: As new classes arrive, their feature representations fill regions that previously separated known examples from outliers. OOD samples become closer to known-class clusters, making detection based on feature distances more difficult.

Together, these observations explain why remembering class labels is not enough to preserve OOD detection. TOOD addresses the confidence gap by correcting task-wise output scores. The loss of separation in feature space requires a complementary solution.

Task-zero classification accuracy and OOD AUROC across five CIFAR-10 tasks, comparing uncalibrated energy with TOOD mean shift
OOD forgetting on a five-task CIFAR-10 stream with iCaRL. At task step T3, uncalibrated energy has lost roughly 35 AUROC points from T0, while TOOD mean shift recovers about 30 points. Classification accuracy and OOD AUROC use separate vertical axes; this diagnostic tracks task 0, rather than the average over all tasks.

Proposed Framework: TOOD

TOOD is designed to make the scores of older and newer tasks comparable. It is fully post-hoc, meaning it works on an already-trained model without retraining. Using the model’s logits and a small ID calibration memory, we compute an additional detection score while leaving its class predictions unchanged.

TOOD pipeline: split logits by task, align task energy scales using ID calibration statistics, and take the strongest normalized response
TOOD calibrates each task's energy using a small ID memory, then takes the strongest normalized response. The diagram shows the core score without the optional margin term.

The core components of TOOD are:

  1. Per-Task Energy Decomposition: We group the logits according to the classes introduced in each task and compute an energy score for each group. This preserves information about how strongly an input matches each task.
  2. Task-Wise Normalization: We use familiar examples from each task to align the typical values of its energy scores. The Robust Anchor variant also adjusts their spread.
  3. Maximum Normalized Energy: We take the strongest calibrated task response as the final ID score. An input that matches a known task can receive a high score even if that task’s raw logits have weakened over time.

TOOD Components in Detail

Per-Task Energy

For an input \(x\), we first collect the logits associated with each task. Let \(h_c(x)\) be the logit for class \(c\), and let \(\mathcal{C}_t\) contain the classes introduced in task \(t\). We define the task’s energy score as:

\[E_t(x) = \log \sum_{c \in \mathcal{C}_t} \exp(h_c(x)).\]

We use the sign-reversed energy convention, so higher scores indicate ID. Each energy summarizes the model’s response to one task’s classes. An ID input should produce a strong response for a relevant task, while an OOD input should have no similarly strong match. Before comparing these responses, we need to correct the differences in scale that develop during continual learning.

Calibrating with Familiar Examples

After learning each task, we pass a small set of ID examples through the frozen model to estimate the energy statistics for every task seen so far. When the learner already stores examples for replay, we can reuse that memory. For methods without replay, we use a small held-out ID set. This calibration requires no OOD examples and no gradient updates.

We provide two variants:

Mean Shift. Let \(\mu_t\) be the mean of task \(t\)’s energy on its own calibration examples. We align it with the mean of the most recent task, \(\mu_{\text{ref}}\):

\[E_t^{\text{ms}}(x) = E_t(x) + (\mu_{\text{ref}} - \mu_t).\]

This shifts each task’s scores so that its familiar examples have the same mean as those of the reference task. It corrects the difference in average energy while keeping the spread of the scores unchanged.

Robust Anchor. To account for differences in both location and spread, we use the median \(\tilde{e}_t\) and the scaled median absolute deviation \(\widehat{\mathrm{MAD}}_t\):

\[E_t^{\text{rob}}(x) = \frac{E_t(x) - \tilde{e}_t}{\widehat{\mathrm{MAD}}_t}\,\widehat{\mathrm{MAD}}_{\text{ref}} + \tilde{e}_{\text{ref}}.\]

The median describes the typical energy, while MAD measures how far scores tend to lie from that median. Both are less sensitive to outliers than statistics based on the mean. The transformation aligns each task with the most recent task’s reference statistics, correcting differences in both the typical score and its spread.

Final Detection Score

The core TOOD score is the maximum normalized energy across all seen tasks:

\[S_{\text{TOOD}}(x) = \max_t E_t^{\text{norm}}(x).\]

A high score means that at least one known task provides a strong match. A low score suggests that the input is unfamiliar to all the tasks the model has learned and should be considered for OOD rejection. Because we evaluate every task’s response, TOOD does not need the test sample’s task identity. It only needs to know which classes were introduced together during training.

The key is applying calibration before taking the maximum. Simply shifting or positively rescaling one final score would leave the ordering of all samples unchanged. TOOD can change that ordering because different inputs can draw their strongest response from different tasks. This allows calibration to improve the separation between ID and OOD scores.

We also consider an optional margin term. An ID sample often matches one task more strongly than the others, so the gap between its two strongest normalized responses provides another useful signal:

\[S_{\lambda}(x) = E_{(1)}^{\text{norm}}(x) + \lambda\left(E_{(1)}^{\text{norm}}(x) - E_{(2)}^{\text{norm}}(x)\right).\]

Here, \(E_{(1)}^{\text{norm}} \geq E_{(2)}^{\text{norm}}\) are the top two task energies. Setting \(\lambda=0\) recovers the core score; the main experiments use a fixed \(\lambda=0.5\). This optional term rewards a clear task preference, although its benefit varies across continual-learning methods.

Experiments

We evaluated TOOD on standard class-incremental benchmarks, measuring OOD detection after each task to track how it changes throughout learning.

AUROC, the area under the receiver operating characteristic curve, measures how well a detector separates ID and OOD samples across different thresholds. Higher values indicate better separation. We report average incremental AUROC by evaluating all tasks seen so far at each checkpoint, then averaging across checkpoints and OOD datasets. We also measure the false positive rate at 95% ID recall (FPR@95) and OOD forgetting, the drop in a task’s detection AUROC between the point when it was first learned and the final model.

Results

TOOD improves OOD detection over uncalibrated energy in most of the CIFAR settings we evaluated. The table below summarizes the results for both variants using average incremental AUROC (%), reported as mean ± standard deviation over three seeds and averaged over Near- and Far-OOD evaluation.

Dataset Continual Learner Energy TOOD Mean Shift TOOD Robust Anchor
CIFAR-10 iCaRL 65.9 ± 1.5 66.5 ± 1.5 66.4 ± 1.4
CIFAR-10 BiC 67.2 ± 1.0 71.6 ± 0.2 70.6 ± 0.3
CIFAR-10 DER 61.2 ± 0.7 69.3 ± 0.9 68.6 ± 0.9
CIFAR-10 WA 78.1 ± 2.3 78.3 ± 2.1 78.2 ± 2.4
CIFAR-10 LwF 72.2 ± 0.5 73.2 ± 0.3 73.0 ± 0.7
CIFAR-100 iCaRL 58.1 ± 1.3 58.4 ± 1.0 58.9 ± 1.0
CIFAR-100 BiC 65.6 ± 0.5 68.4 ± 1.3 68.8 ± 1.6
CIFAR-100 DER 64.6 ± 1.2 67.1 ± 0.5 66.1 ± 0.9
CIFAR-100 WA 71.9 ± 0.1 71.7 ± 0.2 71.5 ± 0.1
CIFAR-100 LwF 67.0 ± 1.6 66.7 ± 1.7 66.3 ± 1.6

Table 1: Average incremental AUROC (%) for energy and TOOD across the CIFAR learning sequences. Higher is better.

Across the full comparison with other OOD detectors, at least one TOOD variant ranks first or second in eight of ten CIFAR configurations. The strongest improvement over energy in this table is on DER with CIFAR-10, where Mean Shift raises AUROC from 61.2 to 69.3, a gain of 8.1 points. This supports our intuition that correcting task-wise score drift can recover useful OOD separation.

The benefit depends on the learner. WA changes very little on CIFAR-10, and both TOOD variants slightly reduce the mean AUROC for WA and LwF on CIFAR-100. Methods that already stabilize their output scores have less to gain from this correction.

We also evaluated a longer, 100-task ImageNet-1K sequence. Mean Shift improves BiC from 59.2 to 65.2 and DER from 70.2 to 71.4, while WA drops from 70.1 to 65.7. These results show that calibration can help over a longer sequence, but it does not improve every method. The ImageNet study uses a single seed, while the main CIFAR results include variation across three seeds.

Discussion

Our experiments show that OOD forgetting captures a weakness that classification accuracy alone can miss. A continual learner may still distinguish its known classes while becoming less reliable at identifying unfamiliar inputs. The confidence gap helps explain this behavior: useful information remains in the task-wise responses, but changes in their scale make a single global energy score less effective.

By calibrating each task’s energy using familiar examples, TOOD makes those responses comparable again. The approach is fully post-hoc and requires only one forward pass over a small calibration memory after each task. At inference, it computes one energy per task and combines the normalized scores, without changing the underlying model or its training algorithm.

TOOD’s effectiveness depends on having representative ID calibration examples and knowing how classes were grouped during training. It addresses score drift, but it cannot restore the feature-space separation lost through manifold crowding. Developing methods that preserve that separation as new classes arrive is an important direction for future work.

Our work highlights the need to evaluate both what a continual learner remembers and how well it recognizes what it has never learned. TOOD provides a practical step toward maintaining that ability as models acquire new knowledge.

For an introduction to the OOD detection problem, check the following post. You can also read our GROOD article for an approach based on gradient sensitivity. Feel free to check the TOOD paper for more technical details, ablation studies, and analysis (ElAraby et al., 2026).

Collaborators

References

  1. ElAraby, M., Nashed, S. B., & Paull, L. (2026). TOOD: Task-Aware Out-of-Distribution Score Calibration for Continual Learners. https://doi.org/10.48550/arXiv.2607.29592