Continual-learning rules shape representational drift

Lifelong learning requires acquiring new knowledge without erasing the old. Yet neural population codes for familiar stimuli and behaviors change over days and weeks. This coexistence of stable memory and changing internal codes may depend on how a learning system prevents forgetting. We therefore tested whether different continual-learning mechanisms produce distinct patterns of representational drift. We trained convolutional networks on sequential image classification tasks and recurrent networks on sequences of cognitive tasks, tracking fixed probe representations across learning. Experience replay preserved earlier tasks in both architectures while representations drifted progressively with the number of intervening tasks. Drift was structured: later visual-processing stages and recurrent units' temporal tuning were especially labile, whereas coarse class organization and task-relevant temporal structure persisted. In contrast, algorithm that strongly anchored weights nearly froze representations. Directly anchoring an old representation during replay likewise suppressed drift and impaired acquisition of subsequent tasks. Together, these results link representational drift to the stability--plasticity trade-off: its magnitude is shaped by the mechanism that protects old knowledge, and suppressing it can restrict future learning. Drift may therefore provide an observable signature of the constraints that enable continual learning in brains and machines.

Source: arXiv:2608.16141v1 - http://arxiv.org/abs/2608.16141v1 PDF: https://arxiv.org/pdf/2608.16141v1 Original Link: http://arxiv.org/abs/2608.16141v1