The Geometric Root of Catastrophic Forgetting: How to Update Large Language Models Without Losing Existing Capabilities

    A mature large language model can noticeably regress on math and code after being fine-tuned for a new task such as medical consultation or low-resource translation. This article explains catastrophic forgetting and its geometric root cause: because each gradient update takes the form of a global weight increment (an outer product), it perturbs the outputs of all inputs—not just the current sample—so old capabilities are overwritten whenever new and old inputs are not orthogonal. It shows why switching optimizers cannot fix the problem, then surveys the mainstream mitigations: regularization-based constraints, rehearsal/replay, parameter isolation, and data-mixing and training-pipeline engineering. Finally, it describes how to fold this "memory surgery" into the iteration workflow with regression baselines, versioned data mixes, and careful management of intermediate artifacts.

    Introduction: A Familiar and Awkward Situation

    Imagine you already have a mature large language model that performs well on general question answering, mathematical reasoning, and code generation. Now the business team asks for a new capability: teaching the model the phrasing of medical consultation, or adding support for translating a low-resource language. So the team starts incremental training (continued pretraining or instruction tuning). The training curves look healthy—loss on the new task drops steadily, and evaluation scores climb.

    But when you rerun the original benchmarks, you are surprised to find that math and code ability have clearly regressed.

    It is like hiring a tutor to help your child with English. The tutoring works well, but when you test again, the child’s math has gotten worse. This “learn the new, forget the old” phenomenon is nothing new in deep learning. It has a specific name: catastrophic forgetting.

    1. What Is Catastrophic Forgetting

    Catastrophic forgetting refers to a neural network uncontrollably overwriting capabilities it had already acquired while learning a new task—even when the new task has no semantic relationship to the old one. In other words, a model’s parameters form a highly coupled whole. When you update part of them to adapt to a new distribution, you often damage the weights that encode the old capabilities along the way.

    The phenomenon can be traced back to cognitive science and connectionism research in the late 1980s. Researchers noticed back then that artificial neural networks, unlike the biological brain, rapidly lose early knowledge when learning multiple tasks sequentially. For a long time, however, it remained more of an academic observation and was not taken seriously in engineering practice.

    In today’s era of large models, its importance is entirely different. Training a small model used to take hours to days; if you forgot something, you retrained, and the cost was acceptable. Now, pretraining a foundation model with tens or even hundreds of billions of parameters from scratch can consume millions of dollars in compute and take weeks. If every subsequent round of “finishing work” erodes the base capabilities, the economic model of “pretrain once, fine-tune many times” falls apart. This is why continual learning and model editing have once again become hot topics in both research and engineering.

    A research team has offered a rather illuminating explanation from a geometric perspective, and understanding this explanation is the prerequisite for finding a solution.

    2. The Culprit: The “Collateral Damage” Mechanism of Gradient Updates

    To understand why forgetting happens, first look at what a neural network actually changes at each step. You can view the weight matrix W of a fully connected layer (or one of the projections in attention) as a linear mapping: it takes an input vector x and produces y = W x. Training the whole network is essentially the process of continuously adjusting this series of weight matrices so that the model’s outputs match expectations across various inputs.

    The problem is that the way gradient descent adjusts weights has an inherent “globally rippling” quality.

    2.1 The Update Is an Outer Product: It Acts on All Inputs

    A single gradient update on one training sample can usually be written as an outer product:

    1. ΔW = -η · δ · xᵀ

    Here η is the learning rate, x is the current input vector, δ is the gradient of the loss with respect to the output (the error signal), and xᵀ is the transpose of x. This increment is a matrix, but its rank is at most 1, and its direction is determined jointly by the error δ and the input x.

    The key point is this: once ΔW is added to W, it simultaneously changes the model’s output for all inputs—not just the sample you are currently training on. For any input x', the new output is (W + ΔW) x'.

    2.2 A Mathematical Certainty: Non-Orthogonal Means Forgetting

    For an old input x_old, its output after the update becomes:

    1. (W + ΔW) x_old = W x_old - η · δ · (x_newᵀ · x_old)

    The original output was W x_old; now there is an extra term -η · δ · (x_newᵀ · x_old). Here x_newᵀ · x_old is the inner product of the new and old inputs. Only when this inner product is zero—that is, when the two vectors are completely orthogonal—does the old output remain unchanged. As long as the new and old samples share any directional correlation, the update is bound to disturb the old sample’s output, thereby eroding the old capability.

    In reality, data can hardly ever be pairwise orthogonal: word vectors in natural language and pixel distributions in images all share a great deal of statistical structure and semantic correlation. This means “collateral damage” is almost unavoidable.

    2.3 A Toy Experiment

    To make this concrete, consider a minimal classification experiment. Suppose the inputs are coordinate points on a 2D plane, and the task is to sort points into 6 classes across two stages:

    1. # Stage 1: pre-train on the old classes (0-3)
    2. model = LinearClassifier(input_dim=2, num_classes=6)
    3. train(model, data_of_classes=[0, 1, 2, 3], epochs=100)
    4. # Stage 2: fine-tune only on the new classes (4, 5)
    5. finetune(model, data_of_classes=[4, 5], epochs=50)
    6. # Evaluation: old classes drop noticeably, even though they never appeared in Stage 2
    7. eval_accuracy(model, classes=[0, 1, 2, 3]) # significant drop
    8. eval_accuracy(model, classes=[4, 5]) # normal

    After training, you will observe a typical seesaw effect: classes 4 and 5 are correctly separated, but the decision boundaries for classes 0–3 are noticeably distorted, and their accuracy drops sharply. This code is deliberately kept minimal precisely to show that forgetting does not come from the model being too large or trained too long, but from a structural flaw in the update mechanism itself.

    3. Why Switching Optimizers Cannot Save You

    A common intuition is: would a “smarter” optimizer fix this? For example, switching from plain stochastic gradient descent to a method with momentum or adaptive learning rates, or even a more aggressive update scheme.

    The answer is that this treats the symptom, not the root cause. In the derivation above, what actually creates the “collateral damage” is the structural fact that “a weight increment acts on all inputs.” What various optimizers change is only “how to compute and scale this increment”—momentum, adaptive step sizes, second-order approximations, and the like all affect the specific values and direction of δ and η, but they cannot turn ΔW into a local operation that only affects the current sample. Once the increment is formed, its effect on old inputs is written into the mathematical form. Therefore, no matter which optimizer you switch to, as long as the update method is still “a global increment applied to all parameters,” the root cause of catastrophic forgetting remains.

    Here is an analogy. Suppose a warehouse manager is instructed to rearrange one particular shelf. If, instead of adjusting only the local lighting near that shelf, he goes and rewires the entire warehouse’s lighting system, then staff in other areas will be thrown off by the sudden change in light—even though the shelves they are responsible for were never touched. Switching optimizers is more like changing the wiring technique from “rough wiring” to “fine wiring,” but what is still being modified is the circuitry of the whole building, not the local area.

    4. Mainstream Approaches to Mitigating Catastrophic Forgetting

    Once you understand the cause, the direction for mitigation becomes clear. Continual learning methods in industry can be roughly grouped into a few categories, each with its own trade-offs.

    4.1 Regularization-Based Constraints

    The idea is to add a “penalty” term to the loss during fine-tuning, constraining important parameters so they do not drift too far from their pretraining values. A typical approach first estimates the importance of each parameter for the old task (for example, using the Fisher information matrix), then applies stronger constraints to the more important parameters.

    1. loss = loss_new + λ * sum(F_i * (theta_i - theta_i_old) ** 2)
    2. # theta: current parameters, theta_old: pretraining parameters
    3. # F_i: parameter importance weights, λ: constraint strength

    The advantage is that it is simple to implement and does not require retaining old data. The drawback is that when the new task differs greatly from the old one, or when the number of sequential tasks is large, the constraints struggle to cover everything.

    4.2 Rehearsal / Replay

    The idea is “reviewing the old to learn the new”: when training on the new task, mix in a small portion of samples from the old task so the model keeps reviewing. This tends to be quite robust. The cost is that you need to store or replay old data and deal with data privacy and storage overhead. In large-model scenarios, a common practice is to mix “general corpus + new-domain corpus” in a certain ratio, using the replay ratio to balance forgetting and learning.

    4.3 Parameter Isolation

    The idea is to assign different subsets of parameters or subnetworks to different tasks, carrying new capabilities with newly added “slots” while trying not to touch the existing backbone. This kind of method isolates forgetting relatively well, but it brings problems of parameter growth and how to route during inference.

    4.4 Data Mixing and Training-Pipeline Engineering

    Often, forgetting is amplified because the data mix during fine-tuning is too extreme (all new domain, almost no general data). By controlling the mixing ratio of new and old data, using a smaller learning rate, limiting the number of training steps, and freezing more of the lower-layer parameters (fine-tuning only the upper layers), you can significantly mitigate regression. These engineering measures often deliver results faster than changing the algorithm.

    5. Engineering Practice: Folding “Memory Surgery” into the Iteration Workflow

    For engineering teams, the forgetting problem ultimately has to land on a reusable process, not a one-off trick. A few practical recommendations:

    • Establish regression evaluation baselines. Run a benchmark suite covering old capabilities before and after every fine-tuning round, and make the “old-capability retention rate” a hard metric in the release gate.
    • Make data mixing versionable. Manage the mixing ratio of general corpus to domain corpus as a configurable item, making it easy to reproduce and tune.
    • Preserve key intermediate artifacts. Pretraining weights, old-task evaluation sets, parameter importance estimates, and the like all need to be properly archived.
    • Leverage platform capabilities. In scenarios where models are iterated continuously, you can rely on the general capabilities that cloud platforms provide—training scheduling, data management, version management, and monitoring and alerting—to solidify the workflow above. For example, general capabilities such as training, data management, and monitoring offered by platforms like Baidu AI Cloud can help reduce the risk of human omission and turn “memory surgery” into a trackable, reproducible standard step.

    Summary

    The essence of catastrophic forgetting lies neither in the data nor in the training duration, but in one geometric fact: a single global gradient increment ripples across all inputs. As long as the new and old tasks are not orthogonal in representation space, the old capabilities will be rewritten along the way. Switching optimizers only changes the form of the increment; it cannot eliminate this structural “collateral damage.” The real solution is to build a combination around continual learning: use regularization to constrain important parameters, use replay to let the model review the old, use parameter isolation to leave room for new capabilities, and then use rigorous evaluation baselines and data-mixing engineering to make the results tangible. Only by treating “memory surgery” as a standard step in the model iteration workflow can a model hold on to the base capabilities that should not be forgotten while learning new skills.