AI Method Automates Discovery of Local Learning Rules

A new method aims to replace backpropagation with automatically discovered local learning rules

Researchers affiliated with Aalto University and the University of Trieste have proposed a method for automatically discovering symbolic local learning rules that could, in principle, replace backpropagation. According to the source at arXiv:2611.12248, the method is general enough to apply to any deep learning architecture and task. The claim matters because backpropagation remains the default learning procedure in most neural networks, even as questions about how information should be shared across layers are debated.

The source describes a general method that automatically discovers local loss functions specific to each layer. The authors say the method builds a dictionary of monomials of local tensor variables, then regresses over coefficients to create layerwise polynomial losses. According to the paper, the gradients of those losses together minimize a target global loss.

The work comes from researchers who also discuss the plausibility of backpropagation as a model of biological learning. The source says the method was tested on classification, reconstruction, and self-supervised learning tasks with small multilayer networks and minimal datasets. The authors report that the discovered rules induced non-trivial interactions between layers and led to significant reductions in global losses.

How the automatic discovery method works

The method begins with a global loss function and a network. For each layer, it defines a neighborhood of local variables, including inputs, weights, outputs, and variables from nearby downstream layers. It then constructs scalar functions using trace monomials, which are formed by taking up to a limited number of tensors and contracting indices. According to the source, the authors used a physics-inspired notation to generate these monomials.

Each layer receives a local loss defined as a weighted sum of trace monomials. The weights are the rule coefficients. During training, updates use gradients from the local loss, and the global loss is checked. The authors describe an outer loop that updates rule coefficients and repeats episodes until the global loss is sufficiently low. The source notes that normalization, time horizon, early stopping, optimizers, and sparse priors were important for stable training.

The source says the method automatically discovers local loss functions by constructing a dictionary of monomials and regressing over coefficients. The gradients of these losses are then used to update parameters. The authors describe a process that turns local learning rule discovery into a systematic, learnable procedure.

What the method was tested on

The source reports tests on classification, reconstruction, and self-supervised learning. The experiments used small multilayer networks and minimal datasets. According to the authors, the discovered rules produced non-trivial interactions between layers and significant reductions in global losses.

Figure 3 shows results from a classification task on the Sliced MNIST held-out split, including how the rule coefficients evolved and how the learned rules compared with backpropagation. Figure 5 shows a self-supervised learning test on the MNIST held-out split, comparing the learned rules with backpropagation and with backpropagation when layers were frozen.

The source also describes tests of degree and order, and the method selects sparse combinations of monomials. The authors say the selected rules were more interpretable than MLP-based alternatives.

What the results mean and what they do not

The authors argue that local learning rules may offer practical advantages when supported by suitable hardware. They mention reduced cross-layer information transfer, parallel layer updates, and possible energy savings. They also suggest implications for energy-efficient deep learning and biologically plausible learning mechanisms.

But the source presents the method as a general approach in principle. The experiments were small-scale. The authors acknowledge that meta-optimization is numerically delicate. The report does not claim immediate replacement of backpropagation.

How to explore the paper with Grafy

For researchers, Grafy Reader can be used to read, search, and question the paper. Grafy Chat can be used to discuss the method, local loss functions, or symbolic rules. This fits creators, developers, and researchers who use AI tools.

Frequently asked questions

Does the method teach a smaller model to replace backpropagation?

No. The source describes a method that automatically discovers symbolic local learning rules. It does not report that a smaller model is taught to teach another model. The paper's focus is on finding local loss functions that can replace backpropagation in principle.

What tasks did the automatically discovered rules test?

The method was tested on classification, reconstruction, and self-supervised learning tasks. According to the source, the tests used small multilayer networks and minimal datasets, and the discovered rules produced non-trivial layer interactions and reduced global losses. The authors reported significant reductions in the corresponding global losses.

Did the method find interpretable local rules?

Yes, according to the source. The method reportedly selected sparse combinations of monomials, leading to local rules that were more interpretable than alternatives from multi-layer perceptron-based methods. The authors framed it as more interpretable than MLP-based alternatives in the paper.

---

Source: Grafy Research Lab — A method to automatically discover symbolic local learning rules