Investigating Random Variations of the Forward-Forward Algorithm for Training Neural Networks

The Forward-forward (FF) algorithm is a new method for training neural networks, proposed as an alternative to the traditional Backpropagation (BP) algorithm by Hinton. The FF algorithm replaces the backward computations in the learning process with another forward pass. Each layer has an objective function, which aims to be high for positive data and low for negative ones. This paper presents a preliminary investigation into variations of the FF algorithm, such as incorporating a local Backpropagation to create a hybrid network that robustly converges while preserving the ability to avoid backward computations when needed, for example, in non-differentiable areas of the network. Additionally, a pseudo-random logic for selecting trainable stacks of layers at each epoch is proposed to speed up the learning process.