Researchers at the Faculty of Computer Science at the National Research University Higher School of Economics have improved the method of training generative stream neural networks to work with unstructured tasks. This will help to search for new drugs more effectively.

 The results of the work were presented at one of the leading conferences on machine learning, ICLR 2025. The text of the work is available in the Arxiv.org repository. Generative flow networks (GFlowNets) are a special type of machine learning algorithm that builds complex objects step by step. Scientists use them to search for proteins, drugs, and optimize transportation systems.

In order for GFlowNets to find such complex structures, researchers explain to them the properties of the object they would like to obtain. The closer the neural network's solution is to the specified properties, the higher the reward it will receive, according to Naked Science. The network strives to solve the problem in order to receive the maximum prize. It does not pay attention to the data, only to the reward. It is calculated using an equation called a reward function.

The process of searching for a complex object can be compared to assembling a model from Lego blocks. You add details sequentially until you get something whole, with each object having its own predetermined value. An assembled model of a plant, for example, may be valued higher than a model of an animal. Unlike other machine learning methods, which will strive to obtain a plant at any cost, GFlowNets will collect many different objects, but plants more often than animals: it is more profitable.

With this type of search, GFlowNets use two probabilistic models that work in tandem: forward and backward. The forward model is like a foreman who decides where to go next and predicts the probability of the next state, while the backward model is like an expert analyst who determines what the previous step was. It is important that these flows are balanced, but this is very difficult to achieve. First, it requires a lot of computing power, and second, the backward model is not flexible enough: researchers usually prohibit it from changing during the search process and peeking at the actions of the forward model.

Scientists at the National Research University Higher School of Economics have found a way to optimize the backward model using the Trajectory Likelihood Maximization (TLM) method. They refined the algorithms of the inverse model so that it could constantly check against the actions of the forward model.

After TLM was implemented, the reward function describing the success of solving a problem for the inverse model became more complex. However, despite the increased complexity, the entire search system became faster and more efficient.

"Our method scans the space of possible solutions much faster and finds more high-quality options. This approach generally brings generative models closer to reinforcement learning methods," says Nikita Morozov, junior researcher at the Center for Deep Learning and Bayesian Methods at the Institute of Artificial Intelligence and Digital Sciences, Faculty of Computer Science, National Research University Higher School of Economics.

The authors of the study are confident that their work will help specialists who use GFlowNets in various fields, such as the search for new drug compounds, the development of materials with specified properties, and the fine-tuning of large language models. Thanks to the ability of such networks to effectively explore a vast space of solutions and quickly find the most optimal options, the load on computing power can be significantly reduced.