Machines that 'learn how to learn'

Researchers have developed a new approach to machine learning that mimics humans’ ability to ‘learn how to learn’.

The method, called transformational machine learning (TML), out-performs current machine learning methods for drug design, which could accelerate the search for new disease treatments.

It has been developed by a team from Liverpool John Moores, Cambridge University and collaborators in Sweden, India and the Netherlands. It learns from multiple problems and improves performance while it learns.

Dr Ivan Olier, of the School of Computer Science and Mathematics at LJMU and first author on results reported in the Proceedings of the National Academy of Sciences, said: “We believe TML could accelerate the identification and production of new drugs by improving the machine learning systems which are used to identify them.”

Most types of machine learning (ML) use labelled examples, which are almost always represented using intrinsic features, such as the colour or shape of an object. The computer software then forms general rules that relate the features to the labels.

Learning from past lessons

“It’s sort of like teaching a child to identify different animals: this is a rabbit, this is a donkey and so on,” said Professor Ross King from Cambridge’s Department of Chemical Engineering and Biotechnology, who led the research. “If you teach a machine learning algorithm what a rabbit looks like, it will be able to tell whether an animal is or isn’t a rabbit. This is the way that most machine learning works – it deals with problems one at a time.”

However, this is not the way that human learning works: instead of dealing with a single issue at a time, humans get better at learning because we have learned things in the past.

“To develop TML, we applied this approach to machine learning, and developed a system that learns information from previous problems it has encountered in order to better learn new problems,” said Olier.

King added: “Where a typical ML system has to start from scratch when learning to identify a new type of animal - say a kitten - TML can use the similarity to existing animals: kittens are cute like rabbits, but don’t have long ears like rabbits and donkeys. This makes TML a much more powerful approach to machine learning.”

Potential applications

The researchers demonstrated the effectiveness of their idea on thousands of problems from across science and engineering. They say it shows particular promise in the area of drug discovery, where this approach speeds up the process by checking what other ML models say about a particular molecule. A typical ML approach will search for drug molecules of a particular shape, for example. TML instead uses the connection of the drugs to other drug discovery problems.

Results show the system is more effective than humans when choosing drugs.

Reference:
Ivan Olier et al. ‘Transformational Machine Learning: Learning How to Learn from Many Related Scientific Problems.’ Proceedings of the National Academy of Sciences (2021). DOI: 10.1073/pnas.2108013118