Rules discovery from data // deep distilling
Reconstruct the code from the data — great idea, good luck with that)
“Artificial intelligence must distill explicit rules from experimental data” —while it may sound great, the problem is not as simple as it seems (the enormous space of possibilities).
To automate the discovery of new scientific and engineering principles, artificial intelligence must distill explicit rules from experimental data. This has proven difficult because existing methods typically search through the enormous space of possible functions. Here we introduce deep distilling, a machine learning method that does not perform searches but instead learns from data using symbolic essence neural networks and then losslessly condenses the network parameters into a concise algorithm written in computer code. This distilled code, which can contain loops and nested logic, is equivalent to the neural network but is human-comprehensible and orders-of-magnitude more compact. On arithmetic, vision and optimization tasks, the distilled code is capable of out-of-distribution systematic generalization to solve cases orders-of-magnitude larger and more complex than the training data. The distilled algorithms can sometimes outperform human-designed algorithms, demonstrating that deep distilling is able to discover generalizable principles complementary to human expertise.
“Neurosymbolic approaches could potentially allow for more human-like machine learning capabilities,” wrote the team.
