Symbolic Regression: Doing What LLMs cannot - Deriving Arbitrary Mathematical Relations!
14Description:
Do you believe that Large Language Models (LLMs) are good at everything? Well, nearly all. They cannot derive mathematical relations well. This is where Symbolic Regression (SR) can help - this field aims to apply machine learning to generate mathematical expressions for arbitrary input-output relations.
The expressions obtained from SR come in a compact and human-readable form that has fewer parameters than black-box models. These expressions allow for useful scientific insights by mere inspection. This property has led SR to be gradually recognized as a first-class algorithm in various scientific fields, including Physics, Material Sciences and Knowledge in recent years.
We will first cover what SR is and highlight some of the current methods. Then, we show that the current SR methods tend to neglect a large portion of the search space of 'short' expressions in favor of longer expressions which are less explainable. In contrast to most current SR methods, our work focuses on prioritizing obtaining short expression length, and hence explainability and interpretability, while still maintaining prediction performance.
Speaker Info:
Kei Sen is a PhD student under the lab group led by Prof. Mehul Motani. His primary interest lies in improving Symbolic Regression algorithms and in applying Symbolic Regression to real-world problems. He believes that the high intrinsic explainability and interpretability of Symbolic Regression will be the future for industries like Healthcare and Finance, where understanding the decision-making process of models is paramount.
~~~~~~~~~~~~~~~~~
DistilSR Paper: https://dl.acm.org/doi/abs/10.1145/3583133.3590736
GPLearn (a basic template for Genetic Programming): https://gplearn.readthedocs.io/en/stable/intro.html
GPLearn Examples (focus on Regression): https://gplearn.readthedocs.io/en/stable/examples.html
Universal Approximation Theorem and Neural Networks: https://towardsai.net/p/deep-learning/understanding-the-universal-approximation-theorem
Shapley Values: https://arxiv.org/abs/2202.05594
~~~~~~~~~~~~~~~~~
0:00 Symbolic Regression does what LLMs cannot - identify mathematical relations
1:00 Introduction to Symbolic Regression
15:33 Symbolic Regression can find difficult mathematical relationships!
25:18 Explanability is SR’s key strength
39:23 Symbolic Regression on the Iris Dataset
47:06 How is Symbolic Regression performed?
49:30 K-Expression
59:17 Start with most expressive equation and filter out unnecessary terms
1:01:15 Results of DistilSR
1:13:34 Discussion
~~~~~~~~~~~~~~~~~~
AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.
Discord: https://discord.gg/bzp87AHJy5
LinkedIn: https://www.linkedin.com/in/chong-min-tan-94652288/
Online AI blog: https://delvingintotech.wordpress.com/
Twitter: https://twitter.com/johntanchongmin
Try out my games here: https://simmer.io/@chongmin