2023-09-22
Breakthroughs in Efficient Deep Learning
AI-podden with Mert Pilanci, Professor at Stanford University.
We have great pleasure of hosting Mert Pilanci, Professor at Stanford University. Mert Pilanci shares with us his recent research breakthroughs in machine learning.
Mert and his group discovered that the training of shallow neural networks can be transformed into super-efficient algorithms that always achieve the best accuracy possible on the first shot.
This is opposed to state of the art methods where blind parameter search is conducted, requiring a huge amount of time, computation, and energy consumption. Mert Pilanci and his group at Stanford show that this transformation holds for popular and widely used network structures such as the transformer, MLP-mixer, and convolutional neural networks.
Ather and Mert discuss not only the limitations of the current methods, but also their theoretical capacity to approximate any given dataset.
Finally, Mert shares with us his thoughts on where he can see the newly developed tools applied, such as fine tuning large language models and other foundational models.
AI-podden with Mert Pilanci, Professor at Stanford University.