Profile cover photo
Profile photo
Zbigniew Galar
1,773 followers -
Data analyst and risk management specialist - publisher of toster.pandory.pl
Data analyst and risk management specialist - publisher of toster.pandory.pl

1,773 followers
About
Posts

Post has shared content
"Naftali Tishby of the Hebrew University of Jerusalem believes the training processes in neural networks illustrate a branch of information theory that he helped develop two decades ago. He coined the term 'information bottleneck' to describe the most efficient way that a system can find relationships between only the pieces of data that matter for a particular task and treat everything else within the sample as irrelevant noise."

"We believe [Tishby's] ideas are substantially correct, but there are a few technical details that have to be worked out. The fact that we converged to similar ideas is remarkable because we started from completely independent premises."

"Soatto and his UCLA colleague Alessandro Achille used ideas from the information bottleneck to develop training optimizations that help smaller networks tune out noise."
Add a comment...

Post has shared content
The Matrix Calculus You Need For Deep Learning

This paper is an attempt to explain all the matrix calculus you need in order to understand the training of deep neural networks. We assume no math knowledge beyond what you learned in calculus 1, and provide links to help you refresh the necessary math where needed. Note that you do not need to understand this material before you start learning to train and use deep learning in practice; rather, this material is for those who are already familiar with the basics of neural networks, and wish to deepen their understanding of the underlying math. Don't worry if you get stuck at some point along the way---just go back and reread the previous section, and try writing down and working through some examples. And if you're still stuck, we're happy to answer your questions in the Theory category at forums.fast.ai. Note: There is a reference section at the end of the paper summarizing all the key matrix calculus rules and terminology discussed here.
Add a comment...

Post has attachment
Add a comment...

Post has shared content
Add a comment...

Post has shared content
Add a comment...

Post has shared content
A scary real-life tale for our times
Add a comment...

Post has shared content
Milan’s Vertical Forest Skyscraper Revealed In Stunning Drone Footage
Add a comment...

Post has shared content
This scatter plot visualizes the gender gap in school reading and maths scores versus socioeconomic status
https://www.nytimes.com/interactive/2018/06/13/upshot/boys-girls-math-reading-tests.html
Photo
Add a comment...

Post has shared content
The world is complex not complicated - our first article in polish&english.
Add a comment...

Post has shared content
Add a comment...
Wait while more posts are being loaded