Category Archives: Information Theory

Information Theory, Machine Learning & Cross-Entropy Loss

information theory - machine learning

¬†What is information theory? How is information theory related to machine learning? These are some of the questions that we will answer in this blog post. Information theory is the study of how much information is present in the signals or data we receive from our environment. AI / Machine learning (ML) is about extracting interesting representations/information from data which are then used for building the models. Thus, information theory fundamentals are key to processing information while building machine learning models. In this blog post, we will provide examples of information theory concepts and entropy concepts so that you can better understand them. We will also discuss how concepts of …

Continue reading

Posted in Information Theory, Machine Learning. Tagged with , .