Duration: 0:00

4 views

1

Long ago, in the world of mathematics, information was nothing more than an abstract concept, lacking any concrete form. Yet, seekers of numerical truth strove to grasp it. The discovery of logarithms marked one of the first steps. Napier and Euler uncovered logarithms as the inverse of exponentiation, and Boltzmann applied them to entropy. Entropy—once a measure of disorder—became the very foundation of information. Meanwhile, the theory of limits emerged through Newton and Leibniz in the form of calculus, with Cauchy refining the rigorous definition of convergence. In the realm of probability, Bernoulli established the law of large numbers, Markov developed stochastic processes, and Kolmogorov laid the groundwork with measure theory, setting the stage for describing the flow of information. When these streams of thought converged, Shannon's theory was born. He measured information as entropy and mathematically defined the flow of information through a communication channel. Using the method of summation over limits, he uncovered the fundamental capacity of infinitely long codes. This was a revolutionary insight: by accumulating the behavior of individual symbols, he revealed the limits of communication in the realm of infinity. Thus, Shannon’s theory transformed information into something computable, laying the foundation for modern digital communication.