Is There a Quick Introduction to Information Theory Somewhere?
. Imagine that we are in communication and that we have agreed on an alphabet. Before I send you a bunch of characters, you are uncertain (Hbefore) as to what I’m about to send. After you receive a character, your uncertainty goes down (to Hafter). Hafter is never zero because of noise in the communication system. Your decrease in uncertainty is the information (R) that you gain. Since Hbefore and Hafter are state functions, this makes R a function of state. It allows you to lose information (it’s called forgetting). You can put information into a computer and then remove it in a cycle. Many of the statements in the early literature assumed a noiseless channel, so the uncertainty after receipt is zero (Hafter=0). This leads to the SPECIAL CASE where R = Hbefore. But Hbefore is NOT “the uncertainty”, it is the uncertainty of the receiver BEFORE RECEIVING THE MESSAGE. A way to see this is to work out the information in a bunch of DNA binding sites. Definition of “binding”: many proteins
. Imagine that we are in communication and that we have agreed on an alphabet. Before I send you a bunch of characters, you are uncertain (Hbefore) as to what I’m about to send. After you receive a character, your uncertainty goes down (to Hafter). Hafter is never zero because of noise in the communication system. Your decrease in uncertainty is the information (R) that you gain. Since Hbefore and Hafter are state functions, this makes R a function of state. It allows you to lose information (it’s called forgetting). You can put information into a computer and then remove it in a cycle. Many of the statements in the early literature assumed a noiseless channel, so the uncertainty after receipt is zero (Hafter=0). This leads to the SPECIAL CASE where R = Hbefore. But Hbefore is NOT “the uncertainty”, it is the uncertainty of the receiver BEFORE RECEIVING THE MESSAGE. A way to see this is to work out the information in a bunch of DNA binding sites. Definition of “binding”: many proteins
