Why aren information and entropy normally thought of together?
Most scientists recognize that information can be exchanged for entropy and vice versa. but they don’t consider that fact important. The reason is that in typical physical situations the number of bits of entropy is far larger than the number of bits of information that even the largest information processing systems can deal with. In other words, the scales involved are vastly different. This is because there are a large number of atoms in physical systems.
