Topic: Information Theory and Coding
Consider identical four, 3-faced dice. When the dice are rolled, the faces of the dice appear with probabilities given below. Which distribution has the maximum entropy?
- A. (1/2, 1/4, 1/4)
- B. (1/3, 1/3, 1/3)
- C. (1/6, 2/3, 1/6)
- D. (1/4, 1/6, 7/12)
Answer: B. (1/3, 1/3, 1/3)
Explanation: The entropy of a discrete probability distribution is maximized when all outcomes are equally likely (a uniform distribution). For a 3-faced die, the uniform distribution is (1/3, 1/3, 1/3). Therefore, this distribution yields the maximum entropy.