somebody pointed me to this page and I thought you might be interested in work one of the students here at Rochester, Ting Qian, has been doing on constant entropy rate (distribution of entropy throughout discourses) in Chinese. There is a paper under submission with written and spoken Chinese in it and a replication of Genzel and Charniak 2002 on English (using an approach that provides better control for potential confounds). The paper should be soon available at: http://www.tqian.org/pub.html, but I can forward you a copy of the paper, if you’re interested. not quite what you were talking about, but very related.

Transduction can go either way between sounds and characters, depending on whether you’re listening (speech recognition) or producing (synthesis). If you view the channel as carrying characters encoded as sounds, then you have p(characters) and p(sounds|characters) as source and channel model. If you view the channel as carrying sounds encoded as characters, you have p(sounds) and p(characters|sounds) respectively.

The issue of “entropy of language X” is about just the source, p(characters) or p(sounds) depending on whether you’re modeling the language as sequences of characters or sequences of sounds.

The truly interesting case is to take the source as “meaning” and the channel as carrying either characters or sounds. We don’t quite have a handle on that yet, I’m afraid.

It’d be interesting if the various versions of Chinese (e.g. traditional vs. simplified vs. ancient — my knowledge of Chinese is very limited) changed the encoding rates. It’d be the clearest example of the point Mark Liberman was trying to make in the Language Log post.