If you're looking for an introduction to computational mechanics, I also recommend a recent paper of mine, Discovering Non-Critical Organization: Statistical Mechanical, Information Theoretic, and Computational Views of Patterns in One-Dimensional Spin Systems. This paper has fairly lengthy review of information theory and computational mechanics. It's written at a slightly higher level than these lecture notes, but nevertheless should be quite readable.
For more on Shannon entropy convergence and how this leads to a variety of measures of structure and memory, you might want to see Regularities Unseen, Randomness Observed: Levels of Entropy Convergence, another recent paper.
The Lecture Notes
Other tutorials and pedagogical pieces can be found at http://www.santafe.edu/projects/CompMech/tutorials/CompMechTutorials.html