HMM Resources

• Books
• Software
• Web Pages
• Applications
• Papers
• Web Search Results

Books

I. L. MacDonald, With W. Zucchini, CRC Press 1997

Hidden Markov and Other Models for Discrete- Valued Time Series

Hand, Manilla, and Smyth ( MIT Press )

Principles of Data Mining

Bunke and Caelli (WSPC)

HIDDEN MARKOV MODELS Applications in Computer Vision

Manning and Schutze (B and N)

Foundations of Statistical Natural Language Processing
Page doesn't look promising.

Frederick Jelinek (amazon)

Statistical Methods for Speech Recognition (Language, Speech, and Communication)

Elliott, R. J., L. Aggoun, and J. B. Moore. (1995)

Hidden Markov Models: Estimation and Control

Salzberg, Searls and Kasif (Amazon),

Computational Methods in Molecular Biology

Meidanis and Setubal (unicamp.br)

Introduction to Computational Molecular Biology

Durbin, Eddy, Krogh, and Mitchison (Amazon)

Biological Sequence Analysis : Probabilistic Models of Proteins and Nucleic Acids

Baxevanis and Ouellette (Amazon)

Bioinformatics: A Practical Guide to the Analysis of Genes and Proteins, Second Edition

Software

Hidden Markov Model (HMM) Toolbox for Matlab

Kevin Murphy's  GPLd code. Separate code for outputs that are: discrete; vector Gaussian; vector mixture of Gaussians. It looks like EM for MLE, no priors. Straight forward code.

Zoubin Ghahramani's

Machine Learning Software. Two packages of MatLab scripts: One for discrete outputs, and one for vector Gaussian outputs. No Viterbi decoding. Simple no frills software.

HTK Hidden Markov Model Toolkit

-- speech recognition research toolkit from Cambridge. Restrictive license.

ghmm

GNU Hidden Markov Model Library by: Bernhard Knab, Bernd Wichern, Alexander Schliep, Achim Gädke, Thordis Linda "Disa" Thorarinsdottir, and Peter Pipenbacher at ZAIK

HMMER 2.2

Profile hidden Markov models for biological sequence analysis

SAM: Sequence Alignment and Modeling System

Widely used biological sequence software. No charge for non-comercial use.

Web Pages

Ten years of HMMs by Olivier Cappé

This list of references has been obtained by searching in a large bibliographic database for entries which featured the words Hidden and Markov (or the acronym HMM) in their title. Only the reference corresponding to review papers published after 1989 and before the end of 2000 have been retained

ISMB99 Tutorial Material: Making the most of your hidden Markov models

Slides from tutorial and pointers to papers.

Lloyd Allision's (Hidden) Markov Models page.

Contains a list of papers by the Australians on a bunch of applications

Applications

Climate Modeling

Bellone1, Hughes, and Guttorp A hidden Markov model for downscaling synoptic atmospheric patterns to precipitation amounts

Papers

• Ghahramani, Z. (2001) An Introduction to Hidden Markov Models and Bayesian Networks International Journal of Pattern Recognition and Artificial Intelligence 15(1):9-42.
  
• A tutorial on Hidden Markov Models and selected applications in speech recognition , L. Rabiner, 1989, Proc. IEEE 77(2):257--286.
  
• Markovian Models for Sequential Data , Y. Bengio, Neural Computing Surveys 2, 129--162, 1999.
• Notes from Dan Ellis' class on speech processing at Columbia (2002). I snagged the excellent notes for lecture 10 and lecture 11 .

• Bourlard's EPFL matlab lab manual on HMMs .

• A Modified Baum-Welch Algorithm for Hidden markov Models with Multiple Observation Spaces , by Baggenstoss P.M. in IEEE Transactions on Speech and Audio Processing(May 2001)
  • In this paper, an algorithm similar to the Baum-Welch algorith is derivedm for estimating the parameters of a hidden Markov model (HMM). The new algorithm allows theobservation PDF of each state to be defined and estimated using a different feature set. It is shown in the paper that estimating parameters in this manner is equivalent to maximizing the likelihood function for the standard parameterization of the HMM defined on the input data space. The processor becomes optimal if the state-dependent feature sets are sufficient statistics to distinguish each state individually from a common state.

• Hidden Markov Models in computational biology: Applications to protein modeling , by Krogh, M. Brown, I. S. Mian, K. Sjolander, and D. Haussler. In Journal of Molecular
  • Hidden Markov Models (HMMs) are applied to the problems of statistical modeling, database searching and multiple sequence alignment of protein families and protein domains. In each case the parameters of an HMM are estimated from a training set of unaligned sequences. After the HMM is built, it is used to obtain a multiple alignment of all the training sequences.
• Hidden Markov models for sequence analysis: Extension and analysis of the basic method , by Hughey, Richard and Krogh, Anders 1996. H. CABIOS 12(2):95--107.
  • The paper describes the mathematical extensions and heuristics that move the method in (Krogh et.al 94) from the theoretical to the practical. Then, they experimentally analyze the effectiveness of model regularization, dynamic model modification, and optimization strategies.
• Multiple Alignment Using Hidden Markov Models , by Sean R. Eddy (1995). In Proc. Third Int. Conf. Intelligent Systems for Molecular Biology, C. Rawlings et al., eds. AAAI Press, Menlo Park. pp. 114-120. PostScript; 7 pages.
  • Describes a simulated annealing algorithm for HMM training and a probabilistic suboptimal alignment algorithm. Compares HMM-based multiple alignment to CLUSTALW.
• Hidden Markov models of biological primary sequence information, by P. Baldi, Y. Chauvin, T. Hunkapiller, and M. A. McClure. In Proceedings of the National Academy of Sciences of the United States of America, 91(3):10591063, 1994.
• Jia Li, Amir Najmi, and Robert M. Gray,
  Image Classification by a Two Dimensional Hidden Markov Model ,
  IEEE Transactions on Signal Processing, February 2000.
• Climate Research: CR 15:1-12
• Beal, M. J. and Ghahramani, Z. (2002)
  The Variational Bayesian EM Algorithm for Incomplete Data: with Application to Scoring Graphical Model Structures
  Draft submitted for publication
  [pdf ]
• Krogh, "An introduction to hidden Markov models for biological sequences" ( paper )
• Eddy, "Profile hidden Markov models", Bioinformatics, 1998 ( paper )
• Bystroff, Thorsson, and Baker, "HMMSTR: a hidden markov model for local sequence-structure correlations in proteins", JMB, 2000 ( paper )
• http://citeseer.nj.nec.com/99549.html
  HMMs in text recognition
  
• Profile hidden Markov models . S.R. Eddy. Bioinformatics 14:755-763, 1998. A review of the profile HMM literature from 1996-1998. Abstract/reprints: [Bioinformatics Online] [PostScript] . [PDF] .

Unincorporated web search results

Naftali (Tali) Tishby

Look here

Sam Roweis,

Look here

Radford Neal,

Machine Learning at Toronto

Hinton's group

People

• Alexander Schliep
• Timmer
• Tishby
• Gray
• Smyth
• Sam Roweis
• Murphy
• Ephiam Yariv

Fraser's work

I am working on a few collaborative projects related to HMMs.

Here is the most recent source for a book that I am writing entitled "Hidden Markov Models and Dynamical Systems".