Hidden Markov models (HMMs) are widely used models for sequential data. As with other probabilistic models, they require the specification of local conditional probability distributions, which can be too difficult and error-prone, especially when data are scarce or costly to acquire. The imprecise HMM (iHMM) generalizes HMMs by allowing the quantification to be done by sets of, instead of single, probability distributions. iHMMs have the ability to suspend judgment when there is not enough statistical evidence, and can serve as a sensitivity analysis tool for standard non-stationary HMMs. In this paper, we formalize iHMMs and develop efficient inference algorithms to address standard HMM usage such as the computation of likelihoods and most probable explanations. Experiments with real data show that iHMMs produce more reliable inferences without compromising efficiency.