Fringe pattern denoising is of prime importance in phase demodulation, especially for a single fringe pattern in speckle interferometry. A fringe speckle noise removal algorithm using the Kalman filter is proposed. The conventional linear Kalman filter is implemented with a fixed value of the process and measurement noise covariances; the adaptive Kalman filter is implemented with the process, and measurement noise covariances are estimated in an adaptive manner. The fringe denoising is performed sequentially in a rowwise and columnwise manner. The simulated and experimental results demonstrate the practical applicability of the proposed method, and its performance is appraised using metrics such as the quality index, peak signal-to-noise-ratio, and speckle index.