Spectral variability is one of the most limiting factors in hyperspectral unmixing, so it is important to further study the characteristics of spectral variability to improve the accuracy of unmixing. After conducting simulations under varying irradiation conditions, a linear mixed model combining endmember and band is proposed by introducing a band scaling factor to the endmember scaled spectrum. The total variation constraint is used to smooth the spatial distribution of both endmember and band scaling factors and then alternating iterative optimization is applied to solve the optimization problem. Experiments conducted with both simulated and real hyperspectral data sets indicate that the proposed algorithm is effective in hyperspectral unmixing and is superior to other state-of-the-art algorithms based on spectral variability.