We developed an optical cryostat with a sample-rotation unit for polarization-sensitive measurement in terahertz (THz) and infrared (IR) ranges. The cryostat, in combination with two metal-grid polarizers, provides full control of mutual orientation of the sample’s crystallographic axes and the light polarization plane. Importantly, this control is realized in-situ, i.e., during the sample cooling–heating cycle. To demonstrate the abilities of the developed cryostat, we used it in combination with a laboratory-made THz time-domain spectrometer, for polarization-sensitive measurements of an orthoferrite (YFeO₃) in the range of 5 to 50  cm^−  1. These measurements revealed strong angular dependence of the sample transmission. The developed cryostat is capable for solving numerous demanding problems of THz and IR spectroscopy in condensed matter physics and materials science, biophysics, chemical, and pharmaceutical sciences.