Finger vein recognition is widely used in various fields due to its convenience, high safety, and stability of the finger vein. Because of the uniqueness of biometric characteristics, biometric data protection and privacy raise significant concerns. In addition, studies have shown that user information can be stolen by illegal operations and stored in the system. Once this happens, the user’s information will be leaked everlastingly, and the personal finger vein traits will no longer be unique. To solve this problem, a new cancellable encryption template was proposed for finger vein images, which applies double random phase encoding technology to finger vein image partial mutual encryption. In addition, an adaptive finger vein pattern extraction algorithm based on maximum curvature was proposed to ensure recognition performance while maintaining the security of the template of the user. The SDUMLA and FV_USM finger vein datasets were used for testing to verify the effectiveness of the method. The experimental results show that the performance of our proposed encryption satisfies the criteria of irreversibility, revocability, and unlinkability of biometric template protection and can effectively defend against potential attacks.