To meet the requirements of full deformation and strain component measurement by using electronic speckle pattern interferometry (ESPI), which is necessary for the experimental research of the unrepeatable deformation. Three-dimensional ESPI system was introduced in this paper. In this ESPI system, tri-wavelength laser lights were used to independently sense two in-plane and out-of-plane deformation components, respectively. The three dimensional (3D) deformation information encoded by color speckle patterns was recorded by a 3CCD color camera. The color speckle interferograms were split into their red, green, and blue components, and the 3D deformation data were quantitatively determined by utilizing the phase-shifting method and were furtherly denoised by phase filtering. Full strains were calculated by applying numerical differential operation on the unwrapped phase images. Experimental performances on the test sample demonstrated the data processing and verified the validation of the proposed method.