Based on the calculation of the energy flux around the interface, the physical mechanisms of inverse Goos-Hänchen (GH) shifts on the surfaces of noble metal, metamaterial and negatively refractive photonic crystals were investigated.The results show that, although the inverse GH shifts may occur on the surfaces of three kinds of material in a similar manner, the physical mechanisms corresponding to those phenomena are completely distinct.The inverse GH shift of noble metal is based on the fact that the incidence of TM polarized light induces the oscillations of delocalized electrons at the surface of noble metal, which results in a surface plasmon wave.The GH shift of metamaterial is due to the mutation of the phase of reflectivity and the reason for negatively refractive photonic crystal is the synthetic action of effective negative refraction and periodicity of surface, i.e., the excitation of backward surface wave.