We know that fluorine is a strongly electronegative element. So, it has tendency to attract electron density towards itself. Thus, in O2F2 , fluorine attracts electron densities towards itself resulting in smaller O-O bond.
In H2O2, oxygen is not as strong an electronegative element as fluorine is. So, even though it does attract electron density towards itself but not to the extent and intensity of fluorine. So, H-H bond is not as affected as O-O bond in O2F2.