What do we restrict during polarisation? Electric field or magnetic field?

strength of interaction of electric field
The force exerted by the electric field on a charged particle is eEeE which is much stronger than the force by magnetic field ev×Bev×B. Magnetic force only get comparable to electric force when velocity of particle (usually induced by electric field) approaches light velocity.

Due to this reason when light interact with any material the electrons starts to oscillate in the direction of electric field and if a wire grid is setup perpendicular to the electric field the electron motion experience high damping and absorb the light appreciably (principle of wire grid polarizer).

Consideration of magnetic field become important in very intense laser fields when the electron motion become relativistic.