Why did the villagers forget their panic and fear of the invisible Man?

It was the festival of Whitsuntide and hence, the strange story of the invisible man was quickly forgotten once he had passed out of their immediate concern for the villagers.