What made Lord Cnterville's family to abandon Canterville Chase?

Lord Canterville and his family did not care to live at Canterville Chase ever since the Lord's grandaunt, the Dowager Duchess of Bolton was frightened into a fit from which she never really recovered. The sight of two skeleton hands being placed on her shoulders as she was dressing for dinner had thrown her into that fit. The ghost had been seen by several living members of his family, including the rector of the parish, Rev. Augustus Dampier who was a Fellow of King's College, Cambridge. Post the unfortunate accident to the Duchess, all the younger servants had refused to stay with them and Lady Canterville also complained of a lack of sleep at night due to the mysterious noises that came from the corridor and library.