What does the expression "Pulled the few hairs on his upper lips " mean?

The expression pulled the few hairs on his upper lips conveys that the old gardener freaked out when he was told by his master that the wounded soldier in their house was from the enemies' army.