Over 100 people gather to join campaign to 'save our manor'