Downham Market Town Council objected to an application to convert a historic hotel into flats and a house of multiple occupation (HMO).
Councillors raised concerns over the "detrimental" impact to the heritage of Downham Market if the plans for The Castle Hotel were given the go-ahead.
A full council meeting was held on Tuesday, July 13 at the Jubilee Community Centre where councillors discussed their position.
A listed building consent and planning application has been submitted to West Norfolk Council to turn the hotel into seven self-contained flats, a five bedroom small house in multiple occupancy (HMO) and associated amenity and parking area.
A listed building application to convert the hotel into six apartments, to change a brewhouse into a one-bedroom property and to create another one bedroom property had previously been granted in March 2020.
Downham Market Town Council had objected to the previous application last January saying that "the loss of hotel accommodation would adversely affect the viability of Downham Market's town centre".
And on Tuesday, the council voted to recommend refusal of the new application on the grounds the development will have an "adverse impact on the area and affect a key building in the conservation area".
Deputy mayor Jackie Westrop said: "A number of people have raised concerns with me in in terms of the proposed development and the impact it will have on the town and the immediate environment.
"Conservation as we know is an active process of maintenance and managing change as it requires a flexible and thoughtful approach to get the best out of our listed assets.
"I feel the proposed changes do not do that. It actually significantly lessens the integrity of what is an iconic Grade II listed building.
"We have here in Downham lost an awful lot of our heritage assets."
Ben Molyneux-Hethrington said: "Clearly that is one of the major landmarks. People do come and see it as a tourist attraction.
"The last thing we need for the town centre is for there to be less there."
But John Doyle said: "The building has got to be used, if it's not used it will die."
Councillors voted to object to the proposals.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here