Community chiefs are calling for the Fakenham College site in Wells Road to be looked after and well-used after sixth formers move out later this year.
One suggestion is that the swimming pool be reopened.
Fakenham Academy is currently spread over two sites – with the high school on Field Lane and Fakenham College, its sixth form, at the historic Highfield House on Wells Road.
But from September the two will unite, with the sixth form relocating to Field Lane and moving into revamped buildings which are to undergo a £750,000 facelift.
The new facility – to be named Fakenham Sixth Form – will have a refurbished study space, canteen and 10 classrooms in the Perowne Building, but will keep a separate entrance and parking spaces.
The TEN Group – the academy's sponsor – said discussions about the old site's future are ongoing with Norfolk County Council, which owns the Wells Road land.
Matthew Parr-Burman, principal at Fakenham Academy, said: 'We lease both sites from Norfolk County Council, so we are working with them on it. We would like to redraw our leases, so we would only lease the school and sixth form site, not the college site.
'If that's successful, we would hand back the old Wells Road college site to the county council. It would then be up to them what they do with it.
'There are obvious possibilities, including housing. Some of the buildings are listed, while some are not fit for purpose and could be demolished. 'It's really important the buildings are looked after and end up being part of the town.
'The best case scenario is that the county council quite quickly finds a use for the site. We don't want the buildings to become derelict.'
George Acheson, deputy town mayor of Fakenham, speaking as an individual and not for the council, said: 'I regret the loss of the school in the town, however I understand the financial imperatives. I would like to see the site developed, possibly for housing in the better buildings.
'It would be wonderful if we could reopen the swimming pool, which has been closed for about a year.'
Tom FitzPatrick, Leader of North Norfolk District Council, wants to see the site's character 'retained', and added: 'Maybe, a sympathetic residential development.'
What would you like to see happen to the old college site? Email david.bale2@archant.co.uk
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here