Former Broadland MP Keith Simpson believes Nick Conrad was forced to quit as a parliamentary candidate because of 'Boris' problem with women'.
Mr Simpson was addressing the fact that the Conservatives want to move on from scandals surrounding candidates and the prime minister himself.
Mr Johnson has faced criticism in the past after conducting several extra-martial affairs and recently he was accused of being too close to American tech entrepreneur Jennifer Acruri while he was London Mayor.
Mr Conrad was backed as Broadland's candidate on Wednesday but just 24 hours later stood down after a furore about comments he made four years ago which implied that women were "partially responsible" for sexual assault.
MORE: Rival candidates react to Nick Conrad stepping down
Mr Simpson said: "I do not think it was central office's finest hour. There is no doubt in my mind that Nick was told he must stand down. It was the accumulation of several bad headlines at the start of the campaign on other issues and of course Boris' problem with women.
"When Nick stood I thought back to what had been said five years ago but my view was that he had apologised profusely and the BBC had not sacked him of course.
"On the night at the hustings he performed superbly and I was very impressed. He was asked about the comments and replied very well. So I am very sorry to see him go. I feel that people should be given a second chance but these things happen in politics.
"I am not sure what happens next in Broadland but we are somewhat running out of road - one thing I am sure about through is I am not coming back."
In a statement Mr Conrad said: "Five years ago I made ill-judged comments during an on-air radio discussion for which I made a genuine and heartfelt apology.
"I was honoured to be made the Conservative candidate for Broadland and had hoped to become the MP for a constituency which is close to my heart.
"However it has become clear to me that the media attention on my previous comments have become a distraction. For me, the most important thing is for the Conservative Party to be successful in the forthcoming election - getting Brexit done and delivering on the people's priorities.
"This is why I have reluctantly concluded I must stand down to allow one of the other excellent candidates the opportunity to win this fantastic seat.
"I would like to thank Broadland Conservative Association for their support and wish the party every success in the election on December 12."
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here