Mr Johnson stirred up a hornet’s nest after he made the comparison in a column in The Daily Telegraph.
However, Taj Hargey, Imam at Oxford Islamic Congregation, said Mr Johnson “should not apologise for telling the truth”, adding that his comments did not go far enough.
In a letter sent to The Times, Dr Hargey said there was “no Koranic legitimacy” for the burka, adding it was “a nefarious component of a trendy gateway theology for religious extremism and militant Islam”.
He added: “The retrogressive Islamic clergy has succeeded in persuading ill-informed Muslims through suspect secondary sources that God wants women to cover their faces, when in reality it is a toxic patriarchy controlling women.
“It is any wonder that many younger women have internalised this poisonous chauvinism by asserting that it is their human right to hide their faces?
“Johnson did not go far enough.
“If Britain is to become a fully integrated society then it is incumbent that cultural practices, personal preferences and communal customs that aggravate social division should be firmly resisted.
Dr Hargey has frequently criticised the practice of wearing the burka, and has also courted controversy in some circles for allowed men and women to pray together and for discouraging Muslim-only schools.
Meanwhile Scotland’s Conservative leader Ruth Davidson called for the former Foreign Secretary to apologise, despite agreeing with the wider point in his column that face-covering veils should not be banned.
Speaking in East Lothian, she said: “I think that this wasn’t an off-the-cuff slip.
“He knew exactly what he was doing and I think it crossed from being provocative and starting a debate and actually it became rude and gratuitous.
If you use the analogy of Christianity, would you ever write in The Telegraph that you should have a debate about banning Christians for wearing crucifixes?
“It’s the same argument but it’s a in a different faith, so why are the parameters different for one faith and not the other?”
Meanwhile, former attorney general Dominic Grieve said he would quit the party if Mr Johnson became leader, while ex-minister Anna Soubry said that “many” One Nation Tories would also leave.
However, Mr Johnson has also won support from fellow Brexiteer Jacob Rees-Mogg, who criticised Prime Minister Theresa May for calling on Mr Johnson to say sorry.
He said: “She’s clearly wrong to have asked him to have apologised.
“It’s not the job of the Prime Minister to tell backbenchers what to do.
“We are not bound by collective responsibility or that type of discipline and it is the wrong approach to party discipline.”
Shortly after the publication of his article on Monday, a source close to Mr Johnson said that it was “ridiculous” that his views should be under attack.
They said: “We must not fall into the trap of shutting down the debate on difficult subjects.
“We have to call it out. If we fail to speak up for liberal values, then we are simply yielding ground to reactionaries and extremists.”
A Sky News poll of 1,649 people has suggested that 45 per cent of voters thought he should apologise, while 48 per cent thought he should not.