The panel was asked whether congestion charges should be introduced in all cities to improve air quality.
It comes amid new T-charges introduced in London by Mayor Sadiq Khan.
MP for Somerset Jacob Rees-Mogg, said: “In London, the congestion is worse even though there are fewer cars on the road. That’s because they have narrowed half the roads so you can’t get anywhere.
“I don’t believe that Government should make people’s lives more difficult and diesel is one of the real scandals of Government policy in the last 20 years.
The audience member asked whether all cities should introduce congestion charges
“People were encouraged to buy diesel because of worries about carbon dioxide, ignoring nitrous oxides and the particulates from diesel which have killed people.
“It meant people have died younger than they should have done. This is a real scandal of public policy.
“And no I don’t think the answer is penalising the motorist. Most of us want to drive into cities.
“Particularly if you represent a rural constituency like mine, people who want to go into Bath or Bristol they want to drive in. It would take them all week to get a bus.
“They would have to devote their whole life to a bus route. People want to drive in and out.
“I think politics is about making life easier for people, allowing them to do what they want to do and taking obstacles out of their way not ordering them about how they should leave their lives.”
1 of 4
Former SNP leader Alex Salmond said: “I was listening to Jacob and I was thinking this is a man who once campaigned in a Bentley in central Fife. Luckily there were no congestion charges at the time.”
Political editor for the Sunday Express Camilla Tominey said she drives a diesel car after being encouraged by the previous Labour Government.
She said that it makes her a “persona non grata in the environmental world”.
She added: “I was told it was more fuel efficient and that it was better for the environment and all that scientific evidence provided by Labour was roundly debunked.
“I’m interested in buying a hybrid car, but they are very expensive and I am not in the market to replace my car.”
Camilla Tominey said she was encouraged to buy a diesel car by Labour
David Dimbleby said: “And what’s your answer to the question? Your personal history is fascinating but what is the answer to the question?”
Miss Tominey said: “Thank you, David. We do have to take a personal issue on some of these questions because as a diesel driver it does make a difference.
“My answer to the question is this, if transport links are good enough there should be no reason why we need as ordinary citizens to drive into cities.
Germaine Greer said charging people won’t make any difference to energy policy
“However, if you’re a white van man, or a courier or a delivery driver I don’t see why you should be penalised for doing your job in the town centre.”
Baroness Chakrabarti said: “I think there is a role for charging, but only if it is matched with cheap, accessible public transport.
“And we do not have that in large parts of this country.”
The feminist writer and lecturer Germaine Greer said: “Look, taking money off people makes no difference to air quality whatsoever. There’s no connection between the two ideas.”