Max Hill claimed the country was “losing a generation” of men and women by locking them up when they come back from fighting for the terror group in Syria and Iraq.
His controversial comments came days after the head of MI5 Andrew Parker warned the terror threat facing the UK was “greater than ever before”.
Mr Hill, the Independent Reviewer of Terrorism Legislation, told the BBC: “The authorities have looked at them and looked at them hard and have decided that they do not justify prosecution.
“Really we should be looking towards reintegration and moving away from any notion that we are going to lose a generation due to this travel.
“It’s not a decision that MI5 and others will have taken lightly.
“But they have left space, and I think they are right to do so, for those who travelled out of a sense of naivety, possibly with some brainwashing along the way, possibly in their mid-teens and who return in a state of utter disillusionment.
“We have to leave space for those individuals to be diverted away from the criminal courts.”
This week, EU Security Commissioner Julian King said around 8,000 foreign fighters in Iraq and Syria could return to Europe after the fall of Isis stronghold Raqqa.
And Mr Hill’s stance drew immediate criticism from a number of MPs.
Foreign affairs committee member Conservative MP Andrew Rosindell said: “The presumption should be that all people who went to join IS should be dealt with by the criminal justice system unless there is a good reason why not.
“IS is an utterly barbaric organisation still intent on carrying out terror attacks in Britain and everything possible must be done to stop its supporters being involved in such atrocities.”
Labour’s home affairs select committee leader Yvette Cooper complained: “Travelling to a war zone to join an extremist terrorist group is a crime.
“We must be clear that there will be serious consequences for those who take this dangerous course of action.”
British security services say around 850 Britons have travelled to Syria to fight for ISIS, with around 250 thought to have returned to the UK.