A hundred days on and the talk around Brexit is as delusional as ever. This week it fell to Liam Fox, the international trade secretary, who is not allowed to negotiate international trade – that’s Brussels’ job until Britain leaves – to add to the already thick fog of fantasy. Fox declared that, after the UK had left the EU, it would enjoy terms of commerce with Europe “at least as free” as those we had when we were still in. That makes perfect sense – if you think the EU’s remaining 27 members are itching to show that exiting the EU exacts no cost, staying brings no benefits and others should start following us out the door.

Where there is not magical thinking, there is vagueness and opacity. The prime minister might allow us a glimpse of her preferred degree of Brexit when she addresses the Tory party conference on Wednesday, but so far the only clues have come from the serial slapdowns she has delivered to her ministers when they have supplied hints of their own.

In this haze of obfuscation, Jeremy Corbyn shines bright as a beacon of clarity. Admittedly, he too has sent conflicting signals on the hardness or softness of his ideal Brexit, but on the intimately related issue of free movement he made his position crystal clearthis week. In so doing, he confirmed that immigration has emerged as the defining dividing line in his fractious party.

Corbyn’s stance is admirable and principled. “It isn’t migrants who drive down wages,” he said on Wednesday, blaming instead “exploitative employers” and the politicians who had both loosened the labour market and weakened trade unions. In the same way, it wasn’t migrants who were putting strain on the NHS or causing a housing crisis. That was the fault of governments that had failed to invest in the health service or build new homes. And so, there would no “false promises” to bring down immigration. Indeed, Corbyn’s aides briefed that he was “not concerned about numbers” at all.

It is a noble case, a rare defence of migrants, a group routinely maligned. The trouble is, Corbyn’s stance is directly at odds with large swaths of his own party: those who would think of themselves as Labour realists. They are not all the usual suspects. Yes, they include the likes of shadow cabinet resigner Rachel Reeves, who spoke of her fear that “bubbling tensions” could “explode” if the kind of angst over immigration she encounters in her Leeds constituency is not assuaged. Andy Burnham, Chuka Umunna, Stephen Kinnock, Emma Reynolds and others have made a similar argument for controls. But so too has Angela Rayner, whose loyalty to Corbyn led her to take the shadow education portfolio after the summer rebellion. They are united in believing that the verdict of the electorate is now unmistakable. The voters want immigration controlled, they declared that loud and clear on 23 June and they can be ignored no longer.

I confess I find myself deeply torn on this most radioactive of topics. My every instinct is to stand with those who defend migrants and migration. Like so many others, I am the son of a mother born outside this country. A mere glance at my family tree shows the earliest roots were planted in soil far away from here. When I hear anti-immigration rhetoric I know that, 100 years ago, the target of such talk would have been me.

And yet, I cannot ignore what Rayner, Reeves and the rest are saying. None of them is a racist. None hates migrants. None of them would, I believe, be calling for increased controls on immigration if they could see another option. But they are simply listening to what has become the settled will of the people they represent.

This divide goes deeper than just immigration. It touches on class and geography too. For it is often a clash of the middle-class Labour of the big cities against the blue-collar Labour of the towns; London, metropolitan Labour – the Corbyn-McDonnell Labour that is relaxed about diversity – against the Labour of the post-industrial north, Midlands and east.

More deeply still, it is a battle over what Labour should be. For many, especially among the idealists of Momentum who held their own, much livelier conference this week, Labour has to be a social movement that works to change public attitudes on migration and much else – even if that takes a generation. Ranged against them are those who see Labour as a political party that has to meet the voters where they are now, not where they would like them to be.

There could be a pragmatic way to straddle this divide, revealed so vividly in the referendum. You could try to appeal to that part of the Labour coalition that overwhelmingly voted remain by promising to fight to retain British membership of the single market. At the same time, you’d try to placate pro-leave Labour voters by aiming to modify or restrict the free movement of people. Some would call it cynical, but you might just about hold the Labour tribe together that way.

Corbyn is doing the exact opposite. He and his shadow chancellor have signalled their readiness to jettison single market membership – to be replaced by mere “access” – but they are holding firm on free movement. For many, free movement is the price that has to be paid for the prize of single market membership. Corbyn’s position is: let’s give up the prize, but keep paying the price. At a stroke, he angers both Labour’s remainers and its leavers, depriving both of what they cherish most. Whatever the opposite of a political sweet spot is, Corbyn has found it.

To his credit, Corbyn did promise action to mitigate the impact of immigration. Rightly he said he’d revive Gordon Brown’s fund to pay for extra school places or housing in those areas that have taken in newcomers, and promised steps to prevent migrants undercutting the wages of workers already here. This way, he said, Labour would deal with “the real issues of immigration”.

But what if those are not the only real issues? What if it’s not just the strain on services and pressure on pay that makes people fear immigration? What if it’s actually more nebulous, and more toxic, questions of culture and identity that lie at the heart of this matter?

Easing the shortage of homes or GP appointments and boosting wages would certainly draw some of the sting from the immigration question. Government should take those steps. But it’s optimistic to imagine that such action would dispel all the rage MPs such as Reeves are encountering every day. It amounts to telling voters that they don’t understand their own feelings, that their concern over the changing face of their communities is simply an error, a case of false consciousness that will recede with a sufficient injection of public cash.

Corbyn himself said there should be no “lecturing or patronising” of those who voted leave, that “we have to hear their concerns,” including on immigration. That goes for him too – but he may not like what he hears.