Paul Kenny, general secretary of the GMB union, said the party need to take a "strong decision" on whether to back Mr Brown or replace him with an autumn leadership election.
"The MPs have got to have the courage of their convictions - if they've got them," he added.
"They've got the constitutional right under Labour party rules, and my advice to them is get on with it."
But BBC political correspondent James Landale said MPs' summer recess would make it harder to foment a leadership plot.
Mr Cameron demanded that the prime minister call a general election after he returns from his summer holiday.
"I think we need change in this country, and that's how change should come about," Mr Cameron said.
However, Liberal Democrat leader Nick Clegg said it was not "time to play politics with people's lives" by calling an election "when so many people are worried about the price of a loaf of bread, how to fill their car with a tank of petrol".
'Brighter future'
Mr Brown, who has to call a general election by June 2010 at the latest, appeared to suggest he would not go to the country soon.
The prime minister seemed to speak without notes during his address to Labour's National Policy Forum.
Advertisement
Gordon Brown attempts to bolster Labour confidence
In what will be seen as an attempt to boost morale, he said there was "a brighter future that we can build".
He added: "Have confidence that not only do we have the right policies but that when the time comes we will be able to persuade the British people."
Speaking to the BBC's The World at One, Mr Darling said that the best way for the government to reconnect with voters was to help them with the cost of mortgages and rising fuel bills.
He added: "What we've got to do is rediscover the conviction on which we fought the general election 10 years ago and the two since then - and the values that we stand for."
We're not daft enough to ignore the fact that the public are concerned
Tony Lloyd MP
Universities Minister Bill Rammell said that although he was not complacent about the "very difficult economic circumstances" face by the government, Labour had a "a real fighting chance" of winning a fourth term.
But Manchester Blackley MP Graham Stringer urged cabinet ministers to hold a debate on the party's leadership.
He said the government had lost credibility with the voters after "bouncing around like Mexican jumping beans" on issues like the abolition of the 10p tax.
Tony Lloyd MP, chair of the Parliamentary Labour Party, said replacing Mr Brown was not the answer.
"We're not daft enough to ignore the fact that the public are concerned," he said.
"The answer to this, though, isn't looking for Gordon Brown's scalp. "
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.
One-Minute World News
E-mail this to a friend
Printable version
There is now no safe haven for Gordon Brown from the winds of political change
Bookmark with:
What are these?