FEDERAL Housing Minister Tanya Plibersek has given the thumbs down to tax breaks on loans for first home-buyers.

Aussie Home Loans boss John Symond has said the measure would help ease the housing affordability crisis.

But Ms Plibersek said it wasn't the fairest way of helping first home buyers.

"Whenever you make something tax deductible, the biggest benefits go to the highest earners and the people who have spent the most on their house," she told Channel 9 today.

"If you are wealthy and have bought a very big house, then you get a much, much bigger benefit."

What needed to be done was to make sure there was more affordable housing, especially at entry level.

A senate committee report released yesterday said the average cost of a house in the 1980s was three to four times the average annual wage but was now the equivalent to seven years' wages.

Ms Plibersek said the report made clear the real problem in housing affordability was that too few homes were being built and, for a wide variety of reasons, that would take time to fix.

"There are well over 20 recommendations. We are looking at them very closely now," she said.

"I got the report late last night. So I am not going to start agreeing or disagreeing with individual recommendations."

The report recommended changes to planning processes and infrastructure charges.

"They (the committee) also say we need to increase the stock of affordable rental accommodation ... We have already begun to move on a number of the issues that this report goes into in some detail," Ms Plibersek said.

The report also recommended bigger grants for people buying newly built homes.

"The Government agrees that we need to stimulate the building of new homes but I am not yet sure that is the best way.

"We need to make sure that those house and land packages or units are coming onto the market more quickly so that the developers aren't passing on lots of expenses, holding costs, interest rates and so on onto the people who are buying them.

"Most important of all, we have to keep a lid on inflation and downward pressure on interest rates so that people feel confident of going into the market."