South African-born Vickerman played 63 tests for Australia after his debut in 2002 and featured in three World Cup campaigns before being forced to call time on his career because of a leg injury less than a year after the 2011 tournament.
"The rugby world is in shock today after news of the tragic passing of Dan Vickerman," ARU chief Bill Pulver said in a statement.
"He was an enforcer on the field and a much-loved character off the field. Our thoughts and prayers are with Dan's family at this extremely difficult time."
No details of the cause of death were disclosed but local media reported that there were no suspicious circumstances.
Vickerman's international career came in two parts. He left Australia to study at Cambridge University in England in 2007 and made a surprise return to feature in the 2011 World Cup, his last test being the semi-final loss to New Zealand.
In 2010, he had a rod inserted into his right tibia which allowed his return to the international stage but he continued to suffer from considerable discomfort relating to a stress fracture in the bone.
He had the rod removed in February 2012 but was unable to get back to a point where he could play again and retired from the game in August that same year.
The tribute then Wallabies coach Robbie Deans paid him upon retirement gave an insight into why he was so keen to bring Vickerman back from international exile.
"As a player, Dan is an uncompromising competitor who sets high standards for himself, and demands the same of those around him," the New Zealander said.
"He commands enormous respect, from team mates and opponents alike, which says a lot about the man."
After retirement, Vickerman had worked in real estate and as a media pundit, while keeping his hand in the game with a role at the Rugby Union Player's Association (RUPA).
"We encourage all players who are affected by today's terribly sad news to support and be there for each other and to contact RUPA if they would like to talk," RUPA's Ross Xenos said.
The All Blacks and former Wallabies team mate Quade Cooper were among those who tweeted tributes to Vickerman. The 204 cm-tall second row also played Super Rugby for the ACT Brumbies and New South Wales Waratahs as well as two Varsity matches for Cambridge and a handful of games for English club Northampton Saints.
"The shockwaves of his tragic passing will be felt right across our rugby community," Waratahs chief executive Andrew Hore said in a media release.
"Our thoughts are with his wife Sarah, their two sons and other friends and family during this difficult time."
(Reporting by Nick Mulvenney; Editing by Amlan Chakraborty)