Castlemaine, Victoria, is the place that brought Robyn Davidson back from decades of exile in Europe and India.

She has renovated a property there. She sometimes thinks of the house, once a burned-out pub, as an old girl she helped back up onto her feet. I could feel, almost see, all the guilt and pleasure of not writing folded into the house and garden.

“Really, the project of the last six years has been this house. Making a safe place, I guess,” she said. “And that’s been tied up with trying to finish the memoir and everything else. And once the home is completed, then I can fuck off.”

Others have commented on her attractiveness her whole life; to her it has become almost an in-joke. Suitors persist, locally – tradies, retirees, divorcees. One knocked while I was there – he interrupted, nervous, on a spurious errand with a cover story that fooled no one, and when he left, the odour of Jean Paul Gaultier cologne stayed, loitering in the hallway.

She took me on a domestic tour – desert-coloured interiors, a garden on an ambitious scale, camping gear still to be unpacked from her car – and said her favourite place in the house was the reading room. “What I’m really interested in,” she said, “is consciousness.” She pronounced the word carefully, as though clearing away the tang of bong smoke it had accrued in the 1960s.

She carried no literary ambitions in childhood. She trained, in adolescence, as a concert pianist. She grew up in the country, on a cattle station in Queensland, where music lessons were a link to the lost world of “accomplishments”, the means by which middle-class women became cultivated. She enrolled in music school but dropped out, and went to live in a squat with an abandoned piano. After time at the tail end of the Sydney Push, the bohemian enclave of “critical drinkers” that also produced Clive James, Germaine Greer and Robert Hughes, she headed inland, seeking a different kind of elsewhere. Years later, she found herself at the keyboard once more. She still plays – “badly”, she says.

Dread and avoidance are common impediments to writing, but Davidson’s degree of affliction is legendary. A deadline turns her into a flight risk. She has moved countries to avoid finishing a manuscript, and she still likens the act of writing to stage fright, performing a recital for which she has no sheet music, on an instrument she has forgotten how to play.

I made a guess, later confirmed, that Alfred Cortot would be one of her favourite pianists. Cortot hated practising, a habit that contributed to his famous tendency to make mistakes. His compensation was a life lived to the full, which enriched his interpretations of the Romantic composers, especially Chopin, imbuing them with emotion. His playing, even when frayed, was poetic.

Davidson is frank, almost harsh, about her technical imprecisions as a writer, but her calculus is similar. Writing is only a means to express what is most important to her, which is a desire for freedom. As often – more often – this desire takes her away from writing altogether.

I was conscious, in questioning Davidson about her life and work, of how furrowed my path was. I lagged decades of writers and pilgrims, interlopers and fans. Reading interviews to try to chicane through the questions already asked was pointless. They most often sought answers about the same thing – her first book, now published 40 years ago.

“You must have heard the word ‘tracks’ tens of thousands of times. Perhaps millions of times,” I tried.

“A gazillion,” she replied.

I tried to fight Tracks. I read the rest of Robyn’s work, seeking digressions and unorthodox openings, trying to reframe things. In the end this felt only avoidant, or artificial. There was no fighting it. She had walked the desert with camels, and this had come to define her.

Her account of the journey “hit something mythic”, as she put it.

A book was not part of the initial plan, but the duration of the trek – nine months – meant she needed money, for camels and supplies. National Geographic gave it to her, on the condition that she write about her experience.

The resulting article was called “Alone”, though she had not been alone the whole mission. Apart from her camels, and her dog, Diggity, she was accompanied intermittently by National Geographic photographer Rick Smolan, and an Anangu elder named Mr Eddie, who guided her past sacred sites. But the figure of the lone, questing woman was irresistible. She was famous before the desert crossing was completed – newspaper reporters sought her out while she was still miles from the coast – and at her destination she found a media-constructed persona already waiting for her: the “camel lady”.

This character, or caricature, has been unusually durable – younger generations that do not know Davidson’s work still know of “the camel lady” – and she recognised the paradox almost immediately: her act of freedom, and its transmission to the public, would forever enclose her in a kind of iron maiden.

“Had I been a man,” she writes in Tracks, “I’d be lucky to get a mention in the Wiluna Times, let alone international press coverage. Neither could I imagine them coining the phrase ‘camel gentleman’. ‘Camel lady’ had that nice patronising belittling ring to it. Labelling, pigeonholing – what a splendid trick it is.”

In the popular imagination, she became a heroine: the opposite of what she wanted to communicate in the book. “The point was,” she says, “that anyone could do what I did.” Despite the camel-lady moniker, that was the message received by generations of writers, readers and travellers, especially women.

On Robyn Davidson: Writers on Writers by Richard Cooke is published by Black Inc. Out now.