Who needs the Royal Academy, eh Damien?

One of Damien Hirst's friends made a piquant observation when the Evening Standard broke the news yesterday that Hirst had apparently declined an invitation to join the Royal Academy. "He doesn't have time for that kind of thing," the anonymous friend said. "He doesn't need it."

Hirst was always a maverick but then so was Tracey Emin, and she saw fit to accept the RA's medal of membership. So what makes Hirst believe he has no need of the country's most venerable and distinguished artists' institution?

The answer may be that Hirst is planning to create one of his own. Hirst is pouring money into restoring Toddington Manor, a Victorian stately home set in a cleft of the Cotswolds in Gloucestershire. He bought the house for a knockdown £3.2 million and plans to turn it into a world-class venue to showcase great art, including his own.

He has been collecting assiduously over a number of years and is believed to have one of the finest private collections in Britain. He spent $33 million (£15.9 million) at Sotheby's last November on a 14in x 12in self-portrait by Francis Bacon - a record for a small work by the artist.

He now owns four pictures by Bacon, whom he describes as having enormous influence on his own output. Hirst also has work by Warhol and other important 20th-century artists, as well as more contemporary-pieces from people including gold-ingot artist Paul Insect. There have been suggestions that he still owns his diamond-encrusted skull - titled For the Love of God and valued at £50 million - which would be an impressive centrepiece for his Toddington Manor collection.

The plans indicate that the ground floor, with stone arches and vaulted overheads, is to be turned into a vast gallery space. The first-floor apartments in the 200- room house are to be completely refurbished and the magnificent oak-panelled library - with shelves 20ft high - is to be restored to its original condition. Private apartments for his family are planned and further accommodation is to be built, possibly for visiting artists and academics.

When Hirst bought the house nearly three years ago he said he would make its restoration his "life's work". Unfortunately, those words have come back to haunt him. If he does indeed wish to create an alternative to the "big, fat, stuffy old pompous institution", as he once described the Royal Academy, he will have to be patient.

Toddington Manor today is an echoing shell, encased in plastic sheet and scaffolding. It looks like a work by Christo, the artist famous for wrapping buildings. The roof is a temporary arrangement of corrugated panels and work on refurbishing the magnificent interior of the house has yet to begin. One expert reckoned it could take at least 15 years to complete the project at the current rate of progress.

By then, Hirst will be pushing 60. His dream of turning Toddington Manor into a new home for the best of British art has collided with the reality experienced by most people who do up old places. The old manor, complete with resident ghost, has proved stubbornly resistant to salvage. Indeed, some might wonder whether Hirst has bitten off more than even he can comfortably chew.

He always knew it was going to be a massive task, taking at least 10 years or more. But a source familiar with the project said only when builders moved in did the true scale of the deterioration in the 180-year-old building become clear.

"It was a horror story," the source said. "The dry-rot was frightening. The roof was partly collapsed and water had been pouring in. It had got into the timbers and the floors - virtually everything had to be cut out over a major part of the building." When scaffolding was ordered Hirst was advised he would be better off buying it than renting it, because it would be needed for such a long period. Roof slates delivered months ago remain stacked on pallets because work on rebuilding the wooden structure, with huge beams and trusses, is still going on.

Hirst's love affair with Toddington Manor began when it was put on the market around three years ago. It attracted enormous interest from the start. Novelist Robert Harris wanted it but his wife, Gill Hornby (sister of Nick) talked him out of what Harris later described as an insane impulse - "a moment of madness".

According to locals, Madonna was an early viewer but found the place "too spooky". As well she might. Toddington Manor is famous among ghost hunters for the ghastly figure of a woman, half flesh, half skeleton, who is said to stalk the ground floor.

The house was built in the early 19th century near the site of a former Jacobean mansion. The owner, Charles Hanbury, later Baron Sudeley, designed most of it himself in Gothic Revival style, spending 15 years completing the house which took a large slice of his family's fortune.

The Hanbury family l ived at Toddington until 1901, and it was last used as a private home in the 1930s. During the war it housed American troops, then became a monastery and was owned briefly in the 1970s by David Wickens, the car auction tycoon. When he sold the house, it became a school for international students. In the late 1980s, the school moved out and the manor fell into disrepair.

"It was a tragedy," Geoff Smiles, a former caretaker at the house said. "It suffered a lot over the years. The school knocked up dormitories with styrofoam ceiling tiles and plasterboard, there was a makeover in some of the apartments in the 1970s with malachite wall coverings and pink Jacuzzis but nobody really got to grips with the fact the roof was leaking and there was dry rot everywhere."

Mr Smiles was among villagers who campaigned against a scheme to turn the manor into a 213-bedroom hotel and holiday complex. "That would have been a disaster," he said. "This area is one of the most beautiful parts of the Cotswolds. It is unspoilt and the traffic of a big hotel would have ruined it."

When Hirst stepped in to buy the house he was warmly welcomed by villagers, Mr Smiles said. "He made a brilliant impression. Last summer he gave part of the grounds for the village fête and everyone hopes he and his family will move in and the house will become part of village life again."

That may be some way away. Hirst, 42, and his wife Maia Norman, a Californian, and their three sons spend most of their time at their converted inn on the edge of Exmoor in North Devon. They also have a home in Mexico where they spend several months a year.

ENGLISH Heritage says the current efforts are only "emergency works" to save the building and that discussions on full restoration will come later. The manor is Grade I-listed and, as a national monument, is subject to stringent building conditions. Every move the builders make is closely scrutinised. In an effort to preserve as much as possible, after cutting out timbers affected by rot, the building team sprayed the rest with preserving chemicals.

It prompted one villager to remark that having made a successful career out of pickling dead animals, Hirst had now pickled a house. The presence of the toxic preservative solutions prevented access to some areas and work is currently concentrated on the east wing. It is hoped to finish roofing this part of the building this year.

But no one is making confident estimates on precisely how long it will take to realise Hirst's ambitions for Toddington Manor. At the moment, around a dozen workmen are engaged in the roof work and continuing demolition of some of the additions made to the house during its use as a school.

"Ultimately, the speed of the work will be geared to the amount he can spend on it," an insider said. "When it started, it was thought it could be done for maybe £5-£10 million. You could probably double those numbers now."

This shouldn't present too much of a problem for Hirst. Until fairly recently he was the world's most expensive living artist, displaced by Jeff Koons and then pushed into third place by Lucian Freud. His personal wealth, according to richlist estimates, is around £130 million.

And he clearly has money to spare, as his purchase of the Bacon self-portrait proved. He has also just ordered a £1.5 million solar power plant for his studios in Stroud. The plant will generate more than 300 kilowatts of electricity and is the second biggest in Britain.

His interest in solar power may provide a clue to his future plans for Toddington Manor, which requires a complete rewire. That, however, will have to wait. The builders are currently using a temporary set-up.

Hirst's people say he always thought of Toddington Manor as a long-term project. One said: "He doesn't feel there is any rush to complete it and work is being done gradually."

His ambitions may take longer to realise than he had hoped but he hasn't lost sight of them, or the motive for giving new life to a dying house. He said recently: "Buy art, build a museum, put your name on it, let people in for free. That's as close as you can get to immortality."

Quite so. The Royal Academy has been doing it for nearly 250 years.