Today it is still an exhilarating memory but one wrapped up under tarpaulin sheets and sharing a rickety shed with a child's pony.

But the name that appears when the covers are pulled back at the secret location in East Anglia would set the heart racing of any sailing fan beyond a certain age: Crossbow.

The dust-covered assemblage of bits and pieces – hulls, keels and rudders long since separated from mast – bears no resemblance to the sleek speed machine that captured the world's attention almost 40 years ago to the day. And its owner, Sir Timothy Colman, confesses it would be no easy task to put it all back together now although he is holding on to it in case one day it might take pride of place in a museum.

It was back in October 1972 that he joined fellow free spirits at Portland in Dorset for an attempt on a new world speed sailing record.

The wide ranging assembly of contenders which included sophisticated as well as experimental boats with wing sails and hydrofoils as well as a three-masted wind surfer were described by Hugh Somerville, the Sunday Times yachting correspondent of the day, as 'those magnificent men in their sailing machines'. But it was Crossbar – with Sir Timothy at the helm and a crew of three – that sped across the 500m course to establish a new world sailing speed record of 26.3 knots (30mph).

While attracting international Press attention at the time, what had become a fading memory for the retired Lord Lieutenant of Norfolk was revived as he watched the sailing at this summer's Olympics.

'They were sailing over the same water as us off the Dorset coast and it certainly brought back all the excitement of that time,' he said.

Sir Timothy, who went on to record a further six speed records in Crossbow and its successor Crossbow II up to 1980, said he had been a keen sailor from a young age and had raced in a variety of dinghies and small boats, including Dragons off Lowestoft. However, a busy work and family life had led him to give up competitive sailing until he read about the RYA's proposal to set up an international competition to establish a world sailing speed record.

He said: 'I was perhaps missing the challenge and when I saw that Peter Scott [the renowned conservationist, artist and sailor] who I knew, respected and as a boy had sailed with in his international 14ft dinghy was among the promoters, the temptation to take part proved irresistible and my wife, as always, was immediately supportive.'

Sir Timothy, who lives in Bixley Manor on the edge of Norwich – the house where he was born – commissioned designer Roderick Macalpine Downie and Reg White, both acclaimed catamaran sailors from Essex, to build the boat.

He said: 'Today's racing yachts and record breakers make full use of a steadily developing technology, including solid wing mats, carbon fibre and hydrofoils.

'Forty years ago we guessed that would be the end of the story, but with only a few months available for design, construction and preparation, the original Crossbow was built on largely conventional lines albeit with extreme dimensions with one object, flat out, wound up terminal speed.

'She was 56ft long, built of cold moulded plywood, with a 60ft mast but only 22 inches wide.'

To keep her upright, the crew of three joined the helmsman on a second smaller hull attached to an outrigger 30ft outboard. Joining the team early was former Olympian Tim Whelpton, who went on to build Crossbow II at his boatyard in Upton near Acle.

Recalling the remarkable interest and enthusiastic support of the local community in Weymouth, Sir Timothy said: 'For me as an amateur it was a thrill to take part in the pioneering stage of an aspect of sailing which has since become worldwide as well as progressively competitive. To sail in a large boat at high speed cutting through the water in almost complete silence was a memorable experience.'

While ending in triumph, the week on the Dorset coast had not passed without mishap. 'To minimise weight the mast had been deliberately rigged as lightly as possible – perhaps too lightly – and on the fifth day it broke,' he said.

Sir Timothy said the attraction of the speed challenge was that it could be fitted around busy lives in a couple of weeks of the year.

'We said among ourselves at the outset that if we happened to establish a new world record we would enter again the following year to prove it had not been a fluke!' he said.

'In the end, the two Crossbows sailed over eight years and held their final record of 36 knots (41mph) until 1986 when it was beaten by a Frenchman Pascal Maka on a sailboard in the Canary Islands with a speed of 38 knots.'

Over recent years the record has risen gradually with contenders favouring sites where there is a steady wind but with the sea sheltered by a spit of low-lying land; Namibia on the west coast of Africa is a current favourite.

The very latest record of 55.65 knots is held by Rob Douglas of the USA sailing a kite surfer which harnesses the power of the wind to propel a rider on a small surfboard.

There are more than 250,000 in the world today and they are scheduled to become a new class in the 2016 Olympics.

Sir Timothy said: 'My greatest hope is one day to see the record return to this country, best of all with a boat from East Anglia. Breaking speed records is nothing compared to the achievements of those like Ellen McArthur who sailed round the world single-handed, or the winners of gold medals in the Olympics.

'Crossbow nevertheless was an experience I would not have missed for anything – a view I am sure would be shared by the original team Roderick Macalpine Downie, Reg White, Tom Hall and Tim Whelpton who sadly are no longer with us'.