Waisale Serevi of Fiji is possibly the greatest rugby union player of all time. At least, he is if you are daft enough to think that rugby is something to do with a ball. I include him in my top five World Cup rugby players not because of what he has achieved in the World Cup, but because of what might have been.

The rugby World Cup was made in Hong Kong. The annual Hong Kong Sevens tournament was the first genuinely global gathering of the rugby nations, and it was a revelation to the game. It was first held in 1976: rugby teams from all over the world came there to play seven-a-side rugby and to sit and gasp in amazement at the fact that the game was about 10 times bigger than they realised.

The power-house of those early days was Fiji, who won in 1977, 78 and 80, years when the tournament was still held at the intimate old football stadium. I was there for the last of these. The Fijians played rugby like swashbuckling freebooters: and all Hong Kong identified with them heart and soul. Me among them, for I was a Hong Konger myself in those days.

It was suddenly obvious that rugby wasn't just about the home nations, France and the old dominions. It was the world's game: Fiji, yes, and Japan, Tonga, Samoa, United States, Papua New Guinea and many other nations who competed for the lesser prizes. Glory be, rugby union was a global game after all and so, 11 years after the first Hong Kong Sevens, they held the first Rugby Union World Cup.

And that was great too, but not great enough. That's because the sharp end of the competition was still reserved for the traditional centres of power. I had wild romantic notions about Fiji becoming a global power and running the whole world ragged, but it was not to be. The World Cup has now been held seven times and only four nations have won it. Only five have ever reached the final. There have been 56 places in the quarter-finals of the World Cup: only eight nations have filled them.

And never Fiji, despite the presence of Serevi. I went to the Hong Kong Sevens several times during the 1980s and Serevi was the best I ever saw. He had that gift that is so much more obvious in sevens than the 15-man game: the ability to create something from nothing.

Time and again, he'd take the ball with seven players ahead of him, rocked back onto his heels, seemingly at a loss. About 1.5 seconds later, all seven members of the opposition would be setting off in vain pursuit. He seemed to have the ability to make a hole in the defence by thinking about it.

Speed, yes, and ball-skills, yes, a great passer. But it was that streak of imagination that separated him from anyone else: to create something that looked and felt like magic. How the hell did he do that? I don't believe I just saw that. That was Serevi.

Waisale Serevi in action against England© Ross Kinnaird /Allsport

He played in the World Cups of 1991, 1999 and 2003, but he was always fighting an underdog battle against vastly superior odds. Fiji and the rest were allowed to take part, but it seemed they were never allowed to improve. They were valuable only because they could be beaten.

Serevi decided to become a rugby player in 1977. Fiji - in another famous false dawn - had just beaten the British and Irish Lions. The young Serevi had no idea what was going on. He just noticed a tidal wave of joy sweep over the island and asked why everybody was happy. On hearing the answer, he decided to become a rugby player: "So that I could make everybody happy."

Serevi seems to me to mark two wrong turns made by rugby since the invention of the World Cup and the subsequent professionalisation of the game. The first is that the second and third tier nations have been insufficiently cherished. New Zealand notoriously uses the Pacific islands as feeder nations, and no one in rugby ever dares to come up with a suggestion that might upset New Zealand, so that's a situation we're stuck with.

The second is that rugby is now a game about physical size. Backs are the size that forwards were when the first World Cup was played. It's become a game for giants and its physical exchanges are ever more robust. This increasingly takes the game away from ball-players. I doubt if Serevi would make any of the top teams for the current World Cup.

As a result of this, rugby has become two games. Sevens has become an area for specialists and it will make its debut as an Olympic sport in Rio next year, something quite separate from 15-man rugby. The game where people run round opponents has split off from the game where people run though them, almost as rugby league split from rugby union in 1895.

As a result, 15-man rugby union has become a little one dimensional (though if you watch England you're always likely to get that impression). Rugby is now less of a universal game than it might be. The mighty physical confrontations are seldom counterpointed by brilliant running and ball-handing.

Injuries are increasingly frequent as collisions get more and more fierce, because they are contested by very large, very strong, very fit people. There are times when I wonder how much longer the game will remain viable.

Rugby has made its choice, in terms of legislation, and it seems that there is no going back. There's not a lot of point in suggesting that it might be interesting to experiment by, say, doing away with the scrum and the lineout and making more space by taking two men from every team.

Serevi remains everybody's choice as the finest sevens player of all time, and he would still find a good few votes for the best player of all time. It's customary to end such a eulogy with the phrase "we shall not see his like again". But it's not often that the sport itself seems determined to make sure that's the case.