Cavendish: The 'Manx Missile' who sprinted his way to cycling's peak
The 38-year-old from the Isle of Man, though, will no longer have that sense of grievance as he rides into retirement at the end of the season.
"I've lost the desire to prove people wrong," he told The Guardian in 2021.
By the time he retires he may have secured for himself the greatest number of stage wins in the Tour de France - he is joint record holder on 34 with the sport's all-time great Eddy Merckx.
"In terms of pure sprinters he is the Tour de France's greatest ever at that discipline," Tour director Christian Prudhomme told AFP.
As for breaking the record he shares with 'The Cannibal', it seems the outside world is more interested than he is.
"Nah," he replied when asked by The Guardian whether it would mean anything to him.
He was not just a one show pony either, being crowned road race world champion in 2011 and winning the Milan-San Remo in 2009.
Like Tour de France champions Bradley Wiggins and Geraint Thomas, he also sparkled on the track - winning three world titles in the Madison (2005, 2008 and 2016) and striking gold for the Isle of Man in the 2006 Commonwealth Games in the scratch.
The one empty space on his mantlepiece which he never filled is an Olympic title. Cavendish claimed silver in the omnium in the 2016 Games in Rio.
'Fake it'
As a young man growing up on the Isle of Man he pored over technical guides on the art of bunch sprint finishing which was to serve him so well years later.
It was also in the Isle of Man aged 14 where he first came across Millar.
"A cocky little Manx kid came up and asked me for my autograph," Millar wrote in his autobiography 'Racing Through The Dark'.
Millar - eight years his senior - agreed to and also posed for a photograph.
Millar, who admitted to doping in 2004 but then on his return from the ban became a fervent anti-doping campaigner, was to cross paths with 'The Manx Missile' years later.
"You have a lot of kids come up to you," Millar told ESPN in 2012.
"To have one of those kids end up being 10 times better than you, and you being at their service within the timespan of your career (he was on the 2012 Olympic road race team along with leader Cavendish) is a bit surreal."
Family, as Cavendish said at his press conference on Monday, is primary for him. He has had some painful experiences with his brother Andy having served a prison sentence for drugs offences.
Even more traumatic was a home invasion in which he and his wife Peta - with whom he has four children - were threatened by knife-wielding burglars in November 2021.
Cavendish did not muck about in the wake of the burglary, investing in two Alsatians at £25,000 ($31,100) each.
Overflowing with competitive instinct, Cavendish will want to add to the 161 road race wins - including 16 stage wins at the Giro and three at the Vuelta - to bow out with a roar.
That total could have been many more had it not been for the Epstein-Barr virus - which causes glandular fever - which he suffered from for years after being misdiagnosed.
It sparked a serious bout of depression, but clear of that, he is frank about how he wishes to be recalled.
"In sport you have a character, a style, but it doesn't translate to what you do as a person," he told The Guardian in 2021.
"You can fake it, and a lot of riders do.
"They play super-nice but they're actually assholes.
"I'd rather be the other way round. I'd rather be proud of the guy I see in the mirror than the guy I see on TV."
