James, 35, umpired 206 games since his debut in round one, 1999 and officiated in the 2001, 2002 and 2004 grand finals.
He umpired 16 finals and was named the All-Australian umpire in 2004. He also umpired International Rules.
James' injury has prevented him from umpiring any games this season and he told his fellow senior umpires of his decision to retire on Wednesday night.
His last match was in round 22 last year.
James said a combination of factors contributed to his decision.
"I used to just really flog myself with my training, so I think it's about training smarter now and there's no doubt that you can umpire until you're 45 now, if you wanted to - Hayden [Kennedy] is doing that successfully now," he said.
"But my body just gave up and it wasn't as much my body, but probably my mind as well - that desire just wilted and I just don't have the passion to actually umpire anymore."
AFL umpires manager Jeff Gieschen said James was a wonderful umpire who had an outstanding career.
"Everybody who watched Matthew at the top of his game between 2001 and 2005 would say that at that particular stage, he was probably the prototype of what AFL umpiring was all about - athletic, fantastic decision-maker, a brilliant bouncer and he had a terrific relationship with the players," Gieschen said.
"He really umpired consistently at a very strong and high level through that period."
James said it was a privilege to be referred as a prototype umpire.
"There is no doubt that through that period from 2001 to 2005, I was at my peak and I was doing everything I could in terms of my training - it felt like a great reward for me to put in all that work and all of the sacrifices from the time I was 16," he said.
"It's just great to be known as a bit of role model."
He began his umpiring career in the old Footscray District league, now known as the Western Region league, before spending five seasons as a VFL umpire.
He officiated in the 1997 and 1998 grand finals before being appointed to the AFL umpiring panel for the 1999 season.
James said umpiring grand finals were an honour, but his lighter moment came when colleague Peter Carey marked the football during the Fremantle and St Kilda match in Perth in 1999.
"When it happened, I looked at Aussie [Jones] and we looked at each other and said 'that's not supposed to happen, is it?'" he said.
"I responded with 'I'm glad it's him and not me'."
