Voss is out of contract at the end of the season and the pressure is growing.
After winning the NAB Cup, the Lions have been poor and are struggling at 2-5.
They have not made the finals since 2009, when Voss took over from Matthews.
Following a newspaper report on Saturday morning that Voss had until round 15 to secure his future as coach, the Lions put out a media release insisting there would be no review until the end of the season.
The Lions took it up to West Coast on Saturday before losing their round-seven clash by 26 points.
"If it continues on and they don't play finals and they only win seven or eight games and therefore you haven't played finals for four years, it almost becomes a statistical reality that the club looks for a change," Matthews said on Channel Seven's Game Day.
"The coach is the representative of the team - if the team isn't winning enough games of footy after a certain period of time, unless there's enormous faith from the board ... the numbers just get you.
"I had four years out of the finals and I sacked myself.
"It's a difficult decision because Vossy is so well regarded by everyone around the Brisbane footy club."
After coaching the Lions to the 2001-03 premierships and the 2004 grand final, Matthews quit in 2008.
"If you choose coaching, you know ... it's regarded that the team is reflective of the coach," Matthews said.
"Sometimes there is a great connection, sometimes you just don't have the (player) talent."
Meanwhile, Matthews said the Brisbane Lions teams of 2002-03 would easily have beaten Geelong's ladder-leading combination this season.
The Cats beat Essendon on Friday night to be the league's only unbeaten team.
"Geelong have had a better seven years than the Brisbane Lions had," Matthews said.
"But the Lions of 2002-03 would eat that Geelong team of Friday night - not necessarily the Geelong team of 2009.
"I don't think Geelong are as good as they were three or four years ago.
"But that might be as good, or better, than anyone else (now)."
Matthews pointed to the many young players in the current Geelong team, plus the dilution of talent in the league after the introduction of the two expansion clubs.
