The Lions have started 1-3 with uncompetitive losses against the Western Bulldogs and North Melbourne, putting extra pressure on the out-of-contract Voss.
The Lions coach was in a similar position in 2011 when the Lions began their season with eight successive losses.
The club and coach have put off contract negotiations until the end of the season, but Voss is fully aware of the significance of Sunday's home match against Melbourne.
"I'm not surprised at all (at the scrutiny)," Voss said to a packed media contingent on Wednesday.
"(A) coach is not who I am, it's what I do. We've got a job to do, the job at hand is for us to concentrate on this week and knock that over and hopefully get some pressure on the ball.
"The expectation and result we're after is pretty clear."
Voss confirmed Brownlow medallist Simon Black would not be part of Sunday's match after playing his first game of the season in the reserves last weekend.
Black has headed north to Mackay with the reserves team again to play an ANZAC Day match against the Gold Coast reserves.
Voss said he might need a couple more weeks to fully regain match fitness after knee surgery in December.
There was some serious soul-searching after Sunday's 63-point defeat by the Kangaroos, according to the coach.
"I guess we didn't have a lot of chatter post-game (on Sunday), everyone was extremely disappointed and embarrassed about the performance.
"It's disappointing but at the same time I know the group's improved and that's pretty hard to convince people when you haven't got a lot to go with.
"I'm very confident our guys have got what it takes. It might take us a couple of weeks to get back to where we want to be.
"They're aware we've got some work to do and aware of the task at hand and we've got to put our heads down and bums up and get to work and realise it's not going to be fixed in one week.
"You are calling on the character of your group to turn that around quickly. I think we've got some really good character in our group and I back our players in."
Michael Whiting covers Brisbane Lions news for AFL.com.au. Follow him on Twitter: @AFL_mikewhiting
