The German 12th seed, a semi-finalist in Melbourne in 2020, lost to 107th-ranked Michael Mmoh 6-7 (1/7), 6-4, 6-3, 6-2 on Margaret Court Arena in round two.
It followed two defeats at the recent United Cup, his first ATP matches since tearing ankle ligaments at the French Open in June.
A planned return at the Davis Cup in September failed to materialise when the 25-year-old pulled out after experiencing "extreme pain".
The two-time ATP Finals champion Zverev, who has dropped to 13 in the world, was clearly still struggling against the American Mmoh, hitting 58 unforced errors and having his serve broken eight times.
"I didn't have any expectations. I didn't have any goals, and it was a good test for me," he said of being at the opening Grand Slam of the year.
"I played a five-setter (in the first round), and I know where I am physically. That's about it. I guess I still have a long way back to where I was."
Asked if his ankle was still troubling him, he replied that it was not.
"My foot is healthy, but I'm not at the physical level that I was. I'm not as fast," he said.
"Definitely my conditioning is not as good as it was. So I think I still have a long way to go to there ... I think the last weeks I was getting better and better, and that is something very positive for me."
Mmoh will play countryman J.J Wolf next after he beat Argentina's 23rd-seeded Diego Schwartzman in three sets.
"Wow, this is the biggest win of my career without a doubt," said Mmoh. "Life is crazy, this is insane."
