Competing on home soil, Zverev lost 7-6 (7/2), 6-4 to the Australian, who is ranked 82nd in the world.
O'Connell converted his first match point just short of the two-hour mark to make it through to the last eight.
Zverev took the court on his 26th birthday, a day later than planned after rain postponed the last 16 clash from Wednesday.
In cold temperatures, the 16th-ranked Zverev took time to warm up, losing serve early in the first set.
A strong home support cheering on helped the Hamburg native fight back to force a tiebreak, but he made a number of errors close to the net as he fell behind.
Zverev broke back late in the second to level at 4-4, but lost the following two games to lose the set and the match.
Top seed Holger Rune eliminated Yannick Hanfmann also in straight sets earlier on Thursday.
Rune, up to a career-high ranking of seventh after reaching the Monte Carlo final last week, shrugged off the cold weather as he advanced 6-3, 6-4.
"I'm from Denmark," the 19-year-old said.
"At the beginning of the season there are always days like this and sometimes it's even worse."
