After a VAR check ruled out what looked to be a Leipzig goal, Hart's attempted goal kick went straight to Hungarian midfielder Dominik Szoboszlai, who slid the ball to Andre Silva to put the home side 2-1 up.
The goal broke Celtic's stubborn resistance, with Silva adding another goal ten minutes later to put Leipzig 3-1 up and on course for their first points of the campaign.
Leipzig's Christopher Nkunku started the game strongly, shaking off midweek speculation he had already struck a deal to move to Premier League side Chelsea, heading the ball centimetres from the post with just over a minute played.
Leipzig's early buoyancy was however shattered just minutes later, with captain Peter Gulacsi stretchered off with what looked like a serious knee injury after a seemingly light challenge from a Celtic player.
Nkunku thought he had given the home side the lead just after play resumed, but his goal was ruled out for a narrow offside in the build-up.
Despite the disappointment, Nkunku got Leipzig on the board after 27 minutes, rounding Hart to glide the ball into the net after collecting a long, curling assist from Silva.
In the dying stages of the first half, both Silva and former Chelsea man Timo Werner had chances to double the home side's lead, but it was Celtic who would strike next.
Just two minutes after half-time, Celtic went on the attack, with Japanese winger Kyogo Furuhashi countering right up the middle of the field before putting Jota in acres of space to slam home the equaliser.
Leipzig looked to have won back the lead in the 61st minute, when Szoboszlai unleashed a trademark stunner from outside the box.
However, the goal was ruled out after a lengthy VAR check found an offside Silva - who swiftly dived out of the way when the ball came through - had obscured Hart's view.
Just one minute later, however, the duo united to pounce on Hart's attempted clearance, allowing Leipzig to reclaim the lead.
Silva scored his second of the night in the 77th minute to put the result beyond doubt.
Leipzig have three points after three games while Celtic are bottom of the table with just one point.
In Wednesday's other Group F match, leaders Real Madrid host Shaktar Donetsk.
