Belgium's Bashir Abdi took silver, improving on his bronze from Tokyo, and Kenya's Benson Kipruto claimed bronze.
Tola built a strong lead early on and crossed the finish line in two hours, six minutes and 26 seconds.
Emerging from a pack of frontrunners from the first steep ascent of an exceptionally hilly course, the former cross country specialist seemed only to strengthen on the second hill as others faded behind him.
Tola had an 18-second lead by the 35km mark, which he extended as the Eiffel Tower came into sight and crowds lining the streets roared him on. He became the first Ethiopian winner of the Olympic men's marathon in 24 years.
Tola's victory was all the more sweet given he was not initially on the team, having been called up after Sisay Lemma withdrew due to a hamstring injury. Tola won the New York marathon last year in a course record.
He crossed the finish line cheered on by Haile Gebrselassie, Ethiopia's former double Olympic 10,000m champion, whom he has cited as one of his inspirations to become a marathon runner.
Abdi and Kipruto were jostling with Ethiopia's Deresa Geleta for the next two steps on the podium but Geleta faded in the last two kilometres.
Abdi, who started out competing in the 5,000m and 10,000m, finished in 2:06:47 with 33-year-old Kipruto posting a time of 2:07:00.
Kipruto had posted the fastest time in the world this year having won the Tokyo Marathon in March in a personal best of 2:02:16.
He dedicated his bronze medal to Kelvin Kiptum, who died in a car crash in February at the age of 24, having broken the marathon world record in Chicago last year.
Kipchoge, widely considered the greatest marathon runner of all time, failed to bag a third consecutive gold, defeated by the first ascent of the course that took athletes out to Versailles.
He had been among the frontrunners but that hill broke up the leading pack and proved too much for the 39-year-old, who was participating in his fifth Olympics.
"The first part of the race we were together, talking and trying to move together. But I don't know what happened, so I will go and see him," Kipruto said of Kipchoge after the finish.
Ethiopia's Kenenisa Bekele, who has the fifth fastest marathon time ever, finished 39th.
"It was tough, you know, to challenge those guys," the 42-year-old said.
