Rodman, the daughter of former NBA superstar Dennis Rodman, struck in stoppage time at the end of the first half of extra time to finally break the deadlock, and Japan's resistance, at a packed Parc des Princes in Paris.
The result takes the USA through to a semi-final on Tuesday in Lyon, where they will face either reigning Olympic champions Canada or 2016 gold medallists Germany.
The USA remain on track under new English coach Emma Hayes to win a record-extending fifth women's football gold medal, and a first since defeating Japan in the London Olympic final in 2012.
That silver remains Japan's only medal in the competition, and they go home after thwarting the USA for most of the afternoon but failing to take what few chances came their way on the break.
The presence of Snoop Dogg - who is working at the Olympics for US television - in the crowd in Paris provided more excitement for many spectators than the game itself.
Japan sat back and allowed the USA to have most of the possession, which they did without showing a cutting edge.
Their front three of Rodman, Mallory Swanson and Sophia Smith were so impressive during the group stage but found the going tougher against Japan's back five.
That was until Rodman struck midway through extra time, as she controlled a long diagonal ball by Crystal Dunn on the right, turned Hikaru Kitagawa inside out and fired a shot high into the far corner of the net.
It was her third goal in four games at the tournament, making her the USA's joint-top scorer alongside Swanson.
Rodman was denied a second goal by an Ayaka Yamashita save early in the second half of extra time, but Japan could not find a way through at the other end.
The remaining quarter-finals all take place later Saturday, with World Cup holders Spain up against Colombia in Lyon.
Canada, who qualified from their group despite being docked six points in a drone spying scandal, meet Germany in Marseille before hosts France take on a Brazil side missing suspended captain Marta in Nantes.
