Milenkovic agreed a five-year contract with the Premier League club after spending seven years in Florence.
The 26-year-old centre-back helped Fiorentina reach the Europa Conference League final in 2023 and 2024.
Milenkovic has been capped 57 times by Serbia and played every minute of their Euro 2024 campaign, which ended with a group-stage exit.
Forest chief football officer Ross Wilson said: "He is a key part of our squad planning this summer and I know how enthusiastic he has been to join the club.
"We have long admired his performances with the Serbian national team, and in Serie A and European competitions for Fiorentina."
Milenkovic made 264 appearances for Fiorentina after joining from Partizan Belgrade in 2017.
He will provide Forest boss Nuno Espirito Santo with a replacement for Moussa Niakhate, who moved to French club Lyon earlier in the close-season.
