The 44-year-old - widely considered the greatest quarterback in NFL history - spent 20 seasons with the New England Patriots, winning six Super Bowls, before relocating to Tampa Bay and leading the Buccaneers to a Super Bowl championship last season.
The decision comes as something of a surprise after Brady led the league in touchdown passes and passing yards this season and could still potentially win his fourth league MVP trophy.
ESPN reported the Buccaneers tried to convince him to come back for another season but he chose to prioritize his family and his health.
Brady is the father of three and he and wife Gisele Bundchen have been building a house together near Miami.
The Northern California native is the league's all-time leader in passing yards, passing touchdowns and quarterback wins.
He appeared in a record 10 Super Bowls, winning the game's MVP award five times, and his 15 Pro Bowl selections is the most in league history.
Amid the speculation, Brady's agent Don Yee wouldn't confirm his retirement, telling ESPN "I understand the advance speculation about Tom's future. Without getting into the accuracy or inaccuracy of what's being reported, Tom will be the only person to express his plans with complete accuracy."
Still, as news broke on Saturday, the official NFL Twitter feed tweeted, "Nobody did it better. #ThankYouTom"
