Five-time Super Bowl champ Tom Brady said he is coming back to play for the New England Patriots once again in 2018 — but concedes that he might need to make his family a bit more of a priority.
While visiting the Milken Institute Global Conference in California on Monday, the 40-year-old quarterback talked about the “energy and emotion” he puts into football all year, while also saying he felt compelled to skip the Patriots’ voluntary offseason program to spend more time with family before the start of the new fall season.
“My kids are 10, 8 and 5. They’re not getting younger, so I need to take time so I can be available to them, too,” Brady said, according to ESPN. “I’ve really spent the last two or three months doing those things, and I think I’m really trying to fill my tank up so that when I do go back, I can go back and I think I’ll actually be, in my mind, a better player, a better teammate, because I’ll be really rejuvenated.”
“Part of this offseason for me is certainly about still preparing for what’s ahead in my next journey, my next mountain to climb with this group of teammates,” Brady also told the audience Monday. “But it’s also that a lot of people are getting the short end of the stick in my life — certainly my wife and my kids.”
In an interview with PEOPLE in March, Brady opened up about wanting to continue to play football at a high level despite his age, and how his family gave him their blessing as long as he can still find time for them.
“I love playing and I want to keep going, ’cause I think there’s more to achieve and I still think I can play at a really high level,” Brady told PEOPLE at the time. “As long as I’m willing to make the commitment, my family gets the time they need, hopefully I can keep playing the game I love.”
RELATED VIDEO: Tom Brady Reveals How Wife Gisele Bündchen’s Support Means He Is Not Retiring Just Yet
For Brady, taking his supermodel wife, Gisele Bündchen, to Qatar in early April may have been a part of filling up the tank.
Want to keep up with the latest from PEOPLE? Sign up for our daily newsletter to get our best stories of the day delivered straight to your inbox.
The couple took to social media to showcase their trip, which showed them riding camels and visiting Khalifa International Stadium to throw footballs with local children. Gisele, 37, has been married to Brady since 2009 and the couple have two children together: son Benjamin, 8, and daughter Vivian, 5. Brady also has a 10-year-old son, Jack, with actress Bridget Moynahan.
RELATED: Gisele Bündchen & Tom Brady’s Adorable Family Photo Album
In the months since the Patriots lost Super Bowl LII to the Philadelphia Eagles, there has been speculation about whether Brady would return to the team, and whether he was happy with New England’s front office. But Brady confirmed he would come back in 2018.
He has unfinished business when it comes to proving doubters wrong, he added.
“I have personal goals. I want to keep playing. I’ve said for a long time I want to play to my mid-40s,” he said Monday. “I was told three years, when I was 36-37, ‘You can’t keep playing; no one wins Super Bowls .’ It’s a great challenge for me. I think I’ve been challenged my whole life. I feel like I can do it.”