For many years before that, he and his wife had lived in an estate they built in Los Angeles' San Fernando Valley.
Joe Jackson managed daughter La Toya when she started her career in 1980 but she had little success - unlike Janet, who Jackson helped launch to stardom in 1982.
Joe and Katherine Jackson had 10 children over the 60-plus years they were married.
"No one knew what was going on-we shouldn't have to beg, plead and argue to see our own father, especially at a time like this", he said, according to the New York Post.
"I have seen more sunsets than I have left to see", the caption on the photo read.
When Joe Jackson, 89, passed away on June 27, his family quickly shared their feelings about their beloved father, grandfather, and husband with their fans. He became a controversial figure when Michael Jackson accused his father of physical and emotional abuse in 1993. Instead, he made his sons into superstars.
However, like many who are thwarted in their own paths, Joe Jackson looked to his children to fulfill his dreams.
While Joe outlived Michael, who died in 2009, he is survived by Katherine and their nine other adult children: Maureen, 68; Jackie, 67; Tito, 64; Jermaine, 63; La Toya, 62; Marlon, 61; Randy, 56; and Janet, 52. Their rise lifted the family out of poverty in Gary, Indiana, and into lives of fame and fortune.
Jackson was also the father of Joh'Vonnie Jackson, 43, the daughter of Cheryl Terrell, with whom he had a 25-year-long affair.
Coupe du monde 2018 : Les buts de la balade colombienne en vidéo
La défense polonaise, bien trop désorganisée, a été une vraie bénédiction pour James Rodriguez , Falcao et Cuadrado. La Colombie compte donc trois points, un de moins que le Sénégal - son prochain adversaire - et le Japon.
Jackson was predeceased by two of his sons: Marlon's twin Brandon, who died in 1957, shortly after he was born, and Michael, who died in 2009.
"Once he ripped the wire cord off the refrigerator and whopped me with it. It would make me sad that I would have to go to work instead", he said. "Joe was always pretty clear on what he wanted, and what he wanted wasn't always easy to deliver, but certainly appreciated his directness, and learned a lot from him".
He twice repeated the question back to Oprah before stating: 'Sometimes I do get angry. I will always cherish you. The youngest brother, Randy, also became a member of the group when it was known as the Jacksons. Janet said she found it hard to trust men because of her father's behavior. In a 2003 interview on the BBC, Jackson said, "I whipped [Michael] with a switch and a belt...[but] I never beat him".
No cause of death has been released, but Jackson had reportedly been in ill health. Jackson previously suffered a stroke and three heart attacks in 2015, but recovered with a pacemaker.
The outlet reported Jackson had planned to live out his final days in Vegas, but it had not happened.
In 2017 he, along with daughter Janet, had been involved in a vehicle accident near the Las Vegas Strip after a red SUV hit the Black Jaguar where Joe was in the passenger seat.
The children all eventually severed their management ties with their father but Janet paid tribute at the Radio Disney Music Awards in June 2018 by saying her "incredible father" had driven her "to be the best I can".
"My mother nourished me with the most extravagant love imaginable".