Joseph Bruce is a native of Berkley, Michigan. He was the last born of three children. His father, Richard Bruce, stole all of the family's money and left when Bruce was two years old. Joe's mother, Linda, was forced to care for him and his siblings, Robert and Theresa, off the income she made as a janitor. At age seven, Joe and his brother caught a butterfly, and both were fascinated by the vibrant colors and overall peacefulness of the creature. They kept the butterfly in a jar overnight, and intended to free it the following morning. When they awoke, the brothers found the butterfly had died, and felt as if they had committed a murder. The brothers made a vow that "one day, [they] will make it to heaven, so that [they] can [...] apologize to that Butterfly face-to-face." On every Insane Clown Posse album and EP it reads "Dedicated to the Butterfly."

Bruce received all of his clothes from rummage sales, and his food from canned food drives held at his own school. After moving to Oak Park, Bruce met John Utsler, and his little brother Joey. He also began his gang life by reluctantly getting involved in criminal activity in Royal Oak Township. Along with John and Joey, Bruce got heavily into rap music. In 1989, Bruce, as Jagged Joe, Joseph Utsler, as Kangol Joe, and John Utsler, as Master J, released the single titled Party at the Top of the Hill under the name of JJ Boys, but they did not pursue a serious career in music.

Bruce dropped out of school in ninth grade and moved in with his friend, Rudy, in River Rouge. There he formed his own gang called Inner City Posse, which would terrorize people with Army-issued tear gas and steal car radios for money. One night, Joe's mother, who had just moved to Ferndale, had her house attacked by rival gang Hazel Parks. Fearing for his mother's life, Joe fled to Bonnie Doone, North Carolina, a trailer park town just outside Fort Bragg, where his brother was staying with the U.S. Army. It was there that Bruce witnessed the open racism which would later emerge as the hate for bigots referenced in Insane Clown Posse's lyrics.

At seventeen, Joe returned to Ferndale. He was soon jailed, and the experience convinced him to get away from gang life. In 1990, after a short career in professional wrestling, Bruce and his friend Dale recorded Intelligence and Violence on a karaoke machine, which marked the debut of Bruce's stage name Violent J. Joe bought his own karaoke machine and, along with Joey and John Utsler, formed the music group Inner City Posse, named after their gang. After the release of the album Bass-Ment Cuts, the group hired record store owner Alex Abbiss as their manager, and established the Psychopathic Records record label with him in 1991.