James Louis "J.J." Johnson was born January 22, 1924, in Indianapolis, Indiana. He started on piano at nine before switching to trombone at fourteen, launching his professional career in 1941 with Clarence Love's territory band. In 1942 he worked briefly with Snookum Russell's band, where he met trumpeter Fats Navarro, an early influence on his emerging bebop vocabulary, before joining Benny Carter's orchestra later that year. With Carter he made his recording debut, soloing on "Love for Sale" in 1943, and in July 1944 he appeared at the first Jazz at the Philharmonic concert in Los Angeles. He toured and recorded with the Count Basie Orchestra from 1945 to 1946.

Leaving Basie, Johnson quickly became one of the trombonists most closely identified with the new bebop movement, recording with Charlie Parker at a Dial Records session on December 17, 1947, and working with Illinois Jacquet's band from 1947 to 1949. He played trombone on two of the three recording dates, April 1949 and March 1950, that made up Miles Davis's Birth of the Cool nonet sessions, later collected on Capitol; trombonist Kai Winding, who would soon become his most important musical partner, played the earlier January 1949 date. Johnson also led his own small recording sessions in these years, using sidemen including Bud Powell and a young Sonny Rollins. He worked with bassist Oscar Pettiford in 1951 and again briefly with Miles Davis, then spent roughly 1952 to 1954 largely outside music, supporting himself as a blueprint inspector.

In August 1954, producer Ozzie Cadena brought Johnson together with Kai Winding to form a two-trombone quintet that became known as Jay and Kai. The unusual all-trombone frontline was an immediate success: the group recorded for Savoy (Jay & Kai, 1954) and then Columbia (Trombone for Two, 1955; Jay and Kai, 1957) before splitting up in August 1956 so each leader could pursue his own working group. The two reunited periodically for the rest of their careers.

Johnson spent the years after the breakup leading his own quintets and sextets, drawing on a rotating group of sidemen that included saxophonists Bobby Jaspar and Clifford Jordan, cornetist Nat Adderley, trumpeter Freddie Hubbard, pianists Tommy Flanagan and Cedar Walton, and drummers Max Roach, Elvin Jones, and Albert "Tootie" Heath. The 1957-59 quartet dates First Place and Blue Trombone, cut with Flanagan, bassist Paul Chambers, and Roach, and 1958's Dial J.J. 5, with Jaspar, are among the best-documented records of this period. He increasingly devoted himself to composition and arranging in the Third Stream idiom pursued alongside John Lewis and Gunther Schuller, beginning with 1956's "Poem for Brass" and continuing through pieces including "El Camino Real" and the six-movement 1961 suite Perceptions, written as a feature for Dizzy Gillespie and recorded for Verve; his composition "Lament" went on to become a jazz standard. He toured again with Miles Davis in 1961-62, led further working bands documented on RCA Victor (J.J.!, 1965; Broadway Express, 1966; The Total J.J. Johnson, 1967) and Impulse! (Proof Positive, 1965, by most accounts his last studio recording with a regularly working band for roughly two decades), and reunited with Winding for an Impulse! session (The Great Kai & J.J., 1961, with pianist Bill Evans) and two CTI/A&M albums, Israel (1968, featuring pianist Herbie Hancock) and Betwixt & Between (1969). In January 1967 the pair also backed vocalist Sarah Vaughan on what proved to be her final sessions for Mercury Records.

By the late 1960s Johnson had shifted most of his working time to composing for film and television, and around 1970 he moved to the Los Angeles area to pursue that work more or less full time, contributing scores and cues to series including Mayberry R.F.D. and The Six Million Dollar Man. He performed only occasionally through most of the 1970s (he also played in Sammy Davis Jr.'s Cocoanut Grove show orchestra and in the band for Carol Burnett's television program during this stretch) yet remained popular enough with critics to keep winning DownBeat's trombone polls despite his near-total absence from touring and recording. A 1977 tour of Japan with Nat Adderley, documented on The Yokohama Concert and Chain Reaction: Yokohama Concert, Vol. 2, marked a partial return to performing, followed by further Japanese dates in the early 1980s (some again with Winding) and the leader albums Concepts in Blue (1981, Pablo) and, with fellow trombonist Al Grey, Things Are Getting Better All the Time (1984, Pablo).

Johnson's fullest return to active performing began with engagements at New York's Village Vanguard in November 1987 and July 1988, which supplied the material for two later live albums, Quintergy: Live at the Village Vanguard and Standards: Live at the Village Vanguard (both released 1991 on Antilles). While on tour in Japan in December 1988 he learned that his wife, Vivian, had suffered a stroke; he stopped performing for nearly three years to care for her until her death in 1991, and dedicated his 1992 Concord Jazz album Vivian to her memory. He married Carolyn Reid in 1992 and resumed a busy touring and recording schedule, releasing Let's Hang Out (1993), the string-orchestra album Tangence (1995, arranged and conducted by Robert Farnon, with a guest appearance by trumpeter Wynton Marsalis), the all-trombone summit The Brass Orchestra (1997, whose personnel included fellow trombonists Steve Turre, Slide Hampton, Robin Eubanks, and Jim Pugh), and Heroes (1998, featuring saxophonist Wayne Shorter). Johnson was voted into the DownBeat Hall of Fame in 1995 and named an NEA Jazz Master in 1996. He gave his final concert on November 10, 1996, at William Paterson College in New Jersey, then retired from touring and returned to his native Indianapolis, where he continued composing and arranging using computer notation software and wrote instructional trombone études later published by Hal Leonard. Diagnosed with prostate cancer in his final years, Johnson took his own life at his Indianapolis home on February 4, 2001, at age 77.

J.J. Johnson is widely credited as the trombonist who did for his instrument what Charlie Parker and Dizzy Gillespie did for saxophone and trumpet: he transferred bebop's rapid eighth-note lines and harmonic vocabulary onto the slide trombone with a speed and clean articulation so unusual for the instrument that some listeners assumed he was playing a valve trombone rather than a slide one. Where earlier trombonists had leaned on the instrument's natural glissando and "tailgate" effects, Johnson favored a light, precise, almost horn-like attack, and he paired that technical command with a serious compositional and arranging voice that produced Third Stream concert works and the enduring standard "Lament." Trombonist Steve Turre has said that Johnson did for the trombone what Parker did for the saxophone, and that every trombonist playing today owes something to what Johnson accomplished. Critics and fellow musicians generally credit him with making the trombone a viable frontline bebop voice for the first time and with shaping the playing of virtually every major jazz trombonist who came after him.