Howard Lewis Johnson was born on August 7, 1941, in Montgomery, Alabama, and was raised from the age of two in Massillon, Ohio, where his father worked in a steel mill and his mother was a hairdresser. Neither parent played an instrument, but the household radio was always on. He was self-taught: after two lessons from a junior-high band director he took up the baritone saxophone, and in high school he learned the tuba by watching other players' fingerings during rehearsals and then working things out alone, pushing the instrument well past its customary range. After graduating in 1958 he spent three years in the Navy playing baritone saxophone in a service band. He met the drummer Tony Williams in Boston and lived for a time with the Williams family, and in Chicago he met Eric Dolphy, who urged him to move to New York. He arrived in 1963.

His first New York work was in the bands of Charles Mingus and Hank Crawford. He is credited on baritone saxophone on Crawford's Atlantic album Dig These Blues (1965) and on tuba on Mingus's Music Written for Monterey, 1965, and he went on to the orchestras and large ensembles of the avant-garde: tuba on Archie Shepp's Mama Too Tight (recorded August 19, 1966, and issued by Impulse! in 1967), tuba and baritone saxophone on Gerald Wilson's Live and Swinging (Pacific Jazz, 1967), tuba and baritone saxophone on Gary Burton's A Genuine Tong Funeral (RCA Victor, 1968), tuba on the Jazz Composer's Orchestra's double album for JCOA (1968), and tuba on Charlie Haden's Liberation Music Orchestra (recorded April 27 to 29, 1969, at Judson Hall and issued in 1970). Other sessions of those years include Andrew Hill's Passing Ships (recorded in November 1969 and not released until 2003) and, on Strata-East, Charles Tolliver's Music Inc. and Pharoah Sanders's Izipho Zam.

His longest association was with Gil Evans, which began in 1966 and ran on and off for about two decades. Evans wrote with Johnson's range in mind. The results include a tuba solo on Evans's Svengali (Atlantic, 1973), an arranging credit on the orchestra's Jimi Hendrix album (RCA Victor, 1974), There Comes a Time (RCA, 1976) and the Monday Night Orchestra's Live at Sweet Basil (released in Japan in 1985 and by Gramavision in 1986). In 1984 he was in the Gil Evans Orchestra that accompanied Jaco Pastorius at the Live Under the Sky festival in Japan. He also worked with Carla Bley: he is credited with voice on Escalator Over the Hill (JCOA, 1971) and with tuba, bass clarinet, baritone saxophone and other reeds on Tropic Appetites (WATT, 1974). Other credits include Gato Barbieri's Chapter Three: Viva Emiliano Zapata (Impulse!, 1974), Dexter Gordon's Sophisticated Giant (Columbia, 1977), George Gruntz's Concert Jazz Band on Theatre (ECM, 1984), Jack DeJohnette's Special Edition on Album Album (ECM, 1984), Hank Crawford's Night Beat (Milestone, 1989), John Scofield's Quiet (Verve, 1996), T. S. Monk's Monk on Monk (N2K, 1997), Mario Pavone's Deez to Blues (Playscape, 2006) and the Charles Tolliver Big Band's With Love (Blue Note, 2006).

A second career ran through rock and pop. In 1970 he wrote brass arrangements for Taj Mahal that put a section of low brass behind the singer, and on Mahal's live album The Real Thing (Columbia, 1971), recorded at the Fillmore East, Johnson is credited on double B-flat and F tuba, flugelhorn and baritone saxophone alongside Joe Daley, Earl McIntyre and Bob Stewart. He played on The Band's Rock of Ages (Capitol, 1972), and his obituaries credit him with leading the horn section at the group's farewell concert, filmed for The Last Waltz (1978). He is also heard on John Lennon's Walls and Bridges (Apple, 1974) and Double Fantasy (Geffen, 1980) and on The Muddy Waters Woodstock Album (Chess, 1975), and he worked with Levon Helm for decades. In 1975 he joined the house band of the new late-night program Saturday Night Live and stayed for about five years.

Johnson's own band began in the late 1960s as a tuba ensemble called Substructure, was renamed Gravity in 1972, and was revived after he left Saturday Night Live in 1980; Gravity was his best-known group. He first recorded as a leader in Hamburg, on July 4 to 9, 1994, for Arrival: A Pharoah Sanders Tribute, a Verve album credited to Howard Johnson's Nubia on which he plays baritone saxophone with a rhythm section and percussionists (issued in 1995). The Gravity albums followed on Verve: Gravity!!! (1996), recorded at Clinton Recording Studios in New York with a tuba and euphonium section of up to seven players, a piano-bass-drums rhythm section and Johnson's own arrangements, and Right Now! (recorded in December 1996 at Manhattan Center Studios and issued in 1997), with Taj Mahal as guest vocalist on three tracks. His fourth and last album as a leader, Testimony (Tuscarora, 2017), paired him with the tuba players Velvet Brown, Joe Daley, Earl McIntyre, Dave Bargeron and Bob Stewart, with Joe Exley as a guest and his daughter, the singer Nedra Johnson, singing on one track.

Howard Johnson passed away on January 11, 2021, at his home in Harlem, at the age of 79, after a long illness. His longtime partner, Nancy Olewine, announced his death. He was survived by Olewine, by his daughter Nedra Johnson, a singer and songwriter, and by two sisters, Teri Nichols and Connie Armstrong; his son, David Johnson, a musician and artist, had died earlier.

Johnson is credited with enlarging what the tuba can do in jazz: he played it as a lead voice, high and fast, rather than as a bass line or a novelty, and he wrote for groups of tubas as an ensemble in their own right. Bob Stewart says no one else could match Johnson's speed and range on the instrument, and Stewart, who sat in with Johnson's Substructure in 1967 and early 1968 before moving to New York, took the opposite route and made the tuba a rhythm-section bass. On baritone saxophone he worked as a section player and soloist for leaders as different as Charles Mingus, Hank Crawford and George Gruntz, and he doubled on bass clarinet, flugelhorn and penny whistle. His range of employers was unusually wide: the avant-garde of Archie Shepp and Pharoah Sanders, the orchestral colors of Gil Evans, the soul-jazz blues of Hank Crawford, and the roots-rock and studio world of Taj Mahal, The Band and Saturday Night Live. Reviewers praised Gravity!!! as a big step forward from his debut and noted that a band of five to seven tubas still swung and offered more variety than the instrumentation suggests; Right Now! drew comparisons to the textures of Gil Evans, and Testimony was called proof that the tuba can carry both lead and improvised lines.