John Arnold Griffin III was born on April 24, 1928, in Chicago. He studied at DuSable High School under the bandleader Walter Dyett, moving from clarinet to oboe and then to alto saxophone, and at 15 he was already working in a band led by T-Bone Walker's brother. Three days after his high school graduation he joined Lionel Hampton's big band, where Hampton urged him to take up the tenor saxophone beside Arnett Cobb, and he appeared on record with the band in Los Angeles in December 1945, at 17. In 1947 he left with the trumpeter Joe Morris, a fellow Hampton sideman, to form a sextet that toured and made rhythm-and-blues records for Atlantic. By 1951 he was playing baritone saxophone in a septet led by Cobb, and an Army stint followed, much of it in a band stationed in Hawaii. Back in Chicago he became one of the city's leading saxophonists and took part in tenor cutting contests with Hank Mobley and Gene Ammons.

Thelonious Monk urged Orrin Keepnews of Riverside Records to sign the young tenor, but Blue Note got to him first. Introducing Johnny Griffin, recorded at Rudy Van Gelder's Hackensack studio on April 17, 1956, with Wynton Kelly, the bassist Curly Russell and Max Roach, won him critical attention. A Blue Note date followed on April 6, 1957, A Blowing Session, which put him on a front line with John Coltrane and Hank Mobley, backed by Lee Morgan, Kelly, Paul Chambers and Art Blakey. His own quartet date The Congregation, with Sonny Clark, Chambers and the drummer Kenny Dennis, was recorded that October 23. Through 1957 he also recorded with Blakey's Jazz Messengers, including the Atlantic album that paired the band with Thelonious Monk, and he made Riverside sessions for Clark Terry and the bassist Wilbur Ware.

In the summer of 1958 he succeeded Coltrane in Monk's quartet at the Five Spot Cafe in New York. The live albums Thelonious in Action and Misterioso, both drawn from the Five Spot show of August 7, 1958, with the bassist Ahmed Abdul-Malik and Roy Haynes, preserve that engagement. His own Riverside dates ran from 1958 to 1963: a sextet record with Donald Byrd and Pepper Adams; Way Out!; The Little Giant (August 1959) with Blue Mitchell and the trombonist Julian Priester; the large-ensemble The Big Soul-Band (1960), arranged by Norman Simmons; and White Gardenia (July 1961), a Billie Holiday tribute with strings and brass scored by Melba Liston. In 1960 he began a long partnership with the tenor saxophonist Eddie "Lockjaw" Davis, the bluesier of the two tenors. Their quintet recorded Battle Stations on September 2, 1960, Tough Tenors that November, an all-Monk repertoire album Lookin' at Monk! in February 1961, and four sets at Minton's Playhouse in Harlem on January 6, 1961, issued across several Prestige albums.

Griffin left the United States in 1963 and settled near Paris. Tax problems, a failing marriage and bitterness over the critical embrace of free jazz are usually given as the reasons, and he did not move back. He became a first-choice tenor for visiting American musicians and appeared regularly under his own name at clubs such as Ronnie Scott's in London. He played with Bud Powell in Paris and recorded with Wes Montgomery there in 1965, rejoined Monk for a 1967 European tour with an octet and nonet, and from 1967 to 1969 sat in the saxophone section of the Kenny Clarke and Francy Boland big band. He and Davis reunited in 1970 for Tough Tenors Again 'n' Again (MPS) and played together at intervals afterward, and in July 1975 he was a member of Dizzy Gillespie's Big 7 at the Montreux Jazz Festival. In the 1970s he moved from France to the Netherlands, settling in Bergambacht, and in 1978 he returned briefly to the United States with Dexter Gordon for the Eclipse Jazz Festival in Ann Arbor and a Carnegie Hall concert. That autumn he cut Return of the Griffin and Bush Dance for Galaxy in Berkeley.

In 1980 he moved to the Côte d'Azur, and in 1984 to Availles-Limouzine in the Vienne department, where he stayed for the rest of his life. He toured Europe, the United States, Australia and Japan, returned every year to Chicago's Jazz Showcase, and often played a week at the Village Vanguard in New York. His 1990 quartet session The Cat (Antilles), with Michael Weiss, Dennis Irwin and Kenny Washington, was widely read as a comeback, and he followed it with Dance of Passion (1992), Chicago, New York, Paris (Verve, 1994) and later dates with Martial Solal and Steve Grossman. Berklee College of Music gave him an honorary doctorate in 1995. His last concert was in Hyères, France, on July 21, 2008. He passed away on July 25, 2008, at 80, of a heart attack, at his home in Availles-Limouzine. Mike Hennessey's biography The Little Giant: The Story of Johnny Griffin appeared in 2008.

Griffin was called "the Little Giant" for his short stature and forceful playing, and the technique behind it was extraordinary: fast, exact articulation through long bebop lines, with a fund of bebop vocabulary and a habit of quoting from classical works, opera and other music in mid-solo. Charlie Parker was a major influence, and according to his New York Times obituary he chose the saxophone at 12 after seeing Gene Ammons play. His Chicago years were spent in cutting sessions, and he remained a combative, quick-witted soloist in tenor pairings, most famously with Eddie "Lockjaw" Davis. Hard bop is the label that fits his main recordings, and one JazzWax obituary heard the rhythm-and-blues apprenticeship with Lionel Hampton, Joe Morris and Arnett Cobb running through his bebop and hard bop years. He was a fashionable dresser, a good businessman and a bandleader other musicians liked working for, and from the 1960s he was one of the American tenors most in demand on the European circuit.