Alvin Gilbert Cohn was born on November 24, 1925, in Brooklyn, New York. He took piano and clarinet lessons as a boy without much enthusiasm; the music caught hold in his early teens when he heard Benny Goodman and then Lester Young. He played tenor saxophone in his high school band, with little formal instruction on the horn, and wrote arrangements for the band by copying down big-band charts from records.

At eighteen he was working in the band of clarinetist Joe Marsala (1943), and he then joined Georgie Auld's orchestra, where he began composing and arranging in earnest. Short stays with Boyd Raeburn (1946), Alvino Rey and Buddy Rich (1947) followed. In 1947 he wrote "The Goof and I," which Woody Herman recorded, and in 1948 he took Herbie Steward's chair in Herman's Second Herd, the saxophone section known as the Four Brothers alongside Zoot Sims, Stan Getz and Serge Chaloff. In that band he was overshadowed as a soloist by Getz and Sims but contributed arrangements, including "The Goof and I" and "Music to Dance To." He left in 1949 for Artie Shaw's short-lived bebop orchestra, and he also worked with Charlie Ventura.

His first date as a leader was recorded for Savoy in 1950 (the material later appeared on Al Cohn's Tones). Through the 1950s he was one of New York's busiest freelancers: a recording artist for Progressive, RCA Victor, Coral, Epic and Prestige, a writer for jazz and non-jazz settings, and, according to the writer Steven Cerra, an arranger for television programs including those of Sid Caesar and Ernie Kovacs and The Hit Parade. His charts were recorded by big bands led by Gerry Mulligan, Maynard Ferguson, Terry Gibbs and Bob Brookmeyer, and he arranged for Quincy Jones's 1959 band. In 1953 Miles Davis led a Prestige session devoted to Cohn's compositions, Miles Davis Plays the Compositions of Al Cohn, with Cohn and Zoot Sims on tenor.

From 1956 he co-led a two-tenor quintet with Zoot Sims, a partnership that continued on an irregular basis for decades. Al and Zoot (Coral) was recorded on March 27, 1957, and the group became a fixture at the Half Note in New York, where a live album was taped on February 6 and 7, 1959. The pair reunited for Body and Soul (Muse, recorded March 23, 1973) and Motoring Along, recorded in Stockholm on November 25, 1974. In June 1972 Cohn was a member of the Joe Malin Orchestra on stage with Elvis Presley at Madison Square Garden, and he took on Broadway work as an orchestrator, co-orchestrating Raisin (opened October 18, 1973) with Robert M. Freedman and orchestrating Sophisticated Ladies (opened March 1, 1981).

A run of albums for Don Schlitten's Xanadu label from 1975 renewed his reputation as a player: Play It Now, Al Cohn's America, the two-tenor sessions with Dexter Gordon that became True Blue and Silver Blue, the duo Heavy Love with Jimmy Rowles, and two albums recorded in Dakar, Senegal, in March 1980. He recorded for Concord in the 1980s, including Overtones with his son, the guitarist Joe Cohn, and in his last years he largely gave up writing to concentrate on playing. He died of liver cancer in Stroudsburg, Pennsylvania, on February 15, 1988, at the age of 62. His first wife was the singer Marilyn Moore, and his granddaughter Shaye Cohn plays cornet with the band Tuba Skinny, which she leads. The Al Cohn Memorial Jazz Collection, founded in 1988, is housed at the Kemp Library of East Stroudsburg University in Pennsylvania.

Cohn's first model was Lester Young, and his playing kept Young's light, flowing phrasing and easy, swinging time even as he took in bebop. His tone in the 1950s sat close to Zoot Sims's, so close that the two were often hard to tell apart on record, a complementary pairing that critics have treated as one of the standard two-tenor teams in jazz. Later he was influenced by Dexter Gordon and Sonny Rollins, and his tone grew darker, huskier and more individual. He was as admired for writing as for playing: his charts were recorded by the bands of Maynard Ferguson, Gerry Mulligan, Terry Gibbs and Bob Brookmeyer, and "The Goof and I" became a Herman standard. Though tied to Woody Herman and cool jazz, he was equally at ease with swing players, as on The Natural Seven, a septet date that looks toward the Kansas City Seven and used two members of Count Basie's band. The writer Doug Ramsey remembered him as a man whose quick, sharp remarks are still repeated by musicians.