Mose John Allison Jr. was born on November 11, 1927, on his grandfather's farm outside Tippo, a small community in Tallahatchie County, Mississippi, on the eastern edge of the Delta. The farm was known as the Island because Tippo Bayou circles it. His father took over the farm and played stride-style piano, and his mother taught at the local school. Allison began piano lessons at five, picked cotton, played piano in grammar school and trumpet in high school, and wrote his first song at thirteen. The jukebox at the Tippo service station supplied country blues and the big-band swing of Count Basie, and in high school he listened to Louis Armstrong, Fats Waller, Duke Ellington, Louis Jordan and Nat King Cole.

He spent a period at the University of Mississippi, then joined the U.S. Army in 1946 and played piano and trumpet in the 179th Army Ground Forces Band. Back at college he led a small group and arranged for the dance band, and in 1949 he married Audre. He took a bachelor's degree in English, with a minor in philosophy, at Louisiana State University in 1952, working clubs across the Southeast and as far north as Denver. Along the way he discovered Bartók and other twentieth-century composers and began to concentrate on his piano playing. The pianists he most admired were John Lewis, Thelonious Monk and Al Haig, and he took his early vocal cues from the songwriter Percy Mayfield and the singer Charles Brown.

In 1956 he moved to New York City. The tenor saxophonist Al Cohn became his mentor and put him on record as a sideman, on Coral dates that include the Cohn quintet's Al and Zoot, recorded with Zoot Sims on March 27, 1957. Through the drummer Frank Isola, whom he met at the Jazz Loft on 34th Street, Allison began touring and recording with Stan Getz, and he also worked with Gerry Mulligan and Phil Woods. The Soft Swing (recorded in 1957 and not released until 1959) was his only studio session with Getz, and the two later played a month-long duo engagement at the Montmartre club in Copenhagen. In those years he was mostly heard as a pianist rather than a singer.

Prestige signed him in 1957. His debut, Back Country Suite, is a set of piano, bass and drums pieces that evoke the Mississippi Delta, and it holds the first recording of the blues later known as "Young Man Blues." Local Color followed, with the first recording of "Parchman Farm." With the bassist Addison Farmer and the drummers Nick Stabulas and then Ronnie Free, he made Young Man Mose, Creek Bank, Ramblin' with Mose and Autumn Song in 1958 and 1959. He moved to Columbia for Transfiguration of Hiram Brown (1960) and I Love the Life I Live (1960), the second made with three different rhythm sections, one of them Henry Grimes and Paul Motian. In mid-1961 he signed with Nesuhi Ertegun's Atlantic Records after a ten-minute meeting, and his Atlantic debut, I Don't Worry About a Thing (1962), opens with three of his most covered songs, including the title tune and "Your Mind Is on Vacation." He stayed with the label through 1976. Swingin' Machine (1962) put a quintet around him that included the trombonist Jimmy Knepper; The Word from Mose (1964), Wild Man on the Loose and the live Mose Alive! (both 1965), and I've Been Doin' Some Thinkin' (1968), his first self-produced album, with Red Mitchell on bass, kept to the trio format. Hello There, Universe (1970) used larger ensembles with Joe Henderson, Joe Farrell and Pepper Adams, Western Man (1971) added a Fender Rhodes and Billy Cobham on drums, and Your Mind Is on Vacation (1976), his last for Atlantic, re-recorded older songs with Al Cohn, Joe Farrell and David Sanborn.

British blues and rock musicians in particular kept finding his songs. John Mayall and the Bluesbreakers recorded "Parchman Farm," the Yardbirds recorded "I'm Not Talking," and Georgie Fame drew on Allison's material for years. The Who took "Young Man Blues" into their live set and released it on Live at Leeds (1970), the Clash covered "Look Here" on Sandinista! (1980), Leon Russell cut "I'm Smashed," Bonnie Raitt recorded "Everybody's Cryin' Mercy," and Elvis Costello recorded that song and "Your Mind Is on Vacation." Allison toured constantly, at one point playing up to 200 dates a year, according to AllMusic, and he was a regular at European and Japanese festivals. He recorded for Elektra/Musician in the early 1980s (Middle Class White Boy in 1982 and the live Lessons in Living in 1983, with Lou Donaldson, Eric Gale and Billy Cobham) and then made four studio albums for Blue Note between 1987 and 1998 in a partnership with the producer Ben Sidran: Ever Since the World Ended, My Backyard, The Earth Wants You (with Randy Brecker, Joe Lovano, John Scofield and Motian) and Gimcracks and Gewgaws. Blue Note also released The Mose Chronicles: Live in London in two volumes in 2001. In 1996 Van Morrison, Georgie Fame, Sidran and Allison made Tell Me Something: The Songs of Mose Allison, which reached number one on the jazz albums chart.

After 1998 he stayed out of the studio for twelve years, until the producer Joe Henry brought him to Anti- for The Way of the World (2010), which includes a duet with his daughter Amy. He was inducted into the Long Island Music Hall of Fame in 2006, received an honorary doctorate from LSU in 2008, was honored with a Mississippi Blues Trail marker in Tippo in 2012, and was named an NEA Jazz Master in 2013, receiving the honor at a Lincoln Center ceremony on January 14 of that year. He had three Grammy nominations, in 1983 and 1988 for best male jazz vocal performance and in 2001 for best jazz vocal album. AllMusic dates his retirement from live performance to 2012. Allison and his wife Audre lived in Smithtown, New York, on Long Island, and raised four children. He passed away at his home in Hilton Head, South Carolina, on November 15, 2016, four days after his 89th birthday.

Allison combined bebop-era piano with the blues he grew up hearing, and writers have long said the mix resisted any single marketing category. Prestige tried to sell him as a pop star, Columbia and later Atlantic as a blues artist, and Jet magazine, hearing him sing the blues, assumed he was Black and asked to interview him. At the keyboard he favored fast, whirling runs over marching left-hand countermelodies, a style that drew on Nat King Cole and Erroll Garner for relaxed swing, on Thelonious Monk, John Lewis and Al Haig for modern harmony, and on Bartók and Stravinsky for texture. His voice was a light, Southern-inflected speaking tone, closer to conversation than to a blues shout, and John Fordham of the Guardian likened its rhythmic delivery to an almost rap-like monotone. He occasionally returned to the trumpet he had played as a student, and the Guardian notes a muted trumpet on Local Color. He wrote around 150 songs, many of them dry, ironic observations on human folly, among them "Your Mind Is on Vacation," "Everybody's Cryin' Mercy," and "Ever Since the World Ended," and he also sang songs by Willie Dixon, Sonny Boy Williamson and Percy Mayfield. One interviewer called him a social critic before Bob Dylan and a satirist before Randy Newman. According to the NEA, his way with a lyric influenced Tom Waits, Pete Townshend, Ray Davies and Elvis Costello, and Georgie Fame called him more important than Bob Dylan. Though the best-known recordings of his songs are on rock records, he began his New York career accompanying saxophonists such as Stan Getz and Zoot Sims, kept his music in jazz settings, and was named an NEA Jazz Master.