Carmen Mercedes McRae was born on April 8, 1920, in Harlem, New York City, to Osmond and Evadne (Gale) McRae, who had both emigrated from Jamaica. She began piano lessons at eight, in a home where records by Louis Armstrong and Duke Ellington filled the rooms. At seventeen she met Billie Holiday, who became her central influence, and as a teenager she came to the attention of the pianist Teddy Wilson and his wife, the songwriter Irene Kitchings. One of McRae's early songs, "Dream of Life," reached Holiday through that circle and was recorded by her in 1939. Some reference works give 1922 as her birth year, but the National Endowment for the Arts, Britannica and Discogs all give 1920.

In her late teens and early twenties she played piano at Minton's Playhouse, sang as a chorus girl and worked as a secretary. At Minton's she met Dizzy Gillespie, the bassist Oscar Pettiford and the drummer Kenny Clarke. In 1944 she worked with Benny Carter's big band and with the Count Basie Orchestra, and under the name Carmen Clarke, after marrying Kenny Clarke, she made her first recordings with Mercer Ellington's band in 1946 and 1947. She and Clarke separated in 1948, the year she moved to Chicago with the comedian George Kirby. For nearly four years she played piano and sang in clubs there, among them the Archway Lounge, before returning to New York in 1952, and she later told Jazz Forum that the Chicago years gave her whatever she had as a stylist. Back in New York she worked in the early 1950s with the Mat Mathews quintet and recorded for Stardust and, between October and December 1954, for Bethlehem, in sessions that included Herbie Mann and Clarke.

The break came when Milt Gabler of Decca heard her at a Brooklyn engagement. She signed with Decca in 1954 and was voted best new female vocalist of that year in the DownBeat poll. Her Decca albums of 1955 to 1959 include By Special Request, the ballad set Torchy! with orchestrations by Ralph Burns and Jack Pleis, Blue Moon, the rhythm-section date After Glow, Mad About the Man (an album of songs by Noël Coward), Carmen for Cool Ones and Birds of a Feather, on which Ben Webster plays tenor saxophone behind Burns's arrangements. She also made two duet albums with Sammy Davis Jr., Boy Meets Girl (1957) and Porgy and Bess (1959). A move to Kapp produced Book of Ballads, When You're Away and Something to Swing About. At Columbia she recorded Tonight Only! (1961) with the Dave Brubeck Quartet and, in 1962, a tribute to her idol, Carmen McRae Sings Lover Man and Other Billie Holiday Classics. She was also in Brubeck's jazz musical The Real Ambassadors, recorded in 1961 and premiered at the 1962 Monterey Jazz Festival, with Armstrong and the vocal trio Lambert, Hendricks and Ross, which included Jon Hendricks. She played a two-week run at London's Flamingo club in May 1961 and toured widely in Europe and Japan.

From the mid-1960s she recorded for Mainstream (Second to None, Haven't We Met?, Alfie) and Atlantic (For Once in My Life, Portrait of Carmen with arrangers including Oliver Nelson, Shorty Rogers and Benny Carter, and Just a Little Lovin', produced by Arif Mardin in Miami). She left New York for Southern California in the late 1960s but kept returning to New York clubs, and through most of the 1980s she played two engagements a year at the Blue Note there. The Great American Songbook, a double album recorded live at Donte's in Los Angeles on November 6, 1971, paired her with Jimmy Rowles and Joe Pass. The Groove Merchant albums It Takes a Whole Lot of Human Feeling (1973, with Pass and Ray Brown) and Ms. Jazz (1974, with Zoot Sims and Bucky Pizzarelli) kept her in small-group settings. At Blue Note in the mid-1970s she made I Am Music, produced and mostly arranged by Roger Kellaway, and Can't Hide Love, then the live At the Great American Music Hall, recorded June 15 to 17, 1976 in San Francisco with Gillespie as guest, which drew a Grammy nomination.

The 1980s brought a run of label partnerships and tributes. For Concord she recorded Two for the Road with George Shearing (1980), Heat Wave with Cal Tjader (1982, the vibraphonist's last album) and a Nat King Cole tribute recorded in 1983. Any Old Time with the tenor saxophonist Clifford Jordan appeared on the Japanese label Denon in 1986, and in 1987 came the live The Carmen McRae-Betty Carter Duets from San Francisco's Great American Music Hall, with Betty Carter, followed by the live Fine and Mellow from Birdland West in Long Beach. In 1988 she joined Harry Connick Jr. on "Please Don't Talk About Me When I'm Gone" for his album 20. Carmen Sings Monk, recorded in 1988 and released on Novus in 1990, set the compositions of Thelonious Monk to lyrics by Hendricks, Abbey Lincoln and others, with Jordan, the pianist Eric Gunnison, the bassist George Mraz and the drummer Al Foster. It was followed by Sarah: Dedicated to You (1991), a tribute to Sarah Vaughan with the Shirley Horn trio and her last recording. She withdrew from performing in May 1991 after an episode of respiratory failure only hours after finishing an engagement at the Blue Note. She received an NAACP Image Award in 1993 and was named an NEA Jazz Master in 1994. McRae passed away at her home in Beverly Hills, California, on November 10, 1994, at 74, a month after being hospitalized for a stroke and four days after falling into a semi-coma. Her screen work included The Square Jungle (1955), The Subterraneans (1960) and the television series Roots: The Next Generations (1979).

McRae was known for behind-the-beat phrasing, a smoky, dark contralto, and an ironic, sometimes acerbic way of delivering a lyric; the critic Ralph Gleason described one of her performances as an exercise in dramatic art. Her central model was Billie Holiday, whom she honored on record and by singing at least one Holiday song at every engagement, and she admired Ella Fitzgerald and Sarah Vaughan without imitating either, as the NEA's biography points out. She was also a pianist who accompanied herself in her early years and came back to the keyboard from time to time later; the NEA connects her piano playing and musical sense in part to hearing Thelonious Monk's music at Minton's. Britannica calls her scat improvisations innovative, complex and elegant, and the NEA describes her ballads as tender and warm, finding the depth of feeling in a lyric. Her repertory ran through standards, Broadway, the pop songs of the late 1960s and a series of songwriter and singer tributes to Noël Coward, Nat King Cole, Holiday, Monk and Vaughan, all within a vocal-jazz frame with swing and bebop roots. She is regularly grouped with Fitzgerald, Vaughan and Holiday among the central jazz singers of her generation, and later singers such as Nancy Wilson have named her as a formative influence.