Mary Lou Williams was born Mary Elfrieda Scruggs on May 8, 1910, in Atlanta, Georgia, the second of eleven children, and grew up in the East Liberty neighborhood of Pittsburgh. A child prodigy, she was picking out tunes at the age of two, was taught by her mother from the age of three, and at six was playing parties to help support her half-brothers and sisters. Pittsburgh knew her as "The Little Piano Girl" by the time she was seven. She named the Chicago pianist Lovie Austin as her greatest influence, toured on the theater circuit as a teenager, and in the late 1920s married the saxophonist John Overton Williams, a bandleader whose group she joined in Memphis. In 1929 she took over that band when her husband left to join Andy Kirk's Twelve Clouds of Joy.

She followed him to Oklahoma City and Tulsa, and when Kirk's band settled into a long Kansas City engagement she began sitting in, then became its arranger and composer. For the Clouds of Joy she wrote "Froggy Bottom," "Walkin' and Swingin'," "Little Joe from Chicago," "Roll 'Em," and "Mary's Idea," and she played on the band's Brunswick sessions in Kansas City in 1929 and in Chicago and New York in 1930. On a Chicago trip she recorded two piano solos, "Drag 'Em" and "Night Life," billed as "Mary Lou" at the suggestion of a Brunswick executive; the records sold quickly and made her name. She became Kirk's regular second pianist while freelancing arrangements for Earl Hines, Benny Goodman, and Tommy Dorsey. For Goodman she wrote "Camel Hop," named for his radio sponsor, and in 1937 the boogie-woogie blues "Roll 'Em." He tried to sign her to an exclusive writing contract, and she turned him down to stay independent.

In 1942, divorced from John Williams, she left the Clouds of Joy and went home to Pittsburgh, where she formed a sextet with the Kirk trumpeter Harold "Shorty" Baker and a young drummer, Art Blakey. When Baker left to join Duke Ellington, she followed him to New York, married him in Baltimore, and arranged for Ellington's orchestra, including "Trumpet No End" (1946), her take on Irving Berlin's "Blue Skies." Within a year she and Baker had separated. Back in New York she took a job at Café Society Downtown, started a weekly WNEW radio show called Mary Lou Williams' Piano Workshop, and became a mentor to the younger bebop musicians. Dizzy Gillespie and Thelonious Monk were among those who came to her apartment around four in the morning to trade ideas until noon, and she wrote "In the Land of Oo-Bla-Dee" for Gillespie in 1945.

Her most ambitious work of the period was the Zodiac Suite, twelve pieces for the signs of the zodiac, several dedicated to colleagues, among them Billie Holiday, Art Tatum, and Ben Webster. She began writing it in 1942, played it first with bassist Al Lucas and drummer Jack Parker on WNEW, and recorded it with that trio in June 1945 for Asch Records. On December 31, 1945, she presented an enlarged version at Town Hall in New York with a chamber group that included the clarinetist Edmond Hall, and in June 1946 a version for a seventy-piece symphony orchestra at Carnegie Hall, an engagement arranged through the promoter Norman Granz. Both concerts left her drained. In 1952 she accepted an engagement in England and stayed two years, making the London trio sessions issued by Vogue. In 1954, in the middle of a performance in Paris, she stepped away from the piano and stopped playing.

She returned to the United States and converted to Catholicism in 1954. For about three years she devoted herself to the Bel Canto Foundation, which she started with her own savings to turn her apartment in Hamilton Heights into a refuge for poor people and for musicians struggling with addiction, supported in part by a thrift store in Harlem. Her friend and student Charlie Parker died in 1955, and priests close to her and Gillespie urged her back to music; she returned at the 1957 Newport Jazz Festival with Gillespie's band, and in 1958 she was one of only three women in the A Great Day in Harlem photograph of New York jazz musicians. Her first major work after the hiatus, Black Christ of the Andes, a hymn to the Peruvian saint Martin de Porres together with two shorter sacred pieces, was first performed in November 1962 at St. Francis Xavier Church in Manhattan and recorded in 1963. She founded her own Mary Records label and Cecilia Music publishing company, and with the support of Bishop John Wright of Pittsburgh taught at Seton High School, where she wrote her first Mass. One of her Masses, Music for Peace, was staged by the Alvin Ailey American Dance Theater in 1971 as Mary Lou's Mass. In April 1975 she performed it with youth choirs before about three thousand people at St. Patrick's Cathedral in New York, the first time a jazz musician had played in the church.

The 1970s were among her most productive years. She played nightly at The Cookery, the Greenwich Village club run by her former Café Society boss Barney Josephson, and recorded Zoning (1974) and Free Spirits (recorded 1975) with bassists and drummers of a younger generation. She held Guggenheim Fellowships in 1972 and 1977. On April 17, 1977, she shared two pianos with Cecil Taylor at Carnegie Hall, an encounter issued as Embraced. From 1977 until her death she was artist-in-residence at Duke University, where she taught the history of jazz with her manager Father Peter O'Brien and directed the Duke Jazz Ensemble. She performed at the White House in 1978, took part in Goodman's fortieth-anniversary concert at Carnegie Hall the same year, and recorded a solo recital at the Montreux Jazz Festival. In 1980 she founded the Mary Lou Williams Foundation. She passed away from bladder cancer in Durham, North Carolina, on May 28, 1981, at the age of 71; Gillespie, Goodman, and Kirk attended her funeral, and she is buried in Calvary Catholic Cemetery in Pittsburgh.

Williams played and wrote across the entire span of jazz piano in one lifetime: the blues and boogie-woogie of her Pittsburgh youth, the riff-driven swing of Kansas City, bebop harmony, and, late in life, a spare modern manner that one critic heard as showing the influence of McCoy Tyner and, in places, hinting at free jazz. She described herself as a "soul" player, someone who stays near the melody and the blues, and she spoke of praying through her fingers; after 1954 her composing turned toward sacred music, spirituals, and hymns. Lovie Austin was her stated first model, and Art Tatum was an influence on her touch, according to his own profile. As an arranger she shaped the sound of the Kirk band and supplied the Goodman, Ellington, and Dorsey orchestras with scores, and as a mentor she gave a generation of bebop musicians a place to work out their ideas. The Zodiac Suite is regarded as an early blend of jazz and modern classical writing, and Mary Lou's Mass as her most acclaimed work. Her reputation has grown since her death: the Kennedy Center has held an annual Mary Lou Williams Women in Jazz Festival since 1996, her papers have been held by Rutgers University's Institute of Jazz Studies since 2000, and her music has been taken up by Dave Douglas, whose Soul on Soul (2000) is a tribute album, John Hicks, whose Impressions of Mary Lou (2000) recorded eight of her compositions, and Geri Allen, whose Mary Lou Williams Collective released Zodiac Suite: Revisited in 2006.