Roscoe Mitchell was born August 3, 1940, in Chicago, Illinois. As a teenager he played saxophone and clarinet at Englewood High School, and while serving in the U.S. Army's Europe Band, stationed in Heidelberg, Germany, from 1958 to 1961, he played alongside a young Albert Ayler, an encounter Mitchell later described as formative to how he thought about the saxophone. Back in Chicago on the G.I. Bill, he enrolled at Wilson Junior College and fell in with a circle of young bebop-playing students that included bassist Malachi Favors and saxophonists Joseph Jarman, Henry Threadgill, and Anthony Braxton. Around 1961 he began studying with pianist Muhal Richard Abrams, and in 1962 joined Abrams's Experimental Band, a rehearsal ensemble exploring alternatives to conventional jazz composition and improvisation.

In 1965 Mitchell became one of the first members of the Association for the Advancement of Creative Musicians (AACM), the Chicago collective organized by Abrams, pianist Jodie Christian, drummer Steve McCall, and trumpeter/composer Phil Cohran. In 1966 his sextet, with trumpeter Lester Bowie, tenor saxophonist Kalaparusha Maurice McIntyre, Favors on bass, trombonist Lester Lashley, and drummer Alvin Fielder, recorded Sound for Delmark, the first album released by any AACM member and a landmark of the emerging Chicago avant-garde, built around collective improvisation, toy instruments, and extended silence rather than the more extroverted free playing coming out of New York at the time. By 1967 a quartet of Mitchell, Favors, Bowie, and (after drummer Phillip Wilson's departure) Jarman had formed; in June 1969 the group relocated to Paris as part of a wave of American free-jazz musicians drawn to a more receptive European audience, and a promoter's billing, "the Art Ensemble, from Chicago," became the group's permanent name. The Art Ensemble of Chicago recorded prolifically in Paris, including the acclaimed single-take People in Sorrow (1969), before returning to the United States in 1971; Mitchell remained one of the group's two co-leaders (with drummer Famoudou Don Moye) for the rest of his career, continuing after the deaths of Bowie (1999), Favors (2004), and Jarman (2019). The group's own catalog and history, including its ECM years and its later personnel, are covered in full in its own JazzProfiles.com profile.

Alongside the Art Ensemble, Mitchell built a parallel and equally prolific career as a leader. He continued developing an unaccompanied-saxophone practice documented on 1974's The Roscoe Mitchell Solo Saxophone Concerts, and in 1977 released the double album Nonaah on Chuck Nessa's Nessa Records, whose tracks included a sopranino saxophone duet with Braxton and a piano feature for Muhal Richard Abrams alongside trombonist George Lewis. After resettling briefly in St. Louis and then Chicago in the early 1970s, Mitchell founded the Creative Arts Collective around 1974 in East Lansing, Michigan, an organization modeled on the AACM's own cooperative structure. Through the late 1970s and 1980s he led a series of working groups, including the Sound Ensemble (Snurdy McGurdy and Her Dancin' Shoes, 1981, with bassist Jaribu Shahid, guitarist A. Spencer Barefield, and trumpeter Hugh Ragin) and a working quartet with Favors, drummer Steve McCall, and pianist Jodie Christian (The Flow of Things, Black Saint, recorded 1986).

In the 1990s Mitchell organized the Note Factory, a flexible sextet-to-nonet ensemble that became his principal vehicle as a leader for two decades, recording This Dance Is for Steve McCall (1993, Black Saint) with dual bassists (Shahid and William Parker) and dual drummers (Tani Tabbal and Vincent Davis), and Nine to Get Ready (recorded 1997, released 1999 on ECM), a nonet featuring trombonist George Lewis, trumpeter Ragin, and a two-piano, two-bass, two-drum rhythm section including pianists Matthew Shipp and Craig Taborn. During these same years Mitchell held a series of academic posts, teaching at the University of Wisconsin-Madison, the University of Illinois at Urbana-Champaign, and the California Institute of the Arts before joining the faculty of Mills College in Oakland, California, in 2007, where he held the Darius Milhaud Chair of Composition until his retirement from teaching in 2019.

Mitchell's output continued without pause into the new century: Song for My Sister (2002, Pi Recordings), Composition/Improvisation Nos. 1, 2 & 3 (2007, ECM) with the classical-jazz hybrid Transatlantic Art Ensemble, and Far Side (2010, ECM), a Note Factory live recording with pianists Craig Taborn and Vijay Iyer. In 2015 he reunited with Muhal Richard Abrams, Jack DeJohnette, and Henry Threadgill for the concert album Made in Chicago (ECM), and in 2017 he premiered Bells for the South Side, a double album commissioned by Chicago's Museum of Contemporary Art for the 50th-anniversary exhibition "The Freedom Principle," combining four of his working trios (including one with Tyshawn Sorey doubling on trombone, piano, and drums). Mitchell was named a National Endowment for the Arts Jazz Master in 2020, one of that year's four honorees alongside Bobby McFerrin. He has continued recording and performing steadily into his eighties, releasing Splatter (2020, a collaboration with an Italian symphony orchestra), Dots/Pieces for Percussion and Woodwinds (2021), One Head Four People (2024), and In 2, a 2025 duo recording with percussionist Michele Rabbia, and performing at the Big Ears Festival in March 2026.

Mitchell's music has centered from the start on what critics and Mitchell himself have called "sound as texture" rather than melody or harmony as the primary organizing idea, treating silence, extended technique, and unconventional "little instruments" as material on equal footing with a horn's conventional voice. His unaccompanied saxophone recitals, beginning in the 1960s and continuing across his career, are widely cited as pioneering the solo-saxophone recital as a jazz form in its own right. That same rigor carried into his large-ensemble writing and into later chamber-scale collaborations with classical performers such as Wadada Leo Smith, Pauline Oliveros, and pianist John Tilbury (on 2014's Nessuno), reflecting a career-long move between jazz-idiom improvisation and notated contemporary composition; AllMusic credits him with more than 250 compositions across both fields. The Penguin Guide to Jazz called him "one of the key figures" in avant-garde jazz, and he is widely regarded, alongside his AACM contemporaries, as a formative influence on the generation of Chicago-rooted improvisers who followed, including musicians drawn to his Note Factory and later large-ensemble projects.