James Marcellus Carter was born January 3, 1969, in Detroit, Michigan, and began playing saxophone at age eleven, studying early on with trumpeter Marcus Belgrave. He also came up under reedman Donald Washington, joining Washington's youth ensemble Bird-Trane-Sco-NOW!! as a teenager. As a young man Carter attended Blue Lake Fine Arts Camp, where he became its youngest faculty member, and in 1985, at sixteen, he toured Scandinavia with the International Jazz Band. By 1986, at seventeen, he had already gone out on tour with Wynton Marsalis. He is the cousin of jazz violinist Regina Carter.

The turning point came on May 31, 1988, at the Detroit Institute of Arts, where Carter was pulled in as a last-minute guest for Lester Bowie; the encounter led to an invitation to join Bowie's new quintet, the forerunner of his New York Organ Ensemble, that November at New York's Carlos 1 club. Carter relocated to New York around 1990 and quickly became one of the most talked-about young saxophonists on the scene, recording with Bowie's New York Organ Ensemble on The Organizer and Funky T. Cool T. (both DIW, 1991). His earliest recorded appearance came the same year on the Tough Young Tenors' Alone Together (Antilles, 1991), a showcase for a generation of young tenor players that also featured Tim Warfield, Todd Williams, and Walter Blanding. He also became a member of the Julius Hemphill Sextet, the multi-saxophone ensemble built around Julius Hemphill's compositions, appearing on Fat Man and the Hard Blues (Black Saint, 1991) and Five Chord Stud (Black Saint, 1994), and he later recorded with the World Saxophone Quartet on Yes We Can (Jazzwerkstatt, 2010).

Carter made his debut as a leader with JC on the Set (DIW, 1994), a quartet date with pianist Craig Taborn, bassist Jaribu Shahid, and drummer Tani Tabbal, and followed it the same year with Jurassic Classics (DIW), billed to the James Carter Quartet. A deal with Atlantic produced a run of strikingly varied leader albums through the rest of the decade: The Real Quietstorm (1995), the standards-focused Conversin' with the Elders (1996), which brought in veterans Harry "Sweets" Edison, Lester Bowie, Hamiet Bluiett, and Buddy Tate, and In Carterian Fashion (1998). In 2000 he released two contrasting Atlantic albums on the same day: Layin' in the Cut, an electric free-funk session with guitarist Marc Ribot and bassist Jamaaladeen Tacuma in the vein of late-1970s Ornette Coleman, and Chasin' the Gypsy, a tribute to Django Reinhardt that reunited him on record with his cousin Regina Carter.

Carter moved to Columbia for Gardenias for Lady Day (2003), a tribute to Billie Holiday, then to Warner Bros. for Live at Baker's Keyboard Lounge (2004, recorded in Detroit in 2001). He formed the James Carter Organ Trio with organist Gerard Gibbs and drummer Leonard King, documenting the group on the live Out of Nowhere (Half Note, 2005, with guests Hamiet Bluiett and James Blood Ulmer) and later At the Crossroads (EmArcy, 2011) and Live from Newport Jazz (Blue Note, 2019). He also led Present Tense (EmArcy, 2008) and the all-star live date Heaven on Earth (Half Note, 2009), featuring organist John Medeski, bassist Christian McBride, guitarist Adam Rogers, and drummer Joey Baron, and paired repeatedly with the Surinamese ensemble De Nazaten on Skratyology (2009) and For Now (2010). Caribbean Rhapsody (EmArcy, 2011) set him against a concerto by classical composer Roberto Sierra. In 2022 he joined saxophonists Ivo Perelman, Tim Berne, and Tony Malaby for the collective session (D)IVO (Mahakala Music), and in 2023 released the solo album Un (Unaccompanied Baritone Saxophone) on JMI Recordings.

Along the way Carter kept up an extensive sideman career, recording with Wynton Marsalis and the Lincoln Center Jazz Orchestra on the Pulitzer Prize-winning Blood on the Fields (Columbia, 1997), Kathleen Battle's So Many Stars (Sony Classical, 1995), Madeleine Peyroux's Dreamland (Atlantic, 1996), Herbie Hancock's Gershwin's World (Verve, 1998), Ginger Baker and the DJQ20's Coward of the County (Atlantic, 1999), Steve Turre's TNT (Telarc, 2001), and Dee Dee Bridgewater's Eleanora Fagan (1915-1959): To Billie with Love (EmArcy, 2010), among many others. In 1996 he also appeared in Robert Altman's film Kansas City, portraying saxophonist Ben Webster alongside fellow musicians playing period jazz figures.

Carter is a virtuoso across the entire saxophone family plus bass clarinet and flute, and one of relatively few players of his generation to move fluidly between dixieland, hard-swinging mainstream jazz, funk and fusion, and free improvisation, often within a single album or set. That range shows in the sheer variety of his leader dates: a Django Reinhardt tribute, an electric free-funk session, a program of Billie Holiday standards, and organ-trio soul jazz all sit within a few years of one another in his catalog. He has won DownBeat magazine's Critics and Readers Choice award for baritone saxophone in multiple years. An avid collector and historian of vintage saxophones, he has performed on horns once played by Don Byas, and that immersion in the instrument's lineage feeds directly into a playing style that can evoke swing-era tenor giants and Albert Ayler-style free-jazz shrieking in the same phrase.