Billy Harper was born in Houston, Texas, on January 17, 1943, and was singing in church by the age of five. A saxophone arrived as a Christmas present when he was eleven, and for a while he taught himself, helped along by an uncle, Earl Harper, a former trumpeter who had been at school with the bop trumpeter Kenny Dorham. Dorham's playing of the 1950s was the first model Harper worked from. As a teenager he played in Houston rhythm and blues bands and formed his own quartet at fourteen. He then went to North Texas State University, now the University of North Texas, took a Bachelor of Music degree in 1965, and was at that time the only African American member of the school's One O'Clock Lab Band.

He moved to New York in 1966 and led an ensemble on an NBC television special, "The Big Apple," in his first year there. Work with established bandleaders followed almost immediately. In 1967 he began the long association with Gil Evans that would run through the next decade, and from 1968 to 1970 he held a saxophone chair in Art Blakey's Jazz Messengers, a band documented on the live set cut at Slug's in August 1968 alongside Julian Priester, Bill Hardman and Ronnie Mathews. A very brief stint with Elvin Jones followed in 1970. In the same period he joined the Thad Jones and Mel Lewis Orchestra, playing tenor and flute on Consummation, recorded at A&R Studios in January and May 1970 in a reed section that also held Pepper Adams, Jerome Richardson and Jerry Dodgion. His first record with Max Roach, the gospel-facing Lift Every Voice and Sing with the J.C. White Singers, came in 1971, and he stayed in Roach's quartet through 1978.

Lee Morgan hired him for what turned out to be the trumpeter's final studio dates, recorded on September 17 and 18, 1971 and released by Blue Note as a double LP later reissued under the title The Last Session. Two of its five extended pieces are Harper compositions, "Capra Black" and "Croquet Ballet," and the band around them included Grachan Moncur III, Bobbi Humphrey, Harold Mabern and Reggie Workman. The Gil Evans years produced Svengali (Atlantic, 1973), where Harper solos on George Russell's "Blues in Orbit" and on his own "Cry of Hunger" and supplies the chart "Thoroughbred"; Where Flamingos Fly, cut in New York in 1971 and not issued until 1981; the Jimi Hendrix project of 1974; and There Comes a Time, recorded at RCA's Studio B between March and June 1975. Two Harper tunes, "Priestess" and "Thoroughbred," became fixtures of the Evans band's book, and "Priestess" eventually gave its name to an Evans album of its own.

The leader catalogue began in 1973 with Capra Black on Strata-East, the musician-owned label run by Charles Tolliver and Stanley Cowell. It is a large-ensemble record with Jimmy Owens on trumpet, Julian Priester and Dick Griffin on trombones, George Cables at the piano, Reggie Workman on bass, a rotating drum chair filled by Billy Cobham, Elvin Jones and Warren Smith, and a small vocal group that includes the singer Eugene McDaniels. Two years later the Italian producer Giovanni Bonandrini launched the Black Saint label with Harper's Black Saint, a quintet date drawn from a 1975 European tour, and in 1979 Bonandrini launched Soul Note the same way, with Harper's In Europe. Japanese labels took up the rest of that decade: Denon issued Love on the Sudan (1977), Soran-Bushi, B.H. (1978) and Knowledge of Self (1979), the last with Kenny Barron at the piano, while MPS recorded Trying to Make Heaven My Home in Stuttgart on March 3 and 4, 1979 and Baystate cut The Believer at the Power Station in a single day, February 15, 1980. He also made a duo-billed record in Tokyo in March 1974 with the trumpeter Jon Faddis, Jon & Billy, in a band with George Mraz, Roland Hanna and Motohiko Hino.

The Billy Harper Quintet has been his main working vehicle ever since, and the pianist Francesca Tanksley has held its piano chair for more than forty years. SteepleChase documented the band through the 1990s: Destiny Is Yours (recorded December 1989), then the three volumes of Live on Tour in the Far East drawn from a 1991 Asian tour, the first of them taped in Pusan, Korea on April 27. Somalia followed at the Power Station in October 1993, If Our Hearts Could Only See for DIW in 1997, and Soul of an Angel at RPM Studio on December 1 and 2, 1999; Eddie Henderson plays trumpet on all four. Blueprints of Jazz, Vol. 2, recorded across three days in August 2006, sets three of Harper's pieces to spoken texts by the poet Amiri Baraka. Alongside the playing, Harper has taught at Livingston College and Rutgers University and has held two grants from the National Endowment for the Arts.

Since the late 2000s he has been one of the front-line voices of The Cookers, the septet assembled by the trumpeter David Weiss around musicians who came up in the 1960s, recording Cast the First Stone (2010), Warriors (2011), Believe (2012), Time and Time Again (2014), The Call of the Wild and Peaceful Heart (2016) and Look Out! (2021) with Eddie Henderson, George Cables, Cecil McBee, Billy Hart and, in turn, Craig Handy and Donald Harrison on alto. He also sat in the tenor section of the Charles Tolliver Big Band for With Love (2006) and Emperor March: Live at the Blue Note (2009), and closed a forty-year association with Randy Weston by recording a duo album with him, The Roots of the Blues, in New York on February 8 and 9, 2013. In 2023 he turned eighty and the Quintet marked both that birthday and the fiftieth anniversary of Capra Black with a year of concerts; the current line-up pairs him with Freddie Hendrix on trumpet, Tanksley, the bassist Dezron Douglas and the drummer Aaron Scott. In 2025 the arranger Mark Masters recorded two albums for Capri with Harper as guest soloist, a centennial tribute to Sam Rivers and Dance, Eternal Spirits, Dance!, an orchestral programme built entirely from Harper's own compositions.

Harper belongs to the generation of tenor players who took John Coltrane as a starting point rather than a template: the harmonic reach and the long, climbing lines are Coltrane's inheritance, but the tone underneath them is a Texas tenor sound, broad and hard-edged, and the phrasing keeps returning to the cadences of the Houston church he sang in as a child. AllMusic's Chris Kelsey describes him as "one of a generation of Coltrane-influenced tenor saxophonists" who built on that work instead of copying it, and the Jazzwise critic Kevin Le Gendre, reviewing a 2025 Ronnie Scott's date, heard him take hard bop's swing and riffing and give it "a wholly personal imprint," singling out a reading of "Priestess" made of short, closely spaced melodic variations rather than harmonic display. He is at least as consequential as a composer. "Priestess," "Croquet Ballet," "Capra Black," "Cry of Hunger" and "Soulfully I Love You" have been recorded by Gil Evans, Lee Morgan, Woody Shaw and The Cookers, and his own long-form pieces, often built on a single modal centre and a repeating gospel-derived cell, are what the Quintet has played for fifty years. His Black Saint LP was named Jazz Record of the Year by the Modern Jazz League of Tokyo in the Voice Grand Prix.