Oliver Edward Nelson was born on June 4, 1932, in St. Louis, into a musical family: a brother played saxophone with Cootie Williams's band in the 1940s, and a sister sang and played piano. He started piano at six and saxophone at eleven, and from 1947 he worked in the territory bands of St. Louis, among them the Jeter-Pillars Orchestra and the groups led by George Hudson and Nat Towles. In 1950 he joined Louis Jordan's band, where he played alto saxophone and wrote arrangements until 1951.

In 1952 he entered the U.S. Marine Corps and played woodwinds in the 3rd Marine Division band in Japan and Korea. In Japan he heard the Tokyo Philharmonic play Ravel's Ma mère l'Oye and Hindemith's Symphony in E flat, which he later called the first really modern music he had heard and the point at which he decided to become a composer. Back in Missouri he studied composition and theory at Washington University in St. Louis and at Lincoln University, finishing a master's degree in 1958, and he also studied with the composers Elliott Carter, Robert Wykes, and George Tremblay.

After graduating he moved to New York, where he played with Erskine Hawkins and Wild Bill Davis and became the house arranger at the Apollo Theater in Harlem. In 1959 he spent a short time on the West Coast in Louie Bellson's big band, and that year he began recording for Prestige as a leader, starting with Meet Oliver Nelson (1959), a quintet date with trumpeter Kenny Dorham. A run of small-group albums followed in 1960 and 1961, among them Taking Care of Business, Screamin' the Blues, Nocturne, Soul Battle, Main Stem, and Straight Ahead. Two of those, Screamin' the Blues and Straight Ahead, brought together Eric Dolphy, drummer Roy Haynes, bassist George Duvivier, and pianist Richard Wyands. He also arranged four tracks for tenor saxophonist Eddie "Lockjaw" Davis's big-band album 'Trane Whistle (1960), including "The Stolen Moment," the first recording of the tune he would soon record under the title "Stolen Moments." In 1960 and 1961 he played tenor saxophone in the Quincy Jones orchestra, in the United States and on its European tour, and he spent brief periods with the bands of Count Basie and Duke Ellington.

The Blues and the Abstract Truth was recorded on February 23, 1961, at Rudy Van Gelder's studio in Englewood Cliffs, New Jersey, for Impulse!, with Creed Taylor producing. Its septet paired Nelson (tenor saxophone, and alto on "Teenie's Blues") with trumpeter Freddie Hubbard, Dolphy on alto saxophone and flute, baritone saxophonist George Barrow, pianist Bill Evans (his only appearance on a Nelson record), bassist Paul Chambers, and Haynes. It is built on his own pieces, most famously "Stolen Moments," a 16-bar tune with solos over a minor blues in C minor, and it made his name as a composer and arranger. In 2026 the Library of Congress selected it for preservation in the National Recording Registry. His first big-band album as a leader, Afro/American Sketches, was recorded on September 29 and November 10, 1961, and came out on Prestige in 1962.

For the rest of the decade he was one of the most requested arrangers in jazz. He wrote the orchestral settings for Jimmy Smith's Bashin' (1962, Smith's first album for Verve), Hobo Flats (1963), and Monster (1965), for the organist's 1966 meeting with guitarist Wes Montgomery, Jimmy & Wes: The Dynamic Duo, and for Montgomery's Goin' Out of My Head, released in 1966; for Cannonball Adderley's Domination (1965); for Sonny Rollins's Alfie (1966), the soundtrack to the British film; and for Thelonious Monk's Monk's Blues, recorded in Los Angeles in November 1968 with Monk's quartet and a group of studio players. He also arranged for singers including Etta Jones (So Warm, 1961, and From the Heart, 1962), Betty Carter, Nancy Wilson, Johnny Hartman, and Carmen McRae, and he wrote for Buddy Rich's 1966 big band. His own orchestra albums from these years include Full Nelson (Verve, 1963), Fantabulous (Argo, recorded March 19, 1964, in Chicago), More Blues and the Abstract Truth (Impulse!, recorded in November 1964, with Nelson arranging and conducting but not playing, and Thad Jones, Phil Woods, Ben Webster, Pepper Adams, and Roger Kellaway among the players), Sound Pieces (1966, on which he plays only soprano saxophone), and The Kennedy Dream (1967). His book of exercises, Patterns for Improvisation, was published in 1966.

In 1967 Nelson moved to Los Angeles to work in film and television and became a sought-after composer of background music. His television credits include Ironside, Night Gallery, Columbo, The Six Million Dollar Man, and Longstreet, and his film and television-movie scores include Death of a Gunfighter (1969), Skullduggery (1970), Zig Zag (1970), and The Alpha Caper (1973). He arranged Gato Barbieri's music for Last Tango in Paris (1972) and arranged and conducted James Brown's Soul on Top (1970), recorded with Louie Bellson's orchestra, and he also worked on albums for pop singers including Diana Ross and the Temptations. He kept leading jazz groups and bands: in 1969 he took a small group with saxophonists John Klemmer, Ernie Watts, and Frank Strozier and trumpeter Freddie Hill on a State Department tour of West Africa, and that summer he led a five-week clinic at Washington University, where Phil Woods, Mel Lewis, Thad Jones, Roland Hanna, and Ron Carter were guest performers and the student saxophonists included Julius Hemphill, Oliver Lake, and Hamiet Bluiett, who later co-founded the World Saxophone Quartet with David Murray.

His later records show the same range. He arranged and conducted 3-2-1-0 (Columbia Japan, 1969) and recorded Oliver Nelson in Tokyo (1970) with Nobuo Hara's Sharps & Flats; wrote Black, Brown and Beautiful (Flying Dutchman, 1970), an orchestral tribute to Martin Luther King; led a European all-star band at the Berliner Jazztage for Berlin Dialogue for Orchestra (released 1971, with Slide Hampton in the trombone section) and at the Montreux Jazz Festival for Swiss Suite (recorded in 1971, with Barbieri and Eddie "Cleanhead" Vinson as soloists); arranged and conducted Afrique (1971) for the Count Basie Orchestra; and in 1974 recorded Oliver Edward Nelson in London with Oily Rags, an album with the future duo Chas & Dave. His last studio album, Stolen Moments (East Wind), was recorded in 1975. Nelson died of a heart attack on October 28, 1975, at the age of 43.

Nelson played alto and tenor saxophone, later soprano. DownBeat's Don DeMicheal wrote in 1961 that his playing was thoughtful, unhackneyed, and well constructed, like his writing, Jack Sohmer later described his sound as singing and richly expressive, and the AllMusic critic Scott Yanow has called him a vastly underrated saxophonist; AllMusic's own biography says his writing eventually overshadowed his playing. Richard B. Hadlock heard in the Afro/American Sketches scores a taste for simple melodies with complex inner lines, close to the secret of Duke Ellington's best writing, and echoes of Gil Evans in the thick linear voicings. The blues is the basis of much of it, and the 12-bar form turns up even in pieces that stretch it: "Hoe-Down," from The Blues and the Abstract Truth, was inspired by the fifth section of Aaron Copland's Rodeo, while "Cascades" expands a minor-blues A section into a 56-bar form. His concert-music training shows in the extended orchestral works, among them the nine-minute "Sound Piece for Jazz Orchestra" and the 18-minute title piece of Berlin Dialogue for Orchestra. "Stolen Moments" became a standard recorded by Phil Woods, J.J. Johnson, Frank Zappa, Ahmad Jamal, and Booker Ervin. After 1967 the studios took most of his time, and AllMusic notes that he was largely lost to jazz for a few years before he died.