Tom Harrell was born in Urbana, Illinois on June 16, 1946 and moved with his family to the San Francisco Bay Area at five. He took up the trumpet at eight and was playing paying jobs with local bands within five years. As a teenager he studied with John Handy and Lee Konitz and talked his way onto bandstands with Philly Joe Jones, Sonny Stitt, Charles McPherson, Lee Morgan and Rahsaan Roland Kirk. He remembered the Kirk encounter precisely: he sat in at Slug's in 1966 on "Three Flowers" and "Cherokee," and Kirk told him afterwards that he had only played scales.

The same year, while he was an undergraduate at Stanford, his mental illness declared itself. Bill Milkowski's 1998 profile in JazzTimes, the fullest account Harrell has given on the record, reports that after a suicide attempt in 1966 he was diagnosed with a schizophrenic disorder and put on the psychotropic medication he has taken ever since. He finished the degree, graduating from Stanford in 1969 with a B.A. in music composition, and joined Stan Kenton's orchestra, touring and recording with the band through that year. Oliver Sacks devoted a passage of Musicophilia to him in 2007. Harrell has never treated the illness as a private matter or as an explanation for his music, and colleagues who have worked with him for decades describe the same person Milkowski found: silent and apparently withdrawn until the horn comes up.

The 1970s were spent inside other people's bands. He played in Woody Herman's Thundering Herd from 1970 to 1971, joined the Escovedo brothers' Latin-rock group Azteca in 1972 for two Columbia albums, and recorded the Peanuts television specials with Vince Guaraldi between 1972 and 1974, arranging the brass on A Charlie Brown Thanksgiving as well as playing it. He moved to New York in October 1973 and joined the Horace Silver Quintet, staying until 1977 and making five records with the band, the "Silver 'n" series that put him alongside Bob Berg, Ron Carter and Al Foster. After Silver came the Sam Jones-Tom Harrell Big Band, the Lee Konitz Nonet from 1979 to 1981, George Russell's New York Band, and the Mel Lewis Orchestra in 1981 and 1982. In August 1979 he took the trumpet chair on Bill Evans's last studio record, We Will Meet Again, which won the Grammy Award for Best Jazz Instrumental Album at the 1981 ceremony.

The association that made his name was Phil Woods's. Harrell held the trumpet chair in the Woods quintet from 1983 to 1989 and made seven records with it, from the live Integrity cut in Bologna in April 1984 to Flash in 1989, alongside Hal Galper, Steve Gilmore and Bill Goodwin. Woods gave him a standing outlet for his writing, which he has said changed how he wrote: he began putting modern harmony inside traditional forms, thirty-two-bar tunes whose phrases ran ten and six. He also began recording under his own name during those years, on Criss Cross, SteepleChase and Blackhawk, and co-led three albums with the Swiss alto saxophonist George Robert and the Italian pianist Dado Moroni.

After leaving Woods he built a leader catalogue in three distinct phases. Five albums for Contemporary between 1988 and 1991 ran from the flugelhorn-only Stories to Form, whose rhythm section was Charlie Haden and Paul Motian. Two audiophile dates for Chesky followed. Then came seven years on RCA and its associated labels from 1996 to 2003, the period of the large ensembles: The Art of Rhythm in 1998 with strings, double reeds and three percussion sections; the seventeen-piece Time's Mirror in 1999, which was nominated for Best Large Jazz Ensemble Performance at the 42nd Grammy Awards and reached number sixteen on the Billboard jazz chart; and Wise Children in 2003, with brass choir, string quartet and West African percussion. Through the same decade he led working quintets: Don Braden, Kenny Werner, Larry Grenadier and Billy Hart from 1994 to 1996, then Jimmy Greene, Xavier Davis, Ugonna Okegwo and Quincy Davis from 2000 to 2005.

From 2007 he has recorded almost exclusively for HighNote, and the first five of those albums document one band: Wayne Escoffery on tenor, Danny Grissett on piano, Okegwo on bass and Johnathan Blake on drums, from Light On, cut at the Van Gelder studio in December 2006, through Number Five in 2012, which brought him his seventh SESAC Jazz Award. He then spent a decade taking the band apart and rebuilding it in different shapes. A nine-piece chamber ensemble debuted at the Highline Ballroom in June 2012 as part of the Blue Note Jazz Festival, playing his arrangements of Debussy and Ravel for trumpet, two saxophones, flutes, violin, cello, guitar, piano, bass and drums; the resulting record, First Impressions, came out in 2015. The piano-less sextet Colors of a Dream, with two basses and Esperanza Spalding singing as well as playing one of them, opened at the Village Vanguard on March 26, 2013, and the second night was broadcast live by NPR and WBGO. TRIP, a piano-less quartet with Mark Turner, Okegwo and Adam Cruz, premiered a six-part suite at Dave Douglas's Festival of New Trumpet Music in October 2012 and recorded the following year. Since then have come the two-trumpet Something Gold, Something Blue with Ambrose Akinmusire in 2016, the guitar-and-tenor quintet Infinity in 2019, and two quartet records with Luis Perdomo, Okegwo and Cruz, Oak Tree in 2022 and Alternate Summer in 2024.

The awards have accumulated steadily rather than in a rush: multiple Trumpeter of the Year wins in the DownBeat critics and readers polls, frequent placings in the JazzTimes polls, eight SESAC Jazz Awards, a BMI Composers Award, the Prix Oscar du Jazz, and the Jazz Journalists Association's Trumpeter of the Year for 2018. He is also a working arranger for other people's ensembles, having written for the Metropole Orchestra, the Danish Radio Big Band, the Vanguard Jazz Orchestra, Carlos Santana, and Elisabeth Kontomanou with the Orchestre National de Lorraine. His composition and big-band arrangement "Humility" is the seventh track of Arturo O'Farrill and the Afro-Latin Jazz Orchestra's Song for Chico, which won the Grammy Award for Best Latin Jazz Album. His solos and tunes are published by Hal Leonard, Jamey Aebersold, Sher Music and Gerard and Sarzin, and his pieces have been recorded by Ron Carter, Kenny Barron, Art Farmer, Chris Potter, Tom Scott, Steve Kuhn, Kenny Werner and Hank Jones.

Harrell's line of descent runs through the trumpet players he named himself: Louis Armstrong first, then Roy Eldridge, Dizzy Gillespie, and Clifford Brown, whom he heard on the radio in the eighth grade and set about absorbing, followed by Blue Mitchell and Clark Terry. He also transcribed saxophone solos as a matter of course, which is audible in the length and evenness of his phrases; he has said the two instruments are intertwined. The sound is dark for a trumpet, closer to the flugelhorn than most players get, and on several records, Stories among them, he puts the trumpet down entirely. What separates him from other post-bop trumpeters of his generation is that the writing carries equal weight with the playing: he thinks in whole records, and the instrumentation changes from one to the next as the material demands, from a two-trumpet quintet with no reeds to a nine-piece chamber group with violin, cello and bass flute. Joe Lovano, who played on six of his leader dates, told JazzTimes that Harrell "really knows his craft," and located it in what he had played before he wrote: Horace Silver's book, Charlie Parker's, Benny Golson's, and outward from there to Stravinsky and Bartok. The saxophonist Greg Tardy, who was on The Art of Rhythm, called him "one of the heaviest writers I know." Phil Woods and Horace Silver both used the word genius about him in print. The harmonic ear that produces the writing is not something he switches off: in the same 1998 interview he heard a car climbing a hill as a dominant seventh with an added ninth, his refrigerator as a B flat pedal, and water running into a steel sink as a chord from The Rite of Spring.