Joseph Rudolph Jones was born July 15, 1923, in Philadelphia, into a family where music was a household requirement: his maternal grandmother insisted that all of her daughters study an instrument, and Jones later credited that rule with putting him on the path he took. As a boy he worked as a featured tap dancer on The Kiddie Show, broadcast on the Philadelphia radio station WIP. He served in the U.S. Army during the Second World War (several biographical accounts put the stretch at 1941 to 1947), and after his discharge he took a job as a Philadelphia trolley car driver, one of the first African American men hired to do it. The wages bought his first drum kit, and the route put him in the way of drummers passing through the city, including Max Roach and Art Blakey.

The "Philly" in front of his name was a piece of practical bookkeeping, and it is worth stating plainly because listeners still confuse the two men. Jo Jones, the drummer who powered the Count Basie Orchestra rhythm section from 1934 to 1948 and was widely called "Papa Jo," was already famous when the younger Joe Jones reached New York. Clarinetist Tony Scott took to introducing his new drummer from the bandstand as the Joe Jones from Philly, and the shortened version stuck hard enough that Jones used it professionally for the rest of his life. They were not related, not from the same generation, and not remotely alike behind a kit.

Jones moved to New York in 1947 and became the house drummer at Café Society, where he worked alongside the leading bebop players of the day, Tadd Dameron among them. He also spent time in Joe Morris's rhythm-and-blues band, a group that at various points carried Johnny Griffin, Elmo Hope, and Percy Heath. His first documented recording sessions date to 1948 and 1949 with the Morris orchestra; by 1953 he was in studios with Miles Davis (January 30), with Clifford Brown and Lou Donaldson for Blue Note (June 9), with Tadd Dameron's orchestra (June 11), and with Elmo Hope (June 18). Some accounts have him with Duke Ellington's orchestra briefly in 1952, a job he gave up because he preferred staying in New York and taking studio calls. To sharpen his reading for that work he studied with Cozy Cole, having earlier picked up brush technique directly from Sidney Catlett.

In the summer of 1955, after his Newport Jazz Festival appearance won him a Columbia contract, Miles Davis assembled a working band for an engagement at the Café Bohemia: Sonny Rollins on tenor, Red Garland on piano, Paul Chambers on bass, and Jones on drums. Rollins left that autumn, and on Jones's recommendation Davis brought in John Coltrane. What followed is the most documented rhythm section of the decade. The quintet cut the two 1956 marathon dates for Prestige that produced Cookin', Relaxin', Workin' and Steamin', then 'Round About Midnight for Columbia in 1957, and in February and March 1958 the sextet session (with Cannonball Adderley added on alto) that became Milestones. Davis called Jones his favorite drummer and wrote in his autobiography that he listened for Jones in every drummer he heard afterward, describing him as the fire that made things happen and noting that Jones anticipated what he was about to play. The relationship was not frictionless. Davis let Jones and Coltrane go in 1957 over the unreliability that came with heroin, and by mid-1958 Jimmy Cobb had taken the drum chair for good. Jones eventually got clean, and he was blunt in later interviews about the cost, saying he did not think he had played badly while using but believed he would have played better without it.

Across the same stretch Jones became one of the busiest studio drummers in jazz, working constantly for Prestige, Riverside and Blue Note. The short list of what he played on in three years includes Coltrane's Blue Train (September 15, 1957), Sonny Clark's Cool Struttin' (January 5, 1958), Sonny Rollins's Tenor Madness (May 24, 1956) and Newk's Time (September 22, 1957), Tadd Dameron's Mating Call with Coltrane (November 30, 1956), Art Pepper's Art Pepper Meets the Rhythm Section (January 19, 1957), Bill Evans's Everybody Digs Bill Evans (December 15, 1958), and Cannonball Adderley's Portrait of Cannonball (July 1, 1958). His own leader career began on September 17, 1958, with Blues for Dracula for Riverside, a sextet date with cornetist Nat Adderley, trombonist Julian Priester, Johnny Griffin, pianist Tommy Flanagan and bassist Jimmy Garrison. The title track opens with Jones delivering a deadpan vampire monologue, not an impression of Bela Lugosi so much as an impression of Lenny Bruce's impression of Bela Lugosi, a routine Bruce reportedly enjoyed enough to fold back into his own act. Two more Riverside albums followed quickly: Drums Around the World, cut across three May 1959 dates with a small orchestra billed as Philly Joe Jones' Big Band Sounds and including Cannonball Adderley, Lee Morgan, Curtis Fuller, Benny Golson, Herbie Mann and Wynton Kelly; and Showcase, recorded November 17 and 18, 1959, with Pepper Adams on baritone and Jones himself sitting down at the piano for part of the session. He moved to Atlantic for Philly Joe's Beat (May 20, 1960) and for a two-drummer summit with Elvin Jones, Together!, taped February 2, 1961 but held back until 1964.

Between late 1967 and 1972 Jones lived in London and Paris. For two years, 1967 to 1969, he taught at a school organized specifically around him in Hampstead, north London, while the British Musicians' Union blocked him from taking paid performing work in the country. One of the drummers who sought him out in England was Rashied Ali, a fellow Philadelphian a decade his junior who had just come off John Coltrane's last band and was spending time in Europe. Jones did record in London: Mo' Joe, also issued as Trailways Express, was cut on October 1 and 31, 1968 with a British and Canadian cast including Peter King, Harold McNair, Chris Pyne and Kenny Wheeler. In Paris he made an album with the Jef Gilson Ensemble (Vogue, 1969) and, that November and December, a live co-billed date with Archie Shepp for the America label whose band also held Anthony Braxton and violinist Leroy Jenkins, about as far from the Miles Davis Quintet as he ever travelled.

Back in Philadelphia from 1972, Jones led a fusion group called Le Grand Prix for roughly three years, then returned to the acoustic mainstream. He toured with Bill Evans in 1976 and 1978 and appears on Evans's Quintessence (Fantasy, 1976); he reunited with Red Garland for Keystones! (Xanadu) and Crossings (Galaxy) in 1977 and 1978. Galaxy also gave him the most substantial run of leader dates of his later career: Philly Mignon, recorded November 29 and 30 and December 1, 1977 with Dexter Gordon, Ira Sullivan, Nat Adderley, George Cables and Ron Carter; and a three-day session at Fantasy Studios in Berkeley, October 10 to 12, 1978, that yielded two albums, Advance! (issued 1979) and Drum Song (held until 1985), both with Blue Mitchell, Slide Hampton, Harold Land, Cedar Walton and Marc Johnson.

The last chapter was a repertory project he had wanted for years. In 1981 Jones and trumpeter-arranger Don Sickler founded Dameronia, a nonet built to play Tadd Dameron's compositions and arrangements as Dameron actually wrote them, and staffed in part with musicians who had worked for him: Johnny Coles, Britt Woodman, Cecil Payne, Frank Wess, Charles Davis, Walter Davis Jr. and bassist Larry Ridley. The band recorded To Tadd with Love at Nola Recording Studios on June 28, 1982, and Look Stop Listen, with Johnny Griffin added out front, at Rudy Van Gelder's Englewood Cliffs studio on July 11, 1983. Jones led it until the end of his life, still recording in 1985 with Clifford Jordan (The Rotterdam Session), on Bobby Hutcherson's Good Bait, and on the Manhattan Transfer's Vocalese. He suffered a fatal heart attack in Philadelphia on August 30, 1985, six weeks after his sixty-second birthday. Dameronia carried on without him, with Kenny Washington in the drum chair. The Philadelphia Music Alliance added Jones to its Walk of Fame in 1996, and DownBeat's Veterans Committee elected him to the magazine's Hall of Fame in 2010.

Jones is the drummer who defined what hard bop time felt like, and his signature is identifiable inside two bars: a hard, forward-leaning ride cymbal, an unusually loud and conversational snare, and the cross-stick rimshot accent on the fourth beat that drummers still call the "Philly lick," heard most famously threaded through Miles Davis's Milestones. Art Blakey used the figure first, but Jones made it structural. He came up out of the swing drummers rather than around them, taking brush technique from Sidney Catlett in person and citing Baby Dodds, Chick Webb, Jo Jones, Denzil Best and Dave Tough, then turning rudimental snare vocabulary into melodic solo lines in a way that the bebop generation had not quite managed. That combination made him the pivot between the swing masters and the Elvin Jones and Tony Williams school that displaced him, and it made him a first-call studio player: he appeared on a figure commonly given as 500 or more albums, and his Discogs credit log runs to over 1,200 releases. Both of the bandleaders who used him most were unusually direct about it. Davis named him his favorite drummer outright, and Bill Evans, whose trios were built on the opposite of Jones's volume, is reported to have said the same.