Pierre Michelot was born in Saint-Denis, just north of Paris, on March 3, 1928. He studied piano from 1936 to 1938 and took up the double bass at sixteen. After the Second World War he played for American troops stationed in France, and his clear tone and steady sense of time spread quickly among the American jazz musicians passing through Paris. By the late 1940s he was on Paris studio sessions with Coleman Hawkins, Sidney Bechet and Rex Stewart. He played with Stewart in 1948, worked at Frisco's in Paris with the drummer Kenny Clarke in the summer of 1949, and joined Clarke in the band that accompanied Hawkins that winter. In 1949 he also recorded with Clarke in an ad hoc band led by Bechet.

Through the 1950s he was a bassist American visitors asked for. He held long residencies at the Club Saint-Germain, the Ringside and the Blue Note, often beside Clarke, and recorded with bop musicians including Lester Young, Dizzy Gillespie, Bud Powell and Zoot Sims. In 1953 he played on the last session recorded by Django Reinhardt, and that October he was the bassist on Clifford Brown's Paris quartet session. He had recorded with Gillespie in Paris on April 11, 1952. He also worked with Stéphane Grappelli, Stan Getz and Don Byas and, according to AllMusic, Thelonious Monk, and he arranged music for Chet Baker's Paris sessions for Barclay in 1955 and 1956. The pianist René Urtreger told the Associated Press that Michelot was the best jazz bassist in Europe in the second half of the 1950s.

In December 1957 Miles Davis, in Paris to write music for Louis Malle's first feature, Ascenseur pour l'échafaud, chose Michelot for the quintet that improvised the score to the film's rushes, with Urtreger on piano, the tenor saxophonist Barney Wilen and Clarke on drums. The soundtrack album appeared on Fontana in 1958 and is the record many listeners know him from. In 1959 the pianist Jacques Loussier formed the Play Bach trio with Michelot and the drummer Christian Garros. Their jazz readings of Bach were a commercial success, produced two sequels and kept the trio working, off and on, for about fifteen years.

In 1960 he formed HUM with Urtreger and the drummer Daniel Humair, a piano trio whose first record, Hum !, was made with an audience at the Club Saint-Germain-des-Prés in September 1960 and which reunited for later albums in 1979 and 1999. A live set from the Blue Note club with the pianist Maurice Vander and Clarke, recorded in 1959, was issued by Fontana in 1961 as Jazz at the Blue Note, and in July 1963 he recorded his own Round About a Bass for Mercury. The same year he won the Prix Django-Reinhardt of the Académie du Jazz, and played on Dexter Gordon's Blue Note album Our Man in Paris with Powell and Clarke.

From the early 1960s he spent about two decades mostly on arranging and studio work. He accompanied the singer Claude Nougaro over several years, often with Vander, and French reference pages list him among the musicians on the scores of Beau-père (1981), La Scarlatine (1983) and Une femme de ménage (2002). In 1986 he appeared as himself in Bertrand Tavernier's Round Midnight, in which Dexter Gordon starred, and he toured later with Gordon. After a 1990 stay at New York's Village Vanguard with the guitarist Christian Escoudé he cut back his schedule, though he made Viaggio (Dreyfus, 1993) with the accordionist Richard Galliano and kept playing with HUM at the end of the decade.

Alzheimer's disease forced him off the stage. Pierre Michelot passed away in Paris on July 3, 2005, at the age of 77, as Urtreger told the Associated Press.

Michelot's reputation rests on his sound and his time. AllMusic credits him with a natural, clear tone and impeccable rhythmic sense, which it says spread his name quickly among American musicians after the war, and Urtreger described the sound as magnificent, deep and true. AllMusic classes him under bop and hard bop, and calls him widely considered the premier European jazz bassist of his generation.

His range is wide for a bassist who never stopped being a jazz player: he is on records with Hawkins, Bechet, Reinhardt and Davis, on the Loussier trio's Bach, on film scores and on chanson albums with Nougaro. His own albums are mostly trio and quartet dates built around a pianist, among them Vander, Al Haig and Walter Davis, with Clarke or Humair on drums. Published sources describe how he played rather than naming his teachers or models.