Henry Alonzo Grimes was born November 3, 1935, in Philadelphia, Pennsylvania, into a musical household: his father, Leon James Grimes Sr., played trumpet, his mother Georgia Elzie Grimes played piano, and his twin brother Leon Grimes became a clarinetist and tenor saxophonist. Grimes started on violin at twelve, then moved through tuba, English horn, and percussion before settling on double bass at Mastbaum Technical High School. He went on to study at Juilliard, arriving in New York with a grounding broad enough to work in almost any jazz idiom.

By the mid-1950s Grimes was one of the city's most in-demand young bassists, working with pianist Lennie Tristano and saxophonists Lee Konitz and Gerry Mulligan (documented on Mulligan's 1957-58 Reunion with Chet Baker sessions), and with Sonny Rollins, appearing on 1958's big-band date Sonny Rollins and the Big Brass alongside Clark Terry and Roy Haynes. He also passed through the bands of Thelonious Monk and clarinetist Benny Goodman. At twenty-two he turned up in the 1958 Newport Jazz Festival documentary Jazz on a Summer's Day, performing that single weekend with six different groups, a versatility a New York Times critic singled out as "remarkable." Charles Mingus, always partial to unconventional bass textures, picked Grimes for stretches when he wanted a second bassist alongside him in the Jazz Workshop.

The early 1960s pulled Grimes toward the emerging avant-garde. He is the bassist on McCoy Tyner's Reaching Fourth (recorded November 1962, Impulse!, with Roy Haynes on drums) and on Roy Haynes's own Out of the Afternoon (1962, Impulse!, with Roland Kirk), and on Sonny Rollins's 1963 RCA date Sonny Meets Hawk!, a summit with Coleman Hawkins that also featured pianist Paul Bley. From there he moved fully into free jazz, becoming one of the music's most sought-after bassists specifically because he could hold his own in the era's dual-bass frontlines: he shares the bass chair with Earle Henderson on Albert Ayler's Spirits (1964), with Gary Peacock on Ayler's Spirits Rejoice (1965, ESP-Disk), and with Lewis Worrell on drummer Sunny Murray's Sonny's Time Now (1966, ESP-Disk, featuring both Ayler and Don Cherry). He appears on Archie Shepp's On This Night (1965, Impulse!); on Cherry's Complete Communion (recorded 1965), Symphony for Improvisers (1966, with Pharoah Sanders), and Where Is Brooklyn? (recorded 1966, also with Sanders), all on Blue Note; and, alongside a second bassist, Alan Silva, on both of Cecil Taylor's landmark Blue Note albums, Unit Structures (recorded May 19, 1966) and Conquistador! (recorded that October). The same year he cut sideman dates for two ESP-Disk leaders, the Frank Wright Trio and the Charles Tyler Ensemble, and made his sole record as a leader before his disappearance: The Call (ESP-Disk, 1966), a trio with clarinetist Perry Robinson and drummer Tom Price.

Then, in the late 1960s, Grimes vanished. He had relocated to Los Angeles for work and, unable to find enough of it, sold his bass and drifted out of contact with the jazz world entirely. Musicians and reference books alike came to assume he had died; in fact he spent more than three decades in a small Los Angeles apartment, writing poetry and taking odd jobs to get by, unaware even that Albert Ayler had died in 1970.

In 2002, social worker and jazz fan Marshall Marrotte tracked Grimes down, alive, still in that Los Angeles apartment. Bassist William Parker donated an instrument, a green-tinted bass Grimes nicknamed "Olive Oil," which luthier David Gage helped ship east; Grimes returned to New York in 2003 to national press coverage, including pieces in the New York Times and on NPR, and immediately went back to work, playing more than two dozen festivals that year alone. He resumed his old partnership with Cecil Taylor in October 2006, forty years after Unit Structures, and recorded steadily with a new generation of players: Marc Ribot, trumpeter Roy Campbell, and drummer Chad Taylor on Spiritual Unity (2005, Pi Recordings, a tribute to Ayler); a duo with Rashied Ali that yielded Going to the Ritual (2008) and, after Ali's 2009 passing, the posthumously released Spirits Aloft (2010), both on Porter Records; a working trio with saxophonist Roberto Pettinato and drummer Tyshawn Sorey that became a quartet with pianist Dave Burrell (My Head Is My Only House Unless It Rains, 2011; Purity, 2012; Same Egg, 2013); and the trio Us Free with tenor saxophonist Bill McHenry and drummer Andrew Cyrille (Fish Stories, 2014). He also took up the violin again, making his professional debut on the instrument at seventy alongside Cecil Taylor at Lincoln Center, published a poetry collection, Signs Along the Road (2007), and taught at City College of New York, Berklee College of Music, and the University of Michigan. Between 2003 and 2016 he played more than 640 concerts in some thirty countries.

Henry Grimes passed away on April 15, 2020, at the Northern Manhattan Rehabilitation and Nursing Center in Harlem, of complications from COVID-19. He was 84. His wife, Margaret Davis Grimes, confirmed the details to the Jazz Foundation of America, which had been assisting with his care.

Grimes came up as a supple, harmonically fluent mainstream bassist, capable enough to satisfy Gerry Mulligan and Sonny Rollins in the same years he was working with Thelonious Monk; it was that same adaptability that made him one of the most trusted bassists in the mid-1960s New York loft scene, where he was repeatedly paired with a second bassist rather than left to hold time alone, as on Albert Ayler's Spirits Rejoice (alongside Gary Peacock) and Cecil Taylor's Unit Structures and Conquistador! (alongside Alan Silva). His comeback-era playing, after more than three decades away from an instrument, was in some ways a second, self-taught vocabulary: dense arco textures, an increasingly prominent use of the violin he had first studied as a child, and a physical, percussive attack that critics and collaborators alike distinguished from a simple repetition of his pre-1968 style. His thirty-five-year disappearance and return, culminating in a resumed partnership with Taylor and duo records with Rashied Ali, became one of the most widely told stories of resilience in jazz.