Grant Green was born June 6, 1935, in St. Louis, Missouri, to John and Martha Green. His father, who worked at various times as a laborer and as a St. Louis police officer, played blues and folk guitar and gave him his first lessons; Green studied for a year with a teacher named Forrest Alcorn but was otherwise self-taught, learning by ear off records. He made his first professional appearance at 13 in a gospel group, then spent his twenties working in St. Louis jazz and R&B bands, absorbing Charlie Christian's guitar recordings alongside saxophonists Charlie Parker and Lester Young and guitarist Jimmy Raney. His first recordings came in 1959, backing tenor saxophonist Jimmy Forrest on the St. Louis-based United label with drummer Elvin Jones in the group.

In 1959, alto saxophonist Lou Donaldson heard Green playing at a St. Louis bar, hired him for his touring band, and brought him to New York, where Green relocated in 1959-60. Donaldson introduced him to Blue Note Records founder Alfred Lion, and from 1961 to 1965 Green served as the label's house guitarist, appearing on more Blue Note sessions as leader or sideman than any other musician during those years. Rather than confine him to sideman work, Lion had Green record as a leader almost immediately; his actual first session, cut November 16, 1960, and October 27, 1961, went unissued at the time because Green doubted it was strong enough, surfacing only in 2001 as First Session. His first released dates as a leader came in quick succession in 1961: Grant's First Stand (recorded January 28, an organ trio with "Baby Face" Willette and drummer Ben Dixon), Green Street (April 1, a trio with bassist Ben Tucker and drummer Dave Bailey), and Grantstand (August 1, a quartet with Yusef Lateef on tenor and flute, organist Jack McDuff, and drummer Al Harewood). DownBeat's critics named him Best New Star in 1962.

Across the early-to-mid 1960s Green split his time between his own dates and heavy sideman duty, recording with organists Big John Patton and Larry Young, saxophonists Hank Mobley, Ike Quebec, and Stanley Turrentine, and pianist Herbie Hancock, among dozens of others, in a working relationship with Blue Note's house rhythm sections and Rudy Van Gelder's studio that made him one of the most heavily documented musicians on the label. He built a run of concept albums around a single musical theme: Sunday Mornin' (recorded June 1961, gospel), The Latin Bit (recorded April 1962, with congas from Carlos "Patato" Valdes and drummer Willie Bobo), and Feelin' the Spirit (recorded December 1962, spirituals, with a young Herbie Hancock on piano). Idle Moments, recorded in November 1963 with Joe Henderson, Bobby Hutcherson, and pianist Duke Pearson, and not released until 1965, is widely regarded as the finest record of his career. Two 1964 sessions with pianist McCoy Tyner and drummer Elvin Jones on loan from John Coltrane's working quartet, Solid and Matador, sat unissued for fifteen years before Blue Note put them out in 1979, the latter first only in Japan.

Green left Blue Note in 1966 and recorded for other labels, including a 1965 Verve date, His Majesty King Funk, produced by Creed Taylor with organist Larry Young and conguero Candido. From 1967 to 1969 he was largely inactive, sidelined by personal problems and heroin addiction. He returned to Blue Note in 1969 in a harder funk and soul-jazz direction geared to a wider R&B audience: Carryin' On (1970) and the commercially successful Green Is Beautiful (1970) both paired him with drummer Idris Muhammad and saxophonist Claude Bartee, while Alive! (1971) captured his working band live at the Cliché Lounge in Newark, New Jersey. He scored the blaxploitation film The Final Comedown (1972) and cut the double live album Live at the Lighthouse (1972) at Hermosa Beach, California, followed by The Main Attraction (1976), a Creed Taylor-produced Kudu date with guitarist Steve Khan and flutist Hubert Laws in the band.

Green spent most of 1978 hospitalized, and against his doctors' advice went back on the road afterward to keep up with medical bills and child support; that year's Easy, cut with saxophonist Hank Crawford and bassist Buster Williams, was his last studio album. On January 31, 1979, in New York City for an engagement at guitarist George Benson's club, the Breezin' Lounge, Green collapsed in his car and died of a heart attack at age 43. He was buried in Greenwood Cemetery in his native St. Louis; his grave went unmarked until 2017, when the Killer Blues Headstone Project placed a headstone. He was survived by six children, including guitarist Grant Green Jr.

Green approached the guitar the way a horn player approaches a solo: single melodic lines built from the vocabulary of Charlie Parker and Lester Young rather than the block chords and dense voicings that defined most jazz guitar of his generation. Critic Michael Erlewine called him "one of the great unsung heroes of jazz guitar," and jazz educator Wolf Marshall credited his "unique mixture of bebop, blues and funk" with making him one of the defining soul-jazz and hard-bop guitarists of the 1960s. That directness, avoiding chromatic complexity in favor of clean, blues-rooted lines, suited the organ-trio format he worked in constantly, and gave his hundreds of Blue Note sides an unmistakable, immediately recognizable sound whether he was backing a saxophonist or leading his own date. He achieved his signature punchy, midrange-heavy tone, according to his protégé George Benson, by turning down the bass and treble on his amplifier rather than boosting anything. His influence reaches well past his own era: he has since become one of the most sampled guitarists in hip-hop and acid jazz production, and later mainstream-jazz guitarists continue to cite him as a foundational reference point alongside Wes Montgomery, Joe Pass, Jim Hall, and Kenny Burrell; guitarist Roni Ben-Hur names all five in the same breath as his own formative guitar influences.