John Laird Abercrombie was born on December 16, 1944, in Port Chester, New York, and grew up in Greenwich, Connecticut. He started guitar lessons at ten, and his first heroes were rock and roll and rhythm and blues players (Chuck Berry, Fats Domino, Bill Haley and the Comets) before two friends with a large jazz collection introduced him to Dave Brubeck and Miles Davis and, on guitar, Barney Kessel. At the Berklee College of Music in Boston, where he graduated in 1967, he fell hardest for Jim Hall and Wes Montgomery. He played at Paul's Mall, the room attached to the Jazz Workshop, where he met Michael Brecker, Randy Brecker and the organist Johnny "Hammond" Smith, who took him on tour. He also shared a room at Berklee with the keyboardist Jan Hammer. After a spell at North Texas State University he moved to New York City in 1969.

Work came quickly. He joined vibraphonist Monty Stark's band Stark Reality, which recorded the album Discovers Hoagy Carmichael's Music Shop (1970), and in 1969 he became the guitarist in the jazz-rock band Dreams with the Brecker brothers and Billy Cobham (their self-titled Columbia album appeared in 1970). Through the early 1970s he was a first-call New York sideman: with pianist Barry Miles on White Heat (1971) and Scatbird (1972), with Chico Hamilton, with Gato Barbieri on the live album Bolivia (1973), with Dave Liebman on Lookout Farm (recorded October 1973) and with Gil Evans on the orchestra's Jimi Hendrix album of 1974. In 1974 he joined Cobham's band and appeared on Crosswinds and Total Eclipse, then on the live Shabazz. The jazz-rock circuit made his name but not his music; he later recalled playing the Spectrum in Philadelphia and wondering what he was doing there.

The way out came from Jack DeJohnette and from Manfred Eicher, the founder of ECM Records. Abercrombie recorded his debut as a leader, Timeless, with DeJohnette and Hammer in New York on June 21 and 22, 1974. In March 1975 he, DeJohnette and Dave Holland recorded the first album of the collective trio Gateway in Ludwigsburg, and that same month the three also backed Collin Walcott on Cloud Dance. A duo album with Ralph Towner, Sargasso Sea (1976), followed, then Gateway 2 (recorded July 1977), the solo guitar and mandolin record Characters (recorded November 1977) and DeJohnette's New Directions band with Lester Bowie and Eddie Gomez. Between December 1978 and November 1980 he recorded three albums for ECM with his first steady quartet, with pianist Richie Beirach, bassist George Mraz and drummer Peter Donald: Arcade, Abercrombie Quartet and M. He later called it his first opportunity to be a leader and to write consistently for the same musicians. The ECM association that began with Timeless lasted for the rest of his life.

In the mid-1980s he took up the guitar synthesizer, using it from 1984 to 1990 in a trio with Marc Johnson and Peter Erskine that recorded Current Events (recorded September 1985), Getting There (recorded April 1987, with Michael Brecker as guest on three tracks) and a live album taped at the Nightstage in Boston in April 1988. He also made Night (1984) with Brecker, Hammer and DeJohnette, the Palo Alto album Solar with John Scofield, a duo disc with Beirach, and Animato (recorded October 1989) with the composer Vince Mendoza and drummer Jon Christensen. In the early 1990s he returned to a plainer sound and to the organ trio, forming a group with organist Dan Wall and drummer Adam Nussbaum that made While We're Young, Speak of the Devil, Tactics (taped live at Visiones in New York in July 1996) and, with Kenny Wheeler, Joe Lovano and violinist Mark Feldman added, Open Land (recorded September 1998). Gateway regrouped for Homecoming (1995) and In the Moment (1996), and between 1999 and 2002 he played on four ECM albums by Charles Lloyd. At some point in the 1990s he gave up the guitar pick and played with his thumb, which gave his tone a fuller, rounder sound.

His last decades centered on two quartets. The first, with Feldman, Marc Johnson and drummer Joey Baron, made Cat 'n' Mouse (recorded December 2000), Class Trip (2004), The Third Quartet (2007) and, with Thomas Morgan in place of Johnson, Wait Till You See Her (2009). The second paired Baron and bassist Drew Gress with either Lovano (Within a Song, 2012) or pianist Marc Copland (39 Steps, 2013, and Up and Coming, recorded in April and May 2016). In 2003 a fire destroyed his home in Putnam Valley, New York, along with a dozen or so guitars and his collection of recordings, and he kept working. He had a stroke early in 2017, and Up and Coming, released on January 13, 2017, proved to be his last album. He passed away of heart failure on August 22, 2017, in Cortlandt Manor, New York, at 72. His wife, Lisa, whom he married in 1986, survived him. JazzTimes counted more than sixty albums as a leader or co-leader.

Abercrombie had one of the most recognizable sounds in ECM's catalogue: a fluid, silvery line, often sparse and a little angular, with a warm tone that critics traced to Jim Hall and Wes Montgomery. Ornette Coleman, Jimi Hendrix and the lyricism of Bill Evans mattered as much to him, and he said he built his early jazz-rock approach out of necessity because he had few guitar models to follow. He came up just after John McLaughlin and Sonny Sharrock but belonged temperamentally to the generation of Pat Metheny, John Scofield and Bill Frisell, and Scofield has said Abercrombie showed him the way when he first moved to New York in the mid-1970s. His music moved easily between acoustic clarity, the synthetic glow of the guitar synthesizer, free playing and standards, and he valued space, both in his comping and in his tune writing. Dry humor ran through his playing and his conversation. Jack DeJohnette praised his sensitivity, dynamics and use of space, and said he deserved more recognition than he got; NPR called him both revered and somehow overshadowed. Timeless has since become a touchstone, and its tracks have been sampled by hip-hop and electronic producers.