Terje Rypdal was born on August 23, 1947, in Oslo, the son of a military-band conductor and clarinetist. He began on piano at five, added trumpet at eight, and at about thirteen turned to the guitar, which he taught himself. From 1962 to 1967 he played in the Vanguards, a Norwegian instrumental rock band modeled on the Shadows and the Ventures, where he was a Hank Marvin-influenced rock guitarist. Hearing Jimi Hendrix changed his direction, and in late 1967 he started the psychedelic band Dream, which made one album for Polydor, Get Dreamy, and where he met Jan Garbarek and Jon Christensen. Under his own name he made Bleak House for Polydor in Norway in 1968, a first mix of rock, classical writing and jazz, and in 1968 he turned toward jazz in earnest, joining Garbarek's group and later the sextet and orchestra of the composer George Russell.

His first international notice came at the 1969 Free Jazz Meeting in Baden-Baden, where he played in a large band conducted by trumpeter Lester Bowie and led a group of his own; both are heard on the MPS album Gittin' to Know Y'All. He studied musicology at the University of Oslo and attended the Oslo conservatory from 1970 to 1972, working with the composers Finn Mortensen and Russell, and while studying he led the orchestra for the Norwegian production of the musical Hair. He was the guitarist on Garbarek's ECM debut Afric Pepperbird (recorded September 1970) and on Sart (April 1971), and he made his own ECM debut, Terje Rypdal, on August 12 and 13, 1971, with Garbarek, pianist Bobo Stenson, bassist Arild Andersen, Jon Christensen, oboist Ekkehard Fintl and his then wife, the singer Inger Lise Rypdal. Manfred Eicher, the label's founder, produced it and most of what followed. His own albums for ECM in the next five years were What Comes After (recorded August 1973), Whenever I Seem to Be Far Away (1974, one side for a rhythm section and one for the Stuttgart radio orchestra), and the double album Odyssey (recorded August 1975), made with a touring quintet of trombonist Torbjørn Sunde, organist Brynjulf Blix, bassist Sveinung Hovensjø and drummer Svein Christiansen. Odyssey won the German record critics' prize, and the band toured widely, including in the United States. After the Rain (1976) was a quieter, largely solo studio record.

From 1977 he worked closely with the Danish trumpeter Palle Mikkelborg, who joined him on Waves (recorded September 1977) and on the trio album Descendre (recorded March 1979, with Jon Christensen). Between them came Terje Rypdal / Miroslav Vitous / Jack DeJohnette, recorded in June 1978, and a second trio set by the same three, To Be Continued (January 1981). After a long break for composing he returned with Eos (recorded May 1983) in duet with the cellist David Darling and then formed the trio he called the Chasers with bassist Bjørn Kjellemyr and drummer Audun Kleive: Chaser (1985), Blue (1987) and The Singles Collection (1989, with keyboardist Allan Dangerfield added). Two pieces from that period, "Mystery Man" and "Last Nite," were used in Michael Mann's film Heat (1995).

Alongside the jazz records he built a parallel career as a composer for orchestra, choir and chamber ensemble. His Symphony No. 1 was commissioned by Norwegian Television in 1975, and German Wikipedia counts six symphonies and two operas among his works. Undisonus / Ineo (released 1990), for violin and orchestra and for choir and chamber orchestra, was named work of the year by the Society of Norwegian Composers, and Q.E.D. (1993, with the Borealis Ensemble under Christian Eggen) followed. He then moved between the two worlds on ECM: the jazz-rooted If Mountains Could Sing (1995) and Skywards (1997, with Mikkelborg, violinist Terje Tønnesen, David Darling and two drummers, Jon Christensen and Paolo Vinaccia), then Double Concerto / 5th Symphony (2000) and Lux Aeterna (recorded July 19, 2000, a five-movement work for soprano, chamber ensemble, organ, trumpet and guitar). Along the way he recorded with John Surman and Karin Krog on Nordic Quartet (1995), with Ketil Bjørnstad and Darling on The Sea (1995) and The Sea II (1998), and with Markus Stockhausen on Karta (2000), and he made three duo albums with the Norwegian rock guitarist Ronni Le Tekrø starting in 1994.

Vossabrygg, recorded live at the Vossajazz festival on April 12, 2003 and released in 2006, brought a sextet with Mikkelborg, keyboardists Bugge Wesseltoft and Ståle Storløkken and his son Marius Rypdal on electronics. Crime Scene (recorded May 2009, released April 16, 2010) set a film-noir suite for his quartet and the Bergen Big Band, and Melodic Warrior (2013) paired two large works: the nine-movement title piece for electric guitar, the Hilliard Ensemble and the Bruckner Orchester Linz under Dennis Russell Davies, and And the Sky Was Coloured with Waterfalls and Angels with the Wrocław Symphony Orchestra. After seven years of composing and guesting on other people's records he returned to the studio for Conspiracy (ECM 2658, September 2020), his first studio recording in two decades, with Storløkken, drummer Pål Thowsen and bassist Endre Hareide Hallre. In October 2024 the Norwegian label Rune Grammofon issued Catching Fire, a double album of a 2017 concert with the trio Elephant9.

He received the Spellemannprisen's special award in 1981, the Norwegian Jazz Forum's Buddy Prize in 1985, the jazz-category Gammleng Prize in 1990, the Spellemannprisen in the open class in 1995 for If Mountains Could Sing, and the Spellemannprisen honorary award in 2005. In 2012 he was inducted into the Rockheim Hall of Fame. He was married to the singer Inger Lise Andersen (Inger Lise Rypdal) from 1969 to 1985, and they had two children, including the electronic musician Marius; his son Jakob Rypdal, from his second marriage, is also a guitarist. As of 2013 he lived in Tresfjord, the village on the Norwegian west coast where his father's family came from.

Rypdal built a sound on the Fender Stratocaster: long sustained single notes shaped with a volume pedal and delay, and a distorted, rock-derived tone that he brought into European jazz. AllMusic's biography calls him the "tone poet of the Fender Stratocaster," and his gear as listed on Norwegian Wikipedia centers on a Stratocaster, Vox and Marshall amplifiers and a few effects pedals. His influences are unusually mixed: Hank Marvin's instrumental rock from his Vanguards years, Jimi Hendrix, the contemporary composer György Ligeti, and the large-form writing of George Russell, who taught him. The critic Michael Tucker described the result as rock and jazz phrasing joined to a rubato, color-driven approach to dynamics that often sounds closer to the classical world. His records divide into several strands: the electric fusion of Odyssey and the Chasers trilogy, the brooding trio and quartet music with Mikkelborg and Jon Christensen, the string-and-choir works for ECM, and the later ensemble pieces such as Vossabrygg, which AllMusic says drew on the band of Miles Davis's Bitches Brew and on early Weather Report. He sits at the overlap of jazz, art rock and contemporary classical music rather than inside any one of them, and the jazz community has treated him as one of its own: he won the Norwegian Jazz Forum's Buddy Prize in 1985 and the jazz Gammleng Prize in 1990, and English Wikipedia describes him as an important member of the Norwegian jazz community. Norwegian Wikipedia counts him among the best-known Norwegian jazz musicians abroad, alongside Jan Garbarek.