Eberhard Weber was born on January 22, 1940, in Stuttgart and grew up in nearby Esslingen. His father taught music and gave him his first cello lessons at the age of six. He took up the double bass at about 16, when his school orchestra needed a bassist, and learned it largely on his own; he never attended a conservatory. After an apprenticeship as a photographer he worked in broadcasting and film before turning fully to music, and he later also worked as a television and theater director.

In the early 1960s he met the pianist Wolfgang Dauner at the Düsseldorf Amateur Jazz Festival, and Dauner became his closest musical partner for the next decade, in projects that ran from mainstream jazz to jazz-rock and avant-garde experiments, among them Dauner's group Et Cetera, formed in 1970. By 1973 Weber estimated that he had played on 35 to 40 records, many of them for the German label MPS, alongside players such as Hampton Hawes, Mal Waldron, the Brazilian guitarist Baden Powell, Joe Pass, Stéphane Grappelli and the Mike Gibbs Orchestra. He also played in Volker Kriegel's band Spectrum in the early 1970s.

In 1972 he bought the body of an old double bass, had a violin maker restore it, and fitted it with a pickup and an extra high string, then spent months removing the resonances that muddied its sound so that notes would ring longer and cleaner. The resulting electric upright, with a reduced solid body, became his only bass; AllMusic reports that he added a further string above that one in the late 1970s. On it he recorded The Colours of Chloë in December 1973, his first album under his own name, released on ECM in 1974 with pianist Rainer Brüninghaus, drummer Peter Giger, flugelhorn player Ack van Rooyen and a cello section from the Stuttgart radio orchestra; Weber himself played bass, cello, ocarina and voice. AllMusic's David R. Adler called it his best-known and most influential record.

The ECM connection drew him into the label's circle. He played on Gary Burton's Ring (recorded July 1974) and, as a featured guest with Burton's quartet, on Passengers (November 1976); on Ralph Towner's Solstice (December 1974) with Jan Garbarek and drummer Jon Christensen, and its sequel Sound and Shadows (1977); and on Pat Metheny's Watercolors (February 1977), a quartet date that also included Lyle Mays. In the mid-1970s he formed Colours with saxophonist Charlie Mariano, Brüninghaus and Christensen; John Marshall replaced Christensen after the first record, Yellow Fields, and the quartet went on to make Silent Feet and Little Movements, toured North America in 1976, 1978 and 1979, and disbanded in 1981. Between those records he led The Following Morning (1977), a drummerless set with Brüninghaus and members of the Oslo Philharmonic, and Fluid Rustle (recorded January 1979), a quintet with Bill Frisell, in his first ECM session, Burton, and the singers Bonnie Herman and Norma Winstone. He was also a member of the United Jazz + Rock Ensemble, a German all-star band.

In December 1978 he first recorded with Garbarek, on Photo with Blue Sky, White Cloud, Wires, Windows and a Red Roof, and from the early 1980s he was a regular member of the Jan Garbarek Group, which he stayed with until 2007; he appears on ten Garbarek albums released between 1979 and 1998. His own records in those years included Later That Evening (March 1982), with Paul McCandless, Michael Di Pasqua, Frisell and Mays; Chorus (recorded September 1984), with Garbarek; Orchestra (1988), with a brass octet; the solo-bass Pendulum (1993); and Endless Days (recorded April 2000). From 1985 he gave solo concerts in which electronic devices recorded his bass as he played and fed it back, so that he could accompany himself. He also played on four albums by Kate Bush: The Dreaming (1982), Hounds of Love (1985), The Sensual World (1989) and Aerial (2005). For his 65th birthday in March 2005 he recorded a concert at the Theaterhaus Stuttgart with Garbarek, Burton, Dauner and the Stuttgart Radio Symphony Orchestra, released as Stages of a Long Journey.

On April 23, 2007, Weber suffered a stroke while on tour with the Garbarek group, and he has not been able to play the bass since; Yuri Daniel took his place in the band. He has kept working as a composer and editor of his own archive. Résumé (2012) and Encore (2015) rework solos recorded live with Garbarek's group between 1990 and 2007, with keyboards and processing added by Weber, Garbarek's saxophone on the first and Ack van Rooyen's flugelhorn on the second. He received the Albert Mangelsdorff Prize in 2009, the year ECM reissued three of his Colours albums as a box set. Two concerts in Stuttgart in January 2015 marked his 75th birthday, with the SWR Big Band playing his music alongside Metheny, Garbarek, Burton and others; the broadcaster's recording was issued by ECM as Hommage à Eberhard Weber. His autobiography, Résumé: Eine deutsche Jazzgeschichte, appeared in 2015, with an English translation by Heidi Kirk, A German Jazz Story, in 2021. That year Lyle Mays's posthumous tribute composition "Eberhard" was released, and it won the 2022 Grammy Award for Best Instrumental Composition; ECM also issued Once Upon a Time (Live in Avignon), a 1994 concert recording. A January 2026 festival report describes Weber as living in the south of France and as patron of the Esslingen Jazz Festival since its beginning.

Weber is known for a highly distinctive tone and phrasing. His electric upright sings in long, sustained lines, often over repeating ostinato figures with finely differentiated tonal colors, and from the mid-1980s he used electronic devices to layer his own playing so that a solo bass could sound like an ensemble. AllMusic traces his use of contrasting ostinato patterns to the composer Steve Reich and describes his influences as primarily European, drawn from contemporary classical and new music rather than from the blues or from an animated, energetic approach. His compositions blend chamber jazz, European classical music, minimalism and ambient music, and they are regarded as characteristic examples of the ECM sound. By his own account his sound ideal was shaped by the muffled low frequencies he heard through a wall as a child, listening to his father's chamber-music evenings. He took titles for several pieces and albums from Richard Adams's novel Watership Down, and his wife Maja Weber, a painter who died in 2011, made the cover images for his albums from 1973. AllMusic's biography calls him not strictly a jazz bassist and one of the least flamboyant improvisers, yet among the finest bassists in Europe; his place in jazz rests on his ECM catalog, his work with Gary Burton, Ralph Towner, Pat Metheny and Jan Garbarek, and honors such as the German jazz prize named for Albert Mangelsdorff, which he received in 2009. The posthumous tribute "Eberhard" by Lyle Mays is the most direct homage to his compositional style, and The Colours of Chloë and Yellow Fields are still cited as defining records of ECM's chamber-jazz sound.