Ellis Louis Marsalis Jr. was born in New Orleans on November 14, 1934, the son of Florence Robertson Marsalis and Ellis Marsalis Sr., who ran a gas station and later a motel in Jefferson Parish where touring Black musicians, barred from the segregated downtown hotels, could stay. He played tenor saxophone and then piano in high school and worked locally with a rhythm and blues band that included the pianist Roger Dickerson. At Dillard University he took a bachelor's degree in music education in 1955 and played modern jazz in a campus group with the drummer Ed Blackwell, the clarinetist Alvin Batiste and the saxophonist Harold Battiste. When Ornette Coleman left New Orleans for California in 1956, Marsalis and some of the others followed him west; he came home after a few months and said years later that he never worked out what a pianist could do behind Coleman's free playing. He then enlisted in the Marine Corps, where he played in a small combo called the Corps Four that appeared on service television programs, and after his discharge he worked at his father's motel by day and gigged at night, including at the Playboy Club. He married Dolores Ferdinand in 1959, and their first sons, Branford Marsalis and Wynton Marsalis, were born in 1960 and 1961.

In the late 1950s he recorded with the American Jazz Quintet, a New Orleans modern-jazz group of Battiste on tenor saxophone, Batiste on clarinet, Blackwell on drums, Marsalis on piano and the bassist William Swanson, whose sides were not issued until 1976, on the Opus 43 set New Orleans Heritage: Jazz 1956-1966, and were reissued later as In the Beginning. In 1961 Battiste founded All For One, or AFO, the first Black musician-owned record label in the American South, and it issued Marsalis's first album, Monkey Puzzle (1963), by his quartet with the tenor saxophonist Nat Perrilliat, the bassist Marshall Smith and the drummer James Black. Two of its tunes, "Whistle Stop" and "12's It," later gave titles to Columbia albums. Battiste and several other AFO players left for Los Angeles in 1963, but Marsalis stayed in New Orleans, where he was a rare pianist working in bebop and modern jazz rather than traditional or rhythm and blues styles. He recorded with Nat Adderley and Cannonball Adderley in the 1960s when the Adderleys passed through town, and from 1967 he played in the band of the trumpeter Al Hirt, whose television appearances included The Tonight Show and The Ed Sullivan Show.

Teaching became his main work. He taught improvisation at Xavier University of Louisiana, earned a master's degree in music education at Loyola University New Orleans in 1974, and that year became the first jazz teacher at the New Orleans Center for Creative Arts, the city's arts high school, where his students included Terence Blanchard, Harry Connick Jr., Donald Harrison and Nicholas Payton, along with Kent Jordan, Marlon Jordan, Victor Goines and Reginald Veal; his own sons Branford, Wynton, Delfeayo and Jason also came through his classroom. His reputation outside the city grew slowly. He made the solo LP Solo Piano Reflections (1978) on his own ELM label, and in 1982 he recorded with Branford and Wynton on Fathers & Sons (Columbia), a generational showcase whose two sides pair the Marsalis family with the Chicago tenor saxophonists Chico Freeman and his father Von Freeman. After the early-1980s success of his sons, he began recording regularly, for ELM, Spindletop, Rounder, Blue Note and Columbia.

In 1986 he left for Virginia Commonwealth University in Richmond, where the drummer Clarence Penn studied with him and, according to Penn's own profile, played in his trio from 1987 to 1991. In 1989 the University of New Orleans brought him back to found its jazz studies program, which he led until he retired in 2001, with Battiste among his faculty colleagues. In the 1990s he made the Columbia albums Heart of Gold (1992), Whistle Stop (1994), Joe Cool's Blues (1995, with Wynton), Loved Ones (1996, with Branford), Twelve's It (1998) and Duke in Blue (1999), mostly produced by his son Delfeayo, and in 1990 he helped steer the singer Nnenna Freelon to Columbia after hearing her at a Southern Arts Federation meeting. For more than thirty years he also played two sets every Friday night at Snug Harbor on Frenchmen Street, a residency that ended in December 2019.

The family marked his retirement with a celebration concert in 2001, released as A Jazz Celebration; it became a PBS special and led to a family tour in 2003. In 2007 he received an honorary doctorate from Tulane University and in 2008 he was inducted into the Louisiana Music Hall of Fame. In 2011 the National Endowment for the Arts named him and his sons Branford, Wynton, Delfeayo and Jason NEA Jazz Masters, the first time the fellowship went to a group, and the Ellis Marsalis Center for Music, named for him, opened that year in the Musicians' Village in the Upper Ninth Ward. His late recordings include An Open Letter to Thelonious (2008), the two-piano set Pure Pleasure for the Piano (2012) with Makoto Ozone, and For All We Know (2020), recorded with Jason in the weeks before his death. He passed away on April 1, 2020, at 85, of pneumonia brought on by COVID-19; his wife Dolores had died in 2017, and he was survived by six sons. A posthumous Grammy Trustees Award followed in 2023, and a 1999 biography, Jazz Man's Journey by D. Antoinette Handy, tells his story at length.

Marsalis was a modernist in a city of traditionalists. His melodic style, with running right-hand lines over bebop chords and rhythms, was often called romantic or contemporary, and one obituary noted it was sometimes described simply as Louisiana jazz. Thelonious Monk was a lasting love, honored on An Open Letter to Thelonious (2008), and he devoted Duke in Blue (1999) to the music of Duke Ellington. His own compositions, among them "Syndrome," "After," "Magnolia Triangle" and "Twelve's It," came back on album after album over half a century, and his trio and quartet records set them beside standards, with two long stretches of unaccompanied playing on Piano in E and Ruminations in New York. As a teacher he urged students to learn from the masters and then find their own routes, and he said he regarded himself as a teacher who became a major musician by accident. A generation of New Orleans players, including Terence Blanchard, Donald Harrison, Nicholas Payton, Harry Connick Jr. and Jon Batiste, came out of his classes or his circle, as did all four of his musician sons.