Joseph Leslie Sample was born February 1, 1939, in Houston, Texas, to Alexander Sample, a mail carrier, and Agatha Osborne Sample, a seamstress. He began piano at five under organist Curtis Mayo and later attended Texas Southern University, studying piano without completing a degree. In high school he formed a combo called the Swingsters with saxophonist Wilton Felder and drummer Nesbert "Stix" Hooper; the group went through several names before becoming the Jazz Crusaders around 1960, the year they relocated to Los Angeles with trombonist Wayne Henderson added to the lineup. Sample's "funky, rhythmically appealing acoustic piano playing" was central to steering the band's sound from hard bop toward soul and funk, a shift the group made official in 1971 by shortening its name to the Crusaders. The band's own story, including its 1979 commercial peak with "Street Life," is told in full in The Crusaders' own JazzProfiles.com profile.

Sample built a solo identity alongside the band from early on, recording Fancy Dance as a leader in 1969 while the Crusaders were still active. His true breakout as a solo artist came with 1978's Rainbow Seeker on ABC Records, made with Crusaders drummer Stix Hooper and bassist Pops Popwell, blending catchy melodies and light funk grooves in a style that would later be filed as "contemporary jazz" and then smooth jazz. He followed with a steady run of solo albums through the 1980s and 1990s, including Carmel (1979), Voices in the Rain (1981), The Hunter (1983), Oasis (1985), Roles (1987), Spellbound (1989, Warner Bros.), Ashes to Ashes (1990), Invitation (1993), Did You Feel That? (1994), and Old Places, Old Faces (1996).

Alongside his own catalog, Sample became one of the most in-demand keyboardists in Los Angeles studios, crediting sessions with Miles Davis, Joni Mitchell, Steely Dan, and Eric Clapton, while the Crusaders as a group worked dates for the Jackson 5, Marvin Gaye, and Randy Newman. In 1997 he recorded Sample This, produced by fellow crossover keyboardist George Duke, and in 1999 he made The Song Lives On with singer Lalah Hathaway. His 1978 composition "One Day I'll Fly Away" reached a new generation of listeners in 2001 when Nicole Kidman performed it in Moulin Rouge!.

Sample relocated back to Houston in 1994 and recorded 2002's The Pecan Tree as a tribute to his hometown, followed by Soul Shadows (2004), Feeling Good (2006), and No Regrets (2008). He was Catholic, active in Catholic charities, and a member of Phi Beta Sigma fraternity. He was working on a project with singer-songwriter Jonatha Brooke, tentatively titled "Quadroon," when he died of mesothelioma in Houston on September 12, 2014, at age 75. His son, bassist Nicklas Sample, has performed with the Coryell Auger Sample Trio.

Sample's piano style is one of the defining sounds of jazz-funk crossover: a rhythmically buoyant, blues- and gospel-rooted right hand married to the electric-keyboard textures (Fender Rhodes, clavinet) that came to define the genre through the 1970s. His solo catalog leaned more explicitly toward pop-melodic "contemporary jazz" than the Crusaders' own records did, positioning him as both a co-architect of the crossover sound within the band and one of its most durable solo practitioners outside it.