Lawrence Ira Kahn was born in the Brownsville section of Brooklyn on March 20, 1939. His father, Nathan "Buddy" Kahn, led the mambo band at the Latin Quarter nightclub in Manhattan under the name Buddy Harlowe, and his mother, Rose Sherman Kahn, was an opera singer. He and his brother Andre, born in 1945, both studied classical piano as children and both were admitted to the New York High School of Music & Art. Larry loved the playing of Art Tatum and worked with jazz bands in Harlem as a teenager, but the music that changed his direction came from record shops and bodegas along 116th Street, which he walked every day on his way to the subway: Tito Puente, Machito and Tito Rodríguez. He began copying the Latin pianists he heard on Spanish-language radio, among them Noro Morales, Joe Loco and René Touzet.

He graduated in 1957 and went to Cuba, where he stayed about two years studying Afro-Cuban music, both the folkloric Yoruba tradition and the popular dance styles. The Cuban Revolution forced him off the island in 1959, before he could finish his degree. Back in New York he took a music degree at Brooklyn College (1963), later earned a master's degree in philosophy at the New School, and worked in both jazz and Latin bands, often combining the two. He took his stage surname from his father's and was nicknamed El Judío Maravilloso, the Marvelous Jew, a nod to the title given to the Cuban tres master Arsenio Rodríguez, El Ciego Maravilloso, whom he idolized. In 1975 he became a santero in the Afro-Cuban religion Santería.

By 1964 he led his own big band, Orquesta Harlow, which put trumpets and trombones side by side in the front line. Eddie Palmieri had made the two-trombones-and-flute front line the sound of his own group, and Harlow widened the horn section further. The band signed with the young Fania label, and its debut album, Heavy Smokin' (Fania LP 331), began a catalog that grew to more than twenty-five Orquesta Harlow albums. With the teenage singer Ismael Miranda, who joined for El Exigente (1967), the band made Abran Paso! (1971), Oportunidad (1972) and other records that became dance-floor staples, and Harlow's 1970 album Electric Harlow brought the clavinet and the Fender Rhodes into salsa.

Harlow was also the pianist and musical director of the Fania All-Stars, the label's showcase band, formed in 1968 and drawing on bandleaders and singers such as Ray Barretto, Johnny Pacheco, Willie Colón, Rubén Blades, Héctor Lavoe, Cheo Feliciano, Celia Cruz, Tito Puente and Eddie Palmieri. A summer 1971 concert by the group at the Cheetah nightclub in Manhattan was filmed by Leon Gast as Our Latin Thing (Nuestra Cosa), released in 1972, which Harlow produced and appeared in; in the film Orquesta Harlow plays "Abran Paso" for a street crowd in East Harlem. He became one of Fania's most prolific producers as well, credited with about 260 releases for the label, among them his brother Andy Harlow's four albums for Vaya between 1972 and 1976, the first of which, Sorpresa la Flauta, went gold.

His most ambitious projects came in the 1970s. Hommy: A Latin Opera (Fania, 1973), a salsa answer to the Who's rock opera Tommy about a deaf and blind boy with a gift for Afro-Cuban percussion, was performed at Carnegie Hall with a cast that included Celia Cruz, Cheo Feliciano, Adalberto Santiago and Pete "El Conde" Rodríguez. Cruz's performance of "Gracia Divina" on the record is widely credited with reviving her career, and Lincoln Center mounted a revival of the work in 2014. Salsa (1974) and La Raza Latina: A Salsa Suite (1977), the second written with Rubén Blades, traced the history of Afro-Latin music across an album each, and La Raza Latina earned a Grammy nomination.

When salsa fell out of fashion in the early 1980s Harlow left Fania, kept touring and recorded less often, making among other records Señor Salsa (1984) and The Miami Sessions (1988, with his brother Andy). In 1994 he joined Barretto, the vocalist Adalberto Santiago and the cuatro player Yomo Toro in forming the Latin Legends Band, which worked around the world and made Larry Harlow's Latin Legends Band 1998; he also composed the music for the bilingual stage piece Sofrito! with the storyteller David Gonzalez (recorded in 2000). The jazz-facing side of his later work is Larry Harlow's Latin Jazz Encounter: Live at Birdland (Latin Cool, 2003). In 2005 he played a piano solo on "L'Via L'Viaquez" on the Mars Volta's album Frances the Mute and sat in with the band live. He campaigned for the creation of the Latin Grammy Awards, was inducted into the International Latin Music Hall of Fame in 2000, received that organization's Beny Moré Memorial Award in 2002, and was given the Latin Recording Academy's Trustees Award in November 2008.

In his last years he worked with the younger keyboardist Marlow Rosado, whom he mentored, and their album Harlow & Marlow, Volume One: Passing the Torch (2016) was his final recording. He kept leading the Fania Latin Legends until shortly before he fell ill. Larry Harlow passed away in the Bronx on August 20, 2021, at the age of 82, of heart failure during a hospital stay for a kidney condition. He was survived by his wife, Maria del Carmen Harlow Kahn, his brother Andy, his son Myles, his daughter Haiby and three grandchildren.

Harlow was a salsa bandleader first, and this profile includes him for the jazz in his playing and his ideas rather than because the records are jazz records. English Wikipedia describes his piano approach to tropical dance music as "jazzified," and NPR called his jazz piano talent, mixed with Afro-Caribbean tradition, his musical calling card. The foundation was classical training, a youthful passion for Art Tatum and jazz-band work in Harlem, laid over a study of Cuban piano from Noro Morales and René Touzet to the tres-driven son of Arsenio Rodríguez, whose music he honored on Tribute to Arsenio Rodríguez (1971). What he added was arrangement and instrumentation: trumpets and trombones together, the Fender Rhodes and clavinet on Electric Harlow, the Yoruba bata drums that he is credited with bringing into salsa, and his own work on flute, oboe, English horn, vibraphone, bass and violin alongside the piano. R&B and funk entered his music as readily as jazz did.

His large-scale works follow the jazz tradition of the extended suite, Hommy as a stage work and La Raza Latina as a concept album, and his late band Latin Jazz Encounter made the jazz connection explicit. Salsa is not a jazz subgenre, so his header uses the nearest tags, Latin Jazz and Afro-Cuban Jazz.