Harold Rubin was born in Johannesburg on May 13, 1932, and began classical clarinet study at 16 with Louis Nicholaeff, a pupil of Rimsky-Korsakov. Jazz pulled him away from the classical repertoire quickly: at 18 he was playing the Skyline night club in Johannesburg alongside singer Dulcy Ryder, working through the Ellington, Basie and Goodman songbooks with his own quartet. That quartet's most consequential gigs happened on the wrong side of the color line. Rubin's group crossed into the black township of Sophiatown to play the Odin Cinema on Sundays, defying South Africa's segregation laws, and it was there that he became friends with alto saxophonist Kippy Moeketsi and trombonist Jonas Gwangwa, two of the defining figures of South African jazz, trading ideas with them on and off the bandstand until Sophiatown's forced demolition ended the scene. Rubin was already a working visual artist by then, having held five solo exhibitions by 1957; his 1961 Sharpeville drawings responded directly to the previous year's massacre, and a 1962 painting titled My Jesus drew a formal blasphemy charge. He was acquitted after a five-week trial in early 1963 and left for Israel that March.

The move effectively ended his playing career for over a decade. Settled in Tel Aviv, Rubin trained and worked as an architect, spending years in Arieh Sharon's office and teaching architecture and design until 1986, while continuing to paint and exhibit. He did not perform again until late 1979, and what drew him back was not the swing-era repertoire he'd started on but records by Cecil Taylor, Michel Portal, Anthony Braxton and clarinetist John Carter, and later live encounters with Peter Brötzmann and Evan Parker: free improvisation, a music with no fixed South African or Israeli address, gave him a way back to the instrument on new terms.

In 1985 Rubin formed the quartet Zaviot with bassist/guitarist Mark Smulian, drummer Reuben Hoch and guitarist Tommy Bellman. Zaviot recorded three albums for the newly founded Israeli label Jazzis Records: the self-titled Zaviot (1987), The Tel-Aviv Connection (1988, with German pianist Christoph Spendel guesting on piano and synthesizer), and Unexpected (1988), on which visiting American soprano saxophonist David Liebman guests on two of the album's five tracks, "Mind Sleep" and "Maurice." Billed as Zaviot with David Liebman on its cover, it stands as the clearest evidence of Rubin's music reaching outside Israel in this first decade back. The band worked steadily in Israel and on European stages into 1989.

Through the 1990s Rubin moved between solo and small-group recording under his own name: the apparently solo Clarinetvoyance (1990), a trio session with Polish saxophonist Leszek Żądło billed as Springtime In Winter (1990), Trialog (1993) and Blue Bag (1996), the latter adding trombonists Rafi Malkiel and Reut Regev and vocalist Gani Tamir to Smulian's rhythm section. The same period put him in the studio with a cohort of Tel Aviv musicians a generation younger than himself, several of whom he is on record describing as protégés: guitarist/bassist Kobi Shefi, drummer Assaf Sirkis, cellist Yuval Mesner, pianist Daniel Sarid, saxophonists Ariel Shibolet and Albert Beger, bassist Dafna Sade and drummer Igal Foni among them. The earliest surviving document of the Rubin/Beger connection is the various-artists session Changes (Jazzis Records, 1993), where Rubin's clarinet and Beger's soprano saxophone both appear on the same two tracks, "Homage H" and "Dawn," alongside Sirkis on drums. Rubin also recorded internationally in this decade as a sideman, adding clarinet to British bandleader Graham Collier's Adam's Marble (Jazzis Records, 1995, cut in Israel) and German pianist Christoph Spendel's The Three Worlds (Konnex Records, 1998).

The 2000s and 2010s were his most prolific stretch as a leader. Ivory Tower (2001) paired him with pianist Daniel Sarid, bassist Adam Sheflan and drummer Hagai Fershtman; the Kadima Collective years produced the duo record Muse & Music (2005) with bassist/electronics player Jean-Claude Jones, A Mono Musical Suite for Three Manic Musicians (2005) with bass clarinetist Yoni Silver and pianist Maya Dunietz, and the solo One Voice: Clarinet (2007, recorded 2002); Trion (2007) added bassist Shmil Frankel and drummer Ram Gabay. His deepest run of international duo and trio dates followed: Suite (Creative Sources, 2012), a duo with German double bassist Alexander Frangenheim recorded in Hod Hasharon and released on the Lisbon-based free-improv label whose founder, violist Ernesto Rodrigues, produced it; 3 On A Thin Line (Hopscotch Records, 2013), a trio with American bassist Barre Phillips and Japanese-American percussionist Tatsuya Nakatani recorded live at Levontin 7 in Tel Aviv on April 23, 2009 and engineered, mixed and produced by Assif Tsahar; East Of Jaffa (OutNow Recordings, 2011), a trio billed to Ehran Elisha, Harold Rubin and Haim Elisha, recorded at Zvi Joffe's Tel Aviv studio in April 2011; and Ten Chamber Kingdoms (Jazzis Records, 2014), with Polish saxophonist Mikołaj Trzaska and bassist Nadav Masel.

Rubin kept working with younger players into his final years. Air Kiss Suite (OutNow Recordings, 2018) documents a January 11, 2018 house concert in his own Jaffa living room, a trio with bassist Asaf Shchori and pianist Illay Sabag on compositions credited to all three. He received the Landau Award for Performing Arts in 2008 for his contribution to Israeli jazz. Rubin passed away on April 1, 2020, at 87. His last album, Friends (Jazzis Records), was released in 2022, two years after his death: an eight-track session pairing his clarinet, present on every track, with bassist Shay Hazan, guitarist/bassist Kobi Shefi, pianist Rami Harel and, on three tracks, Albert Beger's saxophone, closing a musical relationship documented on record across three decades.

Rubin's playing after 1979 owed nothing to the Ellington and Basie material he'd started on in Johannesburg; his stated touchstones were Cecil Taylor, Michel Portal, Anthony Braxton and the clarinetist John Carter, with Peter Brötzmann and Evan Parker entering the picture later as live and recorded encounters. In his own words, quoted in his All About Jazz biography, he tried to "come empty and play what he doesn't know" every time he picked up the instrument, avoiding "laws for the music" even while acknowledging the intellect and vocabulary that inevitably build up around any player's language over decades. He called the periodic need to break that vocabulary down and start over playing "Neanderthal," a raw, pre-idiomatic state he treated as a discipline rather than a gimmick. That same instinct connects his two careers without collapsing them into one: both his drawing and his clarinet playing worked by improvisation under pressure, but the site profiles him here strictly on the recorded and performed musical output, not the parallel and, in his native South Africa, more publicly consequential visual-art career that ran alongside it from the 1950s onward. As a bandleader and duo/trio partner, he functioned less as a soloist chasing individual features than as a magnet for succeeding generations of Tel Aviv improvisers, several of whom (Assaf Sirkis, Albert Beger, Maya Dunietz, Yoni Silver) went on to their own significant careers in and beyond Israeli free jazz. His late-career international duo and trio dates, with Alexander Frangenheim, Barre Phillips, Tatsuya Nakatani and Mikołaj Trzaska among others, place him inside the same small transnational Creative Sources / free-improv festival circuit that carried Israeli free jazz to stages in Germany, Poland, Denmark, Romania and Lithuania.