Elmer "Skippy" Williams (some sources give his first name as Elbert) was born July 27, 1916, in Tuscaloosa, Alabama, younger brother of fellow saxophonist Pinky Williams. The family relocated to Cleveland, where Williams took up soprano saxophone in his early teens and led a band of his own while still a local player. By the late 1930s he had moved into the touring big-band circuit, working brief stints with Count Basie's orchestra and, in the summer of 1939, joining Earl Bostic's orchestra in New York. His earliest documented studio work dates from that period: a July 12, 1939 session with Earl Hines and His Orchestra in New York that produced "Ridin' and Jivin'" and "Grand Terrace Shuffle" for Bluebird, on which he is also credited as arranger. Through the early 1940s he passed through the reed sections of Jimmy Mundy's and Lucky Millinder's orchestras, recording with Mundy's band in 1940 and with Sam Price's Texas Blusicians in 1941, and he is also documented in period sources as having worked with Fletcher Henderson, Chick Webb, Claude Hopkins, and Edgar Hayes during these years, though specific recorded sessions with those leaders are not clearly dated in the surviving discographical record.

Williams's best-documented engagement began in 1943, when he joined Duke Ellington's orchestra as the direct replacement for Ben Webster on tenor saxophone. He appears on Ellington's second Carnegie Hall concert, recorded live on December 11, 1943, a broadcast-derived recording later issued commercially on multiple labels (Prestige, Queen-disc, Circle, and Storyville among them). It was around this same period, while the Ellington band toured and Williams was based nearby, that he gave informal saxophone lessons in Rochester, New York, to a teenage Pepper Adams. Williams left Ellington in May 1944 to try leading a band of his own; Al Sears took over his chair.

After Ellington, Williams continued as a sideman and occasional bandleader, working through the mid-1940s with Jimmy Mundy, Bob Chester, Tommy Reynolds, and Claude Hopkins. In 1946 he led a band on an engagement that included the young Thelonious Monk, a booking-only arrangement in which Williams held the leader contract while Monk played piano in the group. Later in the decade he kept a small combo working in Florida, and by the 1950s he had returned to New York, where period sources describe him leading small ensembles and performing around New York, Cleveland, and Florida into the 1960s. No leader recordings from any of these bands have been located; his career as an active performer thins out in the available record after that point. Skippy Williams passed away February 28, 1994, in New York City, at age 77.

Williams built his career as a section and solo tenor player inside other men's big bands rather than as a stylist who left a distinct critical footprint of his own; no leader recording survives against which to assess his individual voice, and none of the standard jazz critical references (AllMusic, DownBeat retrospectives, contemporary reviews) offer a developed account of his playing. His historical significance rests on the two verifiable facts that anchor this profile: stepping into Ben Webster's chair in Duke Ellington's orchestra during a pivotal 1943-44 transition in the band's reed section, and, on the teaching side, giving informal early instruction to a 12- and 13-year-old Pepper Adams, who would go on to become one of the defining voices on baritone saxophone in postwar jazz.