David Roy Eldridge was born on January 30, 1911, on the North Side of Pittsburgh, Pennsylvania. His mother, Blanche, was a pianist who played by ear, and his father, Alexander, was a wagon teamster. He began on piano at five and took up drums at six, partly to follow his older brother Joe, a saxophonist, clarinetist and violinist who led a local band and talked him into switching to trumpet. Roy did little with the instrument at first. After his mother died when he was eleven he began shutting himself in his room to practice for hours, building up the high register that later became his calling card. He never became a confident sight-reader and leaned on his ear for much of his early career. He taught himself saxophone solos note for note, among them the 1926 turn by Coleman Hawkins on "The Stampede" with Fletcher Henderson's orchestra, and he later said that after hearing Hawkins and Benny Carter he resolved to play the trumpet like a saxophone.

Expelled from high school in the ninth grade, he left at sixteen with a traveling show that folded in Youngstown, Ohio, and then worked carnival and territory bands across the Midwest. At about twenty he led a Pittsburgh group billed as "Roy Elliott and his Palais Royal Orchestra," then joined a band led by Horace Henderson, Fletcher's younger brother, and later Speed Webb's band. In Milwaukee he took part in a well-remembered cutting contest with the trumpeter Jabbo Smith, who became a friend. He reached New York in November 1930 and worked in Harlem dance bands led by Cecil Scott, Elmer Snowden, Charlie Johnson and Teddy Hill, picking up the nickname "Little Jazz" for the contrast between his fierce playing and his small frame. His first widely noticed recorded solos came with Hill in 1935, and that July he played on Teddy Wilson's Brunswick small-group sides with Billie Holiday, among them "What a Little Moonlight Can Do" and "Miss Brown to You." In October 1935 he joined Fletcher Henderson's orchestra as lead trumpeter and featured soloist, singing now and then, and he stayed until September 1936. His solos on "Christopher Columbus" and "Blue Lou" from that year brought him a national following.

In the fall of 1936 he moved to Chicago and formed an octet with Joe Eldridge on alto saxophone and as arranger, with nightly radio broadcasts and recordings such as "After You've Gone" and "Wabash Stomp" that put his long solos in front. Worn down by the racism he met in the music business, he quit playing in 1938 to study radio engineering, then returned in 1939 and put together a ten-piece band that held a residency at New York's Arcadia Ballroom.

In April 1941, after offers from several white swing bands, he joined Gene Krupa's orchestra, becoming one of the first Black musicians to hold a permanent chair in a white big band. With the newcomer Anita O'Day he made "Let Me Off Uptown" and "Knock Me a Kiss," and a Benny Carter arrangement turned "Rockin' Chair" into a showcase for his tone and range. "Drum Boogie," a number he co-wrote with Krupa, was mimed by Barbara Stanwyck in Howard Hawks's 1941 film Ball of Fire, with Eldridge taking the long trumpet solo. The band broke up in July 1943 when Krupa was jailed on a marijuana charge. Eldridge freelanced in New York, then joined Artie Shaw's band in 1944. Racial incidents on the road led him to leave in October 1945, and the big band he formed afterward failed financially. Shaw recalled that crowds asked Eldridge for autographs after the show but that he could not get off the bus with the band to buy a hamburger.

After the war he became a mainstay of Norman Granz's Jazz at the Philharmonic tours, where he traded choruses with Dizzy Gillespie and Charlie Shavers. In 1950, on tour with Benny Goodman, he stayed in Paris, and the year in France restored his confidence in his own sound. Back in New York in 1951 he led a group at Birdland. From 1952 into the early 1960s he worked in small groups with Hawkins, Ella Fitzgerald and Earl Hines, and recorded steadily for Granz, including the two-trumpet sessions with Gillespie issued as Roy and Diz. In 1960 he joined Charles Mingus, Max Roach, Eric Dolphy and others on the Candid sessions of the Jazz Artists Guild issued as Newport Rebels. He toured with Fitzgerald from late 1963 until March 1965 and with the Count Basie Orchestra from July to September 1966, before returning to freelance work and festival appearances.

From 1969 he led the house band at Jimmy Ryan's on West 54th Street, a Dixieland room where he blended the traditional repertoire with his own faster, brasher manner. A stroke in 1970 slowed him, but he went back to the bandstand and sometimes sang or played drums or piano. He recorded for Pablo in the 1970s, including albums with Oscar Peterson, and was inducted into the DownBeat Jazz Hall of Fame in 1971. After a heart attack in 1980 he stopped playing trumpet, though he can still be heard at the piano on a 1986 edition of Marian McPartland's Piano Jazz. He lived in Hollis, Queens, where a street sign on the Van Wyck Expressway at Liberty Avenue honors him. He passed away at Franklin General Hospital in Valley Stream, New York, on February 26, 1989, at 78, three weeks after the death of his wife, Viola.

Eldridge built his trumpet style on saxophone models, chiefly Coleman Hawkins and Benny Carter, rather than on Louis Armstrong, whom he said he did not study closely until 1932. The result was a flashy, many-noted manner that ranged over three octaves, used harmony more adventurously than most trumpeters of his day, including tritone substitutions, and alternated rapid double-time bursts with a return to the main tempo. His tone was never pure or golden. Gary Giddins heard a vocal rasp and an urgent, rough humanity in it, and on "Rockin' Chair" that raspy edge is part of the effect. He loved the top of the horn and wrote the high register into solo after solo. From the mid-1930s he was often rated ahead of Armstrong as the model of modern hot trumpet playing, and Giddins ranked him the most emotionally compelling and far-reaching of the trumpeters who chose a rougher style than Armstrong's, a group that included Red Allen, Hot Lips Page and Rex Stewart. Ella Fitzgerald said he put more soul into one note than many players got into a whole song.

He was a major influence on Dizzy Gillespie, who spoke of him in near-messianic terms and who jammed and dueled with him at Minton's Playhouse in the early 1940s; his 1937 record "Heckler's Hop" was especially influential on Gillespie. Howard McGhee, Fats Navarro and Shorty Sherock also drew on him. Eldridge himself rejected the label of a bridge between Armstrong and bebop, and he said at the time that he did not understand Gillespie's bebop solos. He was fiercely competitive, famously willing to take on any trumpeter anywhere, and he was also a light, hoarse, rhythmic jazz singer, in the vein of Hot Lips Page according to Whitney Balliett.