Johnny Smith was born on June 25, 1922, in Birmingham, Alabama. During the Great Depression his family moved through several cities before settling in Portland, Maine. He taught himself guitar by arranging with local pawnshops to keep their instruments in tune in exchange for the chance to play them, and at thirteen he was giving lessons; one of his students later gave him the first guitar he owned. He played for four dollars a night with a hillbilly band, Uncle Lem and the Mountain Boys, which meant dropping out of high school, then moved toward the jazz he heard on the radio and at eighteen left to join a variety trio called the Airport Boys. He later recalled hearing records by Django Reinhardt as a boy and meeting Charlie Christian when Christian passed through Portland.

Smith enlisted in the United States Army Air Forces hoping to become a pilot, but poor vision in his left eye ended that plan and he chose the military band over mechanic's school. Handed a cornet and an Arban method book and given two weeks to pass the reading test, he practiced in a latrine and passed. After the war he worked at radio station WCSH in Portland, and an air-check he sent to NBC led to an offer from the network in New York. He was a staff studio guitarist and arranger there from 1946 to 1951 and freelanced in New York until 1958, playing in settings from solo guitar to full orchestra, sight-reading scores in orchestra pits, and leading a small group, the Playboys, with Mort Lindsey and Arlo Hults.

His breakthrough came on Roost, the label that issued most of his records. A 1952 recording of "Moonlight in Vermont" with Stan Getz on tenor saxophone became a surprise success and ranked second among jazz records in the 1952 DownBeat readers' poll. It first appeared on a 10-inch LP, Jazz at NBC (Roost RLP 410), and later gave its name to a Roost twelve-inch collection. Smith recorded for Roost from 1952 into the 1960s, and Mosaic Records gathered most of those small-group sessions in an eight-CD set in 2002. At a 1954 session he wrote "Walk, Don't Run," a new melody over the changes of "Softly, as in a Morning Sunrise"; it appeared on his 1954 album In a Sentimental Mood under a title chosen by his producer, Teddy Reig. Chet Atkins recorded his own fingerstyle arrangement on Hi-Fi in Focus, the Ventures heard it, and their version, released in 1960, reached number two on the Billboard Hot 100. They recorded it again as "Walk, Don't Run '64," which also reached the Top 10, and Smith said he was grateful to Atkins and the Ventures for the royalties.

In 1957 Smith's wife died in childbirth, along with their second child. He sent his young daughter to his mother in Colorado Springs, Colorado, and in 1958 left his New York career to join her. He ran a music store there, taught, designed guitars and kept recording for Roost and then Verve into the late 1960s, but declined almost every invitation to tour. One exception was a 1977 tour of England accompanying Bing Crosby, which ended shortly before Crosby's death. His guitar designs became commercial models: a Guild Johnny Smith Award model in the mid-1950s, a Gibson Johnny Smith model from 1961 after Gibson's president Ted McCarty visited him in Colorado Springs, a Heritage model from 1989, and later a Guild version built by Bob Benedetto. He also developed amplifiers with Ampeg and wrote an instructional book, Aids to Technique.

His pupils included Gene Bertoncini, who first studied with him after meeting him at NBC's studios, and Bill Frisell, who took lessons in Colorado in the 1970s; Larry Coryell learned a difficult Ravel passage from him on a visit to Colorado Springs. In 2018 Frisell and Mary Halvorson released The Maid with the Flaxen Hair, a Tzadik album of pieces written or played by Smith. He was inducted into the Alabama Jazz Hall of Fame in 1984 and received the James Smithson Bicentennial Medal in 1998. Smith passed away on June 11, 2013, at his home in Colorado Springs at the age of 90, from complications of a fall.

Smith was one of the most versatile guitarists of the 1950s, equally at home in a jazz club, a studio orchestra and a symphonic pit. His sound was quiet and clear, built on closed-position chord voicings and fast ascending single-note lines that recall Django Reinhardt but lean more on diatonic than chromatic material. He said the hardest thing on the guitar was playing a smooth, melodic chord progression with no gaps between chords, and that he solved it by borrowing an organist's habit of holding one note of a chord while the other voices moved. He is usually classed as a cool and mainstream jazz guitarist, and his reputation rests largely on the Roost records, many of them about three minutes long. Pat Martino named him as an early model, Larry Coryell grew up listening to his records, and his tune "Walk, Don't Run" became a staple of instrumental rock through the Ventures' recordings. His archtop designs for Guild, Gibson and Heritage remain among the best-known signature guitars.