James Sidney Greene Jr. was born February 24, 1975, in Hartford, Connecticut, the son of James Sr., a saxophonist, and Renee Simmons. An alto saxophone was bought for him when he was six and he began lessons at eight, with three siblings in the house and jazz taking over by middle school. A teacher at Bloomfield High School arranged for him to play for alto saxophonist Jackie McLean at the Artists Collective, the Hartford community arts school McLean and his wife Dollie had founded in 1970, and Greene went on to study there and at the Greater Hartford Academy of the Arts. He graduated from Bloomfield with honors in 1993 and took a bachelor of music in African-American Music Studies from the Hartt School at the University of Hartford in 1997, summa cum laude, studying with McLean, trombonist Steve Davis, and saxophonists Ken Radnofsky and Kris Jensen. In 1996 he was named first runner-up in the Thelonious Monk Institute of Jazz International Jazz Saxophone Competition, placing behind Jon Gordon and ahead of John Wojciechowski before a panel that included Wayne Shorter, Jimmy Heath, Joe Lovano, Jackie McLean, and Joshua Redman.

He moved to New York in September 1997 and joined the Horace Silver Quintet almost immediately, recording Silver's Jazz... Has... A Sense Of Humor at Avatar Studios on December 17 and 18, 1998, alongside Ryan Kisor on trumpet, John Webber on bass, and Willie Jones III on drums. He was also playing tenor in Harry Connick Jr.'s big band, where he was a featured soloist on the Harry for the Holidays and Only You concert releases and on Connick's 2003 NBC holiday special, and he was a charter member of trumpeter David Weiss's New Jazz Composers Octet, cutting First Steps Into Reality (Fresh Sound New Talent, sessions running March 28, 1997 to May 3, 1998) with Myron Walden and Nasheet Waits in the band. The octet went on to Walkin' the Line (2002), The Turning Gate (Motéma, 2008), and On the Real Side (2008), the last recorded with Freddie Hubbard fronting the group. Alongside that Greene worked in the Carnegie Hall Jazz Band and in the big bands of Jason Lindner and Kendrick Oliver.

His own leader career started with two albums released within a month of each other in 2000. Introducing Jimmy Greene, a sextet date cut at RPM Studio in New York on October 10, 1997 with John Swana, Steve Davis, Aaron Goldberg, Darryl Hall, and Eric McPherson, came out on Criss Cross Jazz on February 8, 2000; Brand New World, recorded June 6 and 7, 1999 and produced with Bob Belden, came out on RCA Victor on January 11, 2000 with Darren Barrett, Steve Davis, Goldberg, Dwayne Burno, McPherson, and percussionist Kahlil Kwame Bell. Three further Criss Cross dates followed: Forever (recorded November 5, 2003 in Brooklyn, with Xavier Davis, John Benítez, and Jeff "Tain" Watts), True Life Stories (December 3, 2005 at Systems Two, with Jeremy Pelt, Davis, Reuben Rogers, and Eric Harland), and Gifts and Givers (April 5, 2007, adding Marcus Strickland's second tenor, Mike Moreno's guitar, and Danny Grissett's piano). Through the same stretch he held the tenor chair in Tom Harrell's quintet, appearing with the group on CBS's 60 Minutes and on Paradise (2001), Live at the Village Vanguard (2002), and Wise Children (2003), and in Ralph Peterson's quintet on The Art of War (2001), Subliminal Seduction (2002), and Tests of Time (2003). He also recorded with Chick Corea on Originations (Stretch, 2000), with Ben Riley's Monk Legacy Septet on Memories of T (Concord, 2006), and repeatedly with bassist Avishai Cohen, from Devotion (1999) and Colors (2000) through the International Vamp Band album (2001), As Is... Live at the Blue Note (2007), and Seven Seas (2010). It was in this New York circle that he joined the nonet on Omer Avital's leader debut Think With Your Heart, playing tenor and soprano in a five-saxophone front line with Joel Frahm, Greg Tardy, Jay Collins, and Walden, over Avital's bass and the drums of Marlon Browden and Daniel Freedman.

Greene had begun teaching in parallel with the playing, taking a master's in music education from Boston University and a doctorate in jazz arts advancement from the Manhattan School of Music, where he received the Helen Cohn Award. He taught at the Hartt School and at SUNY Purchase, then spent three years on the faculty of the University of Manitoba in Winnipeg, and in 2012 accepted a position at Western Connecticut State University, where he is now Professor of Music and co-coordinator of jazz studies and where he won the 2013 Outstanding Faculty Award. His records in this period moved off the Criss Cross template: The Overcomer's Suite (Nu Jazz, 2008) was issued as a download with a quartet of Xavier Davis, Luques Curtis, and Kendrick Scott; Mission Statement, recorded October 13 and 14, 2008 and released in 2009 on Sunnyside in partnership with Avishai Cohen's Razdaz Recordz imprint, with Cohen as executive producer, added Lage Lund's guitar and Stefon Harris's vibraphone to Davis, Rogers, and Harland; and Live at Smalls (SmallsLIVE, recorded February 19 and 20, 2010) documented his quartet with Davis, Ugonna Okegwo, and Gregory Hutchinson at the Greenwich Village club.

Greene's six-year-old daughter, Ana Grace Márquez-Greene, was among the twenty children and six adults killed at Sandy Hook Elementary School in Newtown, Connecticut, on December 14, 2012, the year the family returned to Connecticut from Winnipeg. Beautiful Life, released on Mack Avenue on November 25, 2014, was made in her memory. Greene produced, arranged, and orchestrated it across sessions in New York, Brooklyn, Winnipeg, and West Hartford, with Renee Rosnes, Kenny Barron, and Cyrus Chestnut splitting the piano chair, Christian McBride on bass, Lewis Nash on drums, Pat Metheny and Jonathan DuBose Jr. on guitar, vocals from Kurt Elling, Javier Colon, and Latanya Farrell, spoken word from Anika Noni Rose, and a string section drawn from the Hartford Symphony Orchestra. It reached No. 5 on Billboard's Jazz Albums chart, No. 6 on Gospel Albums, No. 6 on Heatseekers, and No. 36 on Christian Albums, and brought Greene two nominations at the 58th Grammy Awards, for Best Jazz Instrumental Album and, with Colon, for Best Arrangement, Instruments and Vocals for "When I Come Home." A share of the record's proceeds was directed to the Ana Grace Project of Klingberg Family Centers and to the Artists Collective, the school where Greene had first played for Jackie McLean.

He followed it with Flowers: Beautiful Life, Volume 2 (Mack Avenue, April 28, 2017), a program of his own compositions, and While Looking Up (Mack Avenue, April 3, 2020), on which he played soprano, tenor, flute, clarinet, and bass clarinet alongside Lund, Goldberg, Rogers, Kendrick Scott, and Harris. From 2019 he held the saxophone chair in Ron Carter's Foursight Quartet with Rosnes and drummer Payton Crossley, documented on Foursight: Stockholm (2019), its second volume (2020), The Complete Stockholm Tapes (2022), and Carter's Finding the Right Notes (2022). He won the Rising Star, Soprano Saxophone category in DownBeat's 66th Annual International Critics Poll in 2018, and holds the 2013 State of Connecticut Governor's Arts Award in Music, the 2012 City of Hartford Innovator Award, and a 2005 ASCAP/IAJE commission awarded in Ornette Coleman's honor. His most recent leader album, As We Are Now, was recorded February 26 and 27, 2024 at Big Orange Sheep Studios in Brooklyn and released July 25, 2025 on his own Greene Music Works imprint, with Goldberg, Moreno, Shedrick Mitchell, Dezron Douglas, Jonathan Barber, Rogerio Boccato, Gabriel Globus-Hoenich, and Colon.

Greene came up inside a specific Hartford lineage, taught by Jackie McLean in a degree program McLean built around African-American music, and the hard-bop discipline of that schooling stayed audible in the Criss Cross quartet and quintet records that made his reputation: broad tenor tone, long lines worked out over changes at tempo, standards rebuilt rather than decorated. His soprano playing is the more distinctive of the two horns, and it is what the DownBeat critics singled out in 2018. The gospel side of his background, including his work in the music ministry at Glory Chapel International Cathedral in Hartford, sits alongside the bebop training rather than replacing it, which is why his later Mack Avenue records could take in spirituals, contemporary Christian material, and show tunes without sounding like a change of instrument. Reviewers of Beautiful Life at the New York Times, AllMusic, All About Jazz, and the Irish Times all picked out the same combination of restraint and directness in his writing for that date. As a composer he works mainly from his own book: most of his leader albums are dominated by originals published through his Sidney James Music imprint.