Jerry Herman was born on July 10, 1931, in New York City and raised in Jersey City, New Jersey. Growing up in a musically inclined, middle-class Jewish family, he was introduced to music at an early age. His mother, Ruth, a former singer and pianist who had performed in vaudeville, and later an English teacher, nurtured his musical talents. His father, Harry, worked as a gym teacher and in the summer managed a children's camp in the Catskill Mountains, where Jerry spent his summers from age 6 to 23. It was at this camp that he first became involved in theatrical productions, directing shows like Oklahoma!, Finian’s Rainbow, and A Tree Grows in Brooklyn.

Jerry graduated from Henry Snyder High School in Jersey City and initially enrolled at the Parsons School of Design in New York City, intending to pursue a career in interior design. However, after meeting composer Frank Loesser at age 17, who encouraged him to continue composing, Jerry transferred to the University of Miami in Florida. There, he immersed himself in the university's avant-garde theater department, producing, writing, and directing his first musical, Sketchbook.The show was so popular it ran for an additional 17 performances beyond its initial schedule.