According to Vollmer, The Pizza Underground began as a joke in 2012. They said that they had a light bulb moment when they realized they can replace any song lyric with a slice or cheese.
On November 11, 2013, they recorded their first live demo at Culkin’s house. That same month, they were able to book a performance at the Sidewalk Café in the East Village. In December that year, Culkin uploaded a video of him eating pizza while parodying Andy Warhol. The video became viral.
In January 2014, they released their first music video which was directed by Adrian Arrendondo. The clip was a jest since it was patterned from the Velvet Underground song. Instead of velvets, their background was 40 of the cheapest and grossest pizzas in New York.
They shot the scenes using a VHS to get a homage theme. The video featured the group performing on pizza boxes surrounded by pizza walls.