Digital Storytelling Lab Leads Intensive Vibe Coding Workshop, Story I/O
On a Saturday morning in the Lenfest Center for the Arts, artists, programmers, designers, communication leaders, film students, and moms gathered to play with vibe coding, the artificial intelligence tool that allows laypeople to prompt an agent who will program digital code to make websites, apps, or chatbots for you. The diverse cohort is the kind of group that the Digital Storytelling Lab gathers: tech nerds interested in self-expression, artists looking to interface with cutting-edge technology, and everyone in between. What unites them is curiosity about the emerging forces shaping our world and a desire to participate, to get a seat at the table.
"Hello everyone," a disembodied voice came through the speakers. It's the Director of the Digital Storytelling Lab and Associate Professor of Professional Practice, Lance Weiler. As if to emphasize the central values of collaboration and technological capabilities, Weiler's inconvenient illness meant that he was leading the workshop remotely. A small team of collaborators—from Nick Fortugno, his colleague at the DSL and Director of Gaming Pathways at the City College of New York, to Columbia Engineering student Aryan Kaul—helped facilitate the workshop.
The workshop opened with lectures from practitioners like Associate Professor of English and Director of the Narrative Intelligence Lab Dennis Yi Tenen, Assistant Professor of Computer Science Lydia Chilton, and the game designer Fortungo, who created the interactive, AI-informed artwork Last Human with Weiler.
The lectures established a framework of AI as a tool for democratic, collaborative projects. "The term 'artificial intelligence' obscures the structure of creative collaboration," said Tenen. "Modern intellectual goods, like music or television scripts, are produced on an industrial scale, similar to other commodities like furniture or clothing. When we can perceive the structure of collaboration, we become more modest about what we contribute to it. At the same time, we can see AI not as a self-contained technology, but as a mode of co-authorship within a larger structure of collective production."
In his talk, Tenen also referenced the acceleration of productivity of novelists in the 20th century, when the earliest language models gave procedural writers like those behind Perry Mason easily replicable narrative structures. He also incorporated more modern uses of AI, like a news organization transcribing a high-stakes Zelenskyy interview, where the AI's speedy translation was fact-checked by a team of live journalists and translators.
But the meat of the event was the teamwork, where people from unlikely corners of New York came together to build functional websites from scratch.
The spacious conference room was filled with long tables, and each table had a group project unlike the ones from school days—projects where everyone wanted to be there. A table with a large group including an artist wearing an elaborate crown made of green pipe cleaners, a sleek businessman who had familiarity with AI jargon, and a young student taking notes on the far-reaching conversation came up with an app that would help identify and amplify feelings. They called it Umami, after the Japanese word that gave the Western world a term for a new dimension of flavor.
Another table, one with a few programmers, ideated a historical travel agent, who could identify what time period and geography would be of most interest to the user and provide an informative tour complete with factual narration, speculative scenery, and artifacts from that region for the user to explore. The site came together in a couple hours.
"What can we make that's different than what's already out there?" said Hriyanka Manilal, a product manager who encountered the Digital Storytelling Lab's events through Frank Rose's digital storytelling class. When she and her group were brainstorming what questions the chatbot should ask the user to determine where they'd like to travel in history, she anticipated what a travel agent in real life might ask a client. The exciting challenge seemed to be one of emotional creativity, imagining what would cause a person to search the internet for help and making technology to meet that need.
What brings these people together is a desire to play on the newest frontier of technology, but also a connection to each other and the people who run the Digital Storytelling Lab, like Weiler and Rose.
Kaul attended the workshop, along with a few of his friends he brought along, because of his relationship with Weiler and Chilton, his professors. Kaul's team worked on an app that helped visualize the Brouwer Theory, a physics concept that asserts that one atom in a swirling glass of liquid remains in exactly the same position the whole time.
One of his team members, Mike Fernandez, is an animation designer for websites. Fernandez made a prototype text animator that turned each letter into a digital kinetic sculpture, a million tiny dots swirling around the gravitational center of the shape of a letter. The dots scattered as he dragged a mouse through them, an interaction that would have taken him thirty hours to program on his own, instead of the thirty minutes it took him to program with vibe coding. If he was wasting his time on an insoluble or impractical, dud of an idea, he'd find out a lot faster.
After a few prototypes were up and running, the final event of the workshop was a test run. The historical tour guide app would be demonstrated by a user from another team. To evaluate the effectiveness of the user design, this participant tested the app with no instruction from the team that made it, as if he organically encountered the website on his own. The personality test took some scrolling to find, but he found it and answered the questions: what kinds of topics interested him and what landscapes he was drawn to. At the end, he received a virtual tour of Mughal period India.
Lance wrapped the workshop by inviting everyone to participate in an upcoming event the DSL is hosting at Lincoln Center where they could test and break their prototypes. He stressed even at the beginning of the event that the ideas and products created at the workshop today were our own intellectual property: if we'd like to continue developing these apps and put them on stage, this upcoming event would be the perfect opportunity. I would bet that most people there will show up, maybe not in order to break their prototype, but to continue engaging with these ideas in community.
"One thing I learned from today is that people want more connection," said Jane Nisselson, Director of Multimedia Communications for the Fu School of Engineering. She loved the unexpected surprises that came out of her prototype, a speculative city planning app. "It's been really beautiful to work with a group of people and share," she said.