In line with rapid advances in generative AI technology, efforts are underway across a variety of industries to commercialize digital humans that look exactly like humans and move autonomously.
In the previous article, we introduced the possibilities of using digital humans in the Japanese entertainment industry. Here we look at one of Japan’s traditional performing arts.
Dentsu has partnered with Watanabe Entertainment Co., Ltd. a major talent agency to launch a joint casting business. The first project involved the creation of a digital human model of Shiraku Tatekawa, famed for his presentations of Rakugo, a form of Japanese verbal comedy.
Below, Aaron Zhu of Dentsu BX Creative Center, who is overseeing the project, discusses this initiative with Shiraku.
Eclipsing Language and Time Curbs to Share Performing Arts Overseas
Traditional performing arts, such as Noh, Bunraku, Kabuki, and Rakugo, have been popular in Japan for centuries. Not only do overseas tourists attend performances, but some of the arts have crossed oceans and been performed for audiences abroad.
However, the language barrier remains a major obstacle. The context that causes excitement and laughter among members of performing arts audiences in Japan cannot always be fully conveyed or understood. Further, given that the number of people involved in the traditional arts is limited, it is hard to expand activities abroad while still working in Japan.
Although famous Rakugo storytellers and other traditional performers are affiliated with talent agencies and active in a wide range of media, conventional casting methods are affected by time and travel constraints. Thus sometimes it is not possible for them to respond to the wishes of companies and society.
It is in this context that the use of digital humans holds great potential as a means of resolving the issues and creating new value.
The use of digital humans provides a highly topical and flexible means of communication, especially in such digital media as social media and video content, as also in metaverse spaces. As generative AI technology has advanced over the past few years, digital humans have evolved to the point where they can respond in real time and engage in free dialogue in a smoother, more natural way. As a result, their use in various fields, including advertising, is accelerating.
Greater Use of Real Talent Sought
The Dentsu Group in Japan has launched a joint venture with Watanabe Entertainment for the casting of digital humans.
As a first step, we created a digital human version of Rakugo performer Shiraku Tatekawa who is affiliated with Watanabe Entertainment.
The use of Shiraku’s digital human allows him to transcend the constraints of time and place, and to perform Rakugo stories in his own voice, or to communicate through free conversation, anytime, anywhere.
In the future, we hope to use generative AI technologies to create models supporting multiple languages, and to distribute Shiraku’s Rakugo as global content. We also plan to create new experiences unique to digital humans, including having the AI model learn the patterns of Shiraku’s short stories, which are told before he moves to the main story. Often, they are just small talk, or explanations of key words important to understanding the main story.
An example might be the performance of the classic Rakugo work, Shinigami (The Grim Reaper), performed for this shoot. Our filming of Shiraku’s performance of this classic served as the basis for the creation of the digital human version of Shiraku. Typically, in real life he will intersperse current events in scenes while spells are being chanted. So, with AI learning from arrangement patterns, as well as having outside information, it may be possible to create arrangements that incorporate the latest topics and even local topics when the works are performed abroad.
Known Abroad, But Hard to Perform
How does Shiraku view this groundbreaking initiative? Aaron Zhu, who is overseeing the joint project, asked him about his impressions.
Aaron: Thank you for cooperating in the creation of a digital human. Assuming this is a new endeavor for you, what did you think?
Shiraku: It is an exciting approach. I’ve always been quite interested in performing Rakugo in different languages.
Aaron: Oh really? That’s interesting.
Shiraku: The idea of using AI to perform Rakugo is ingenious, so I thought I’d give it a try.
Aaron: Thank you very much. Of all the performances you could have selected, why did you choose Shinigami?
Shiraku: Shinigami is based on a Grimms’ fairy tale and an opera. So it’s a topic to which even non-Japanese can relate.
Aaron: It certainly is. And it is probably a performance that is easy to develop in multiple languages.
Shiraku: That said, although the subject matter of Shinigami is simple and easy to understand, it requires highly advanced skills to perform. It would be impossible for an apprentice to master the performance.
Aaron: I see, so that is why there is great value in having your digital human perform Shinigami. Given the high standard of the content, we hope to continue broadening the appeal abroad, and I hope you will continue to provide support.
The project aims to both expand the possibilities for excellent Japanese content, and take on new challenges in the digital and real worlds. Starting with this project, Dentsu Group companies in Japan plan to work with various entities to promote casting for corporate marketing activities. The companies will take the lead in supervising the production of digital humans, and engaging in planning and content branding that use digital humans.
By using digital humans, as well as applying AI technology to existing content, we hope to improve talent management and increase production efficiencyAaron, so as to further diversifying talent content.
We encourage everyone with an interest in this project to please get in touch.
Related Link
Digital Human Casting Transforms Rakugo into Global Content (Japanese language only)
