It was born as an initiative of the Finnish National Opera and Ballet to explore the intersections between the arts and technological innovation and already has interesting results: new artistic experiences such as Laila, a project that blends artificial intelligence with co-creation techniques; the XR Stage platform, which makes it possible to improve theater processes and workflows; and an immersive opera show, CircOpera 2.0, which for the first time has a virtual avatar on stage. Opera Beyond, the project led by Lilli Paasikivi and Annastina Hapaasaari, is –according to them– a mark that the new generation leaves to keep opera and ballet alive.

By M. Angélica Navarro O. and Álvaro Molina R.

 

—Hello, Laila gives you her voice.

 

In the heart of Helsinki, in the opera house inaugurated in 1993 and which took over from a lyrical tradition dating from the late 19th century, Laila premiered, the first immersive experience designed by the Finnish National Opera and Ballet. It is a show created under the wing of Opera Beyond – a project that seeks to investigate and make creative crossovers between the performing arts and technology – which was designed in the midst of a pandemic and debuted in August 2020.

 

It is Laila herself who invites attendees to enter the dome where this sound, visual and interactive experience takes place. Inside, the public are part of the stage space and, thanks to artificial intelligence, they can make decisions and create their own way of living Laila. “We create a universe within which things can happen only for you”, explains Esa-Pekka Salonen, composer of this work that also proposes a reflection on the relationship between humanity and artificial intelligence.

 

Laila was, in a certain way, a prelude to the irruption of artificial intelligence not only in the arts but in people’s daily lives, a reality that in the middle of 2023 has ignited an intense debate, among others, in creative communities. The growth of artificial intelligence in the arts –works generated by platforms such as Midjourney or DALL-E, holograms that revive musicians of yesteryear or scenery created in a generative model from a simple description, for example– has risen to the stage of a worldwide debate on the ethical limitations in the use of generative platforms and has shaken the consensus regarding the definitions of what is art or not.

 

Some accuse that it goes against the artistic spirit or that it is a way of “impoverishing” human creativity. Others even think that they are the first steps of technology as an entity that will replace the work of artists. Artificial intelligence, the metaverse and the growing digitalization in culture are also, according to other voices, elements in a toolbox to find spaces for innovation and take traditional arts to places of unlimited experimentation.

 

This is what Opera Beyond believes, which after Laila has developed new projects that link theater work with new technologies. “Technological inventions have contributed and continue to contribute to new aesthetic trends, but there are more and more options on the palette today,” says Annastina Hapaasaari, one of the creators of Opera Beyond along with Lilli Paasikivi, mezzo-soprano and current artistic director. of the Finnish National Opera.

 

 

 

Why did the Finnish National Opera and Ballet decide to promote Opera Beyond and create immersive or virtual experiences?

 

LILLI: To keep opera and ballet alive, we must allow each generation to make its own mark on the artistic canon. They need to be given the opportunity to create new works and interpret old ones, both thematically, reflecting the questions that are relevant to them, and in terms of the form and media that they find interesting. New technologies are tools that are strongly linked to our time. For any art form, the immersive technologies available today allow for new and exciting experimentation.

 

In addition to allowing the crossing between disciplines, experimenting and bringing technology to the production of performing arts, what other purposes can projects like Opera Beyond have for a theater?

 

LILLI: With the Opera Beyond initiative we want, first and foremost, to create new types of audience experiences that enrich our regular programme. In 2020 we finished the first immersive installation, Laila, and in November CircOpera 2.0 premiered, a stage performance that was enriched with special effects.

 

We’ve also used new technologies behind the scenes, such as XR Stage, our stage viewing platform that uses virtual reality, game engines, and 3D modeling to make our workflows more flexible, efficient, and sustainable.

 

The project has given us opportunities to work with some really exciting tech companies, from Finnish startups like 3D ZOAN studio and hardware producer XR Varjo, to Genelec and the prestigious Nokia Bell Labs. I don’t think we’ve ever found ourselves working with those companies at that depth were it not for this project.

 

When you do something special and multidisciplinary that makes you stand out from the crowd, there are often new doors or opportunities that you can open as well. We have discovered that there are new funding opportunities for these types of projects, from targeted grants to technology sponsorships or even private donations.

 

The project has aroused a lot of interest not only from the public, but also from the media and from our colleagues, so we have gained visibility and in that way I think we have also managed to polish our brand.

 

Also, when working with these technologies, additional synergies or by-products are often created, and learning is created within the organization. For example, now that we have a digital twin of our scenery, like a 3D model in a game engine, we can use it in multiple ways. Not only in staging processes, but also in security demos, in ticketing, and why not as the basis for our virtual opera house in the metaverse, if that’s something that’s going to come as strongly as some people. predict.

 

 

Close Encounters with Opera

 

2020 was the year that Opera Beyond’s first immersive project hit the scene. It was Laila, an interactive chamber opera created by composer and conductor Esa-Pekka Salonen, playwright Paula Vesala, sound designer Tuomas Norvio and Ekho Collective for the Finnish National Opera, in which viewers are invited to become part of the scene. The music and the visual aspects of the work change according to the interactions of the public, which, mediated by artificial intelligence, becomes one more actor to shape the artistic experience. That same year, the FEDORA organization – the European organization of philanthropists for opera and ballet – awarded Laila as the best digital work, arguing that “the work makes bold use of new technologies while extending reality into a fascinating and collaborative experience”. .

 

Two years later, in 2022, Opera Beyond premiered CircOpera 2.0 on the main stage of the Finnish National Opera, a multimedia and multidisciplinary work that featured circus artists, lyrical singers and a full orchestra, as well as immersive technology that allowed the audience to immerse themselves in the spectrum of some of the most important opera music classics of all time, such as Lakmé’s “Flower Duet”, Cavalleria rusticana’s “Intermezzo” or “Alabama Song” from The Rise and Fall of the City from Mahagonny. “It was the first time that we received a completely virtual artist on our stage,” says Lilli Paasikivi.

 

 

What are the main differences between creating in virtual or digital environments and creating for a stage? How do the narratives change?

 

ANNASTINA: When creating content for any new medium, you have to consider the strengths and weaknesses of that medium. Therefore, best practices in fully virtual worlds – such as virtual reality and the metaverse – are different, for example, than a stage or space installation that combines digital and physical elements.

 

What is striking in fully virtual worlds is that the laws of physics don’t matter. You can create very dreamlike experiences, for example. You can make the audience lose their sense of time and space, although this can also be the goal in stage performances, in a sense, as psychologically they can be just as “immersive” as virtual reality is.

 

We’ve also had immersive theater with live actors, but this degree of interaction is something that has only recently become possible. I think the interactive layer is a game changer – it doesn’t need to stay solely in video games – and the interaction creates an opportunity for strong audience engagement.

 

“One of our initiatives that I would like to highlight is the XR Stage, an immersive 3D viewing platform for large-scale stage productions. This is a tool that we are now integrating as standard into our stage production workflows.” — Annastina Hapaasaari

 

How has the creation experience been at Opera Beyond?

 

LILLI: There has been a lot of trial and error in the process. Yes, it has been frustrating at times, but also more rewarding.

 

In both Laila and CircOpera 2.0 there were a lot of custom solutions in the technology that we used, which means that we used some of the technology in a way that perhaps had never been used before. So obviously you need patience. But then again, you can really be a pioneer, and what’s more exciting than that!

 

Involving new people is important because they can take you beyond your comfort zone, bring new ideas and make you push your own limits as well. Our teams were very transversal and multidisciplinary and came together specifically for these projects. This also meant that we needed to find our own work culture and language in the process. Organizational culture obviously also plays a role here. Large traditional cultural organizations are not always the most “agile” organizations, but these types of projects are also a good exercise for us.

 

One more thing we have learned from this process is that you can start somewhere, but end up somewhere totally different than you imagined. When we started, we didn’t see all the windows of opportunity that would appear along the journey. The field of technology is also constantly changing, which is also exciting.

 

How does the human intelligence/artificial intelligence binomial work in this type of project?

 

ANNASTINA: In the projects we’ve done so far, humans have created all the art; the music, the images, the narrative, the experience. “Machines” have played a role in human creation and diversification of the artists’ toolbox.

 

For example, in Laila music was still made with the human brain. What was new was how Esa-Pekka Salonen composed the music and how we recorded it. The composition is not only linear, but interactively layered, and all the different families of instruments were recorded separately. The people experiencing the installation can then activate different layers and effects with their actions, so that they appear inside the dome, so each group of people experiences the music differently. Each audience member can also record a piece of her own voice before entering the dome, and will hear her own voice as part of the music at some point. The voice of each audience group will also remain in the installation’s memory, and will be heard by other audiences later on as well.

 

 

How was CircOpera 2.0 conceived and how was La Diva brought to life, the virtual character that is part of the staging?

 

LILLI: Last fall, we premiered a major stage production called CircOpera 2.0. It is a spectacular show that combines circus and classical music.

 

In addition, we elevate the show with new elements, such as the first time we’ve welcomed a fully virtual artist to our stage, a very modern “diva” who is made with motion and facial capture and appears on stage with the other artists.

 

We also use projections that react to the movements of the performers, with sensors that recognize movement and then images that magnify the movements and circus acts.

 

There are different challenges that arise when you want to bring immersive or new elements to masses and large audiences, as in the case of CircOpera 2.0, when elements like this are used in real stage shows. Many of these technologies can bring more magic to the stage, but keep the live performance in the main focus.

 

What other opportunities for artistic development are glimpsed from the inclusion of these technologies, in aesthetic or stylistic terms?

 

ANNASTINA: Technological inventions have contributed and continue to contribute to new aesthetic trends, but there are more and more options on the palette today. There are options to make the digital image photorealistic in its texture and detail, or it can be stylized however one wishes. The digital medium no longer dictates the style as much in that way.

 

 

Opera Beyond’s research and creation projects have not only resulted in artistic experiences for audiences, but also in developments to improve work processes within the theater. “One of our initiatives that I would like to highlight from our colleagues is the XR Stage, an immersive 3D viewing platform for large-scale stage productions. This is a tool that we are now integrating as standard in our stage production workflows”, says Annastina Hapaasaari. It is a virtual replica of the theater, which allows testing and solving problems typical of the design and planning of stage productions and which plans to integrate technical functions of the stage to, for example, preprogram the lights of a show.

 

The work of art in the Age of Artificial Intelligence

 

In his influential 1935 essay The Work of Art in the Age of its Technical Reproducibility, the German philosopher Walter Benjamin addressed his fascination with the technologies that reproduce works of art. Taking the case of photography and mass printing among his examples, Benjamin argued that one of the essential points in this new artistic and technological era is the loss of the “aura”, that is, of the presence of a work of art in a certain time and space, its unique existence in the place where it is.

 

Thus, Benjamin warned that the disappearance of the aura would open the possibility of a new popular art through extensive reproduction. Almost 100 years later, the work of art is now entering an era where changes, this time introduced by artificial intelligence and new creation, dissemination and reproduction technologies, generating different questions and debates around the opportunities and threats that could to imply. Esa-Pekka Salonen, for example, is categorical: “Every artistic medium needs the traditional side, the legacy. We have to take care of the legacy, but we also have to make sure that there are new things that nobody has done before. We also have to take the risk that something will not work. Each new work is a risk in this sense, but if we don’t take risks, art is going to die”.

 

 

What risks do you see in the use of these new technologies in the performing arts? Are we not cannibalizing our own making, since it seems that the physical space of the theater is dispensable or because, with the development of artificial intelligence, even human creation and interpretation could be dispensable?

 

ANNASTINA: Technological development is inevitable, so it doesn’t help much to ignore it. I see it more as art having an important role in testing new technologies, as long as it fulfills its role as one of the early adopters of new technologies. Furthermore, it is only through testing that we will be able to see the full potential and also the drawbacks of these new technologies, and discover best practices.

 

Is this an area in which the entire cultural sector should investigate?

 

LILLI: Yes, it is! Embarking on an expedition does not mean that you have to leave traditions and old classics behind. What we want is to enrich our repertoire and canon. Not replacing, but enriching!

 

“Every artistic medium needs the traditional side, the legacy. We have to take care of the legacy, but we also have to make sure that there are new things that nobody has done before.” — Esa-Pekka Salonen

 

In Latin America there are few experiences in which new technologies intersect with the performing arts. If there is any theater or cultural organization you want to explore, where do you recommend starting?

 

LILLI: Be curious, go out, see things, benchmark, learn. Get new people involved, start imagining what you could do, and then do it.

 

Digital experiences vary enormously in scale. For all-digital content, you don’t need a lot of hardware or material resources, and they are highly scalable. On the other hand, if you get into large physical installations, they can be, for example, logistically much more challenging, but of course they can also be impressive to experience.

 

There is an ocean of possibilities out there and there is definitely something for everyone, including more conservative audiences.