Like Lu-king in a Mirror (August 2022)

Hello and welcome back, all, to the Song Future blog! 

That's right, this time I'm not even going to apologise for the title because, frankly, I am far too proud of it. Other options included 'Here's Lu-king at you', and 'Lu-king good'. Basically, some sort of pun on the word ‘looking’ - you get it. I know, very clever. 

But what is it talking about, you ask? 

Well friends, in case you didn't read the blurb on the page that led you here, allow me to introduce to you Song Future's protagonist, Avatar Lu. 

Lu will be your companion and erstwhile guide in the upcoming experience, and as such I felt that it was only right that we discuss a little bit here her background, and how she came into being. 

You might be surprised to learn that the Lu pictured above is not, in fact, a real person. And not just in the sense that she is a character - the face pictured above was created and rendered entirely digitally. In fact, the visage pictured here is only the first iteration of her design – for now she is a but a working prototype that is still undergoing development. Ultimately, her likeness will be based on Wang Rousang, the star of the original Song of the Female Textile Workers performance. 

Although her presence has always been the central feature of the Song Future project, Lu herself has gone through several design iterations. A number of ideas for how she would be presented to the user were examined and assessed; could she be shown actively around the museum, interacting with the equipment? Or perhaps standing face-to-face with you on your mobile screen? The technological capabilities of the visitor’s mobile devices also needed to be taken into account at this stage; Lu’s ultimate design would have to be lightweight enough that she can run on a wide range of devices. 

In the end, it was decided that the most intuitive and visually appealing method of showing Lu on-screen would be through a video call framing. Who in the last two and a half years hasn’t been on any number of video calls with friends and family, near and far? It is hoped that this familiarity will ease visitors into the experience, and make them more willing to chat openly with Lu. 

Having decided how to present her, the teams at Human and Megaverse could set about creating her. Again, several different options were discussed; should she be more cartoonish in appearance? Or would you, the audience, be more likely to relate to a more realistic character? How could they avoid the uncanny valley [1] effect, and make Lu appear sympathetic in our eyes? 

Eventually, they settled on using ‘MetaHuman’, a proprietary framework using the ‘Unreal Engine’ [2] that can create and animate breathtakingly accurate human faces. By playing around with the nigh-on limitless customisation options available therein, the team produced the initial version of Lu pictured above. 

The output of this process is a graphical model that can be dropped into an Unreal Engine scene – essentially the canvas that the engine uses to operate – and animated and manipulated to look alive. 

Animating the head is one thing, but getting it to talk convincingly and interact with the user is something else entirely. In order to make this work – and again, avoid the dreaded uncanny valley – Megaverse have set up another program called ‘Live Link Face’. This allows an iPhone camera to link directly to a MetaHuman model in an Unreal scene, and lets a visible person drive its motions in real time. Setting up a microphone in tandem allows the sound and video of Lu’s motions to be recorded together, and the resulting assets can be rendered out of the Unreal scene and made available for the Song: Future performance.  

Indeed, Haili Ma – the project’s Principle Investigator and Lu’s voice and facial acting talent – will soon be on her way to the Sheffield-based studio to finally bring Lu to life. 

Of course, Lu is more than just a face; she is a character in her own right. She has hopes and dreams, a history, and a future.  

Born in Shanghai at the height of the textile industry, her family must have long assumed that she’d follow them into the factories to join the prestigious ranks of the female textile workers therein. This wasn’t to be, however, as Shanghai began large-scale reforms to its industries in the 1990s, closing the textile factories as part of its movement away from ‘made in China’ and towards ‘created in China’. 

Though the change was devastating for the older generation, the lack of enforced working classes meant that Lu and her peers enjoyed freedoms and choices the likes of which her parents could only dream. She used this freedom to study English, learn to program and eventually move to the UK to work as a game developer right here in Leeds. 

I won’t tell any more of Lu’s story now, for fear of spoiling the experience, but suffice it to say that intrigue and drama will ensue! So you’ll just have to come along and find out for yourselves! 

I for one cannot wait to finally meet Lu, and I look forward to you all being afforded the same pleasure. 

  

  Footnotes

[1] The Uncanny Valley is an observed feature of graphics and robotics whereby an avatar that is similar to, but not perfectly identical to, a human, appears creepy and, in some cases, deeply upsetting to the audience. Google ‘Telenoid’, if you dare, for a far too real example of the effect that might linger with you. 

[2] The Unreal Engine is a 3D game engine originally developed for 1998’s ‘Unreal Tournament’ by Epic Games. The engine proved incredibly popular, and has since been released as a standalone development tool for the creation of computer games and graphics. It is currently on its 5th generation of release, which was published in April of 2022.