All Hands on Deck (September 2022)

Hello, and welcome back once again, all, to the Song: Future blog! 

It's been a couple of weeks since the last update, and what a busy couple of weeks it's been! As we rapidly approach the start date for the performance, work on all sides is really coming to a head. Indeed, with so much going on, I thought writing an extended post about a single element of the project would be doing something of a disservice to the tremendous efforts of everyone involved. I have decided, therefore, that today I will bring you instead the headlines - teasers, if you will - of our many on-going activities; a look behind the curtain at what goes on in academic developments as deadlines approach. So enjoy, and be enlightened! 

Not wishing to bury the lede, I'll jump straight into updates from Human and Megaverse on the app and the performance itself. The team at Human have been working hard on the full range of graphical assets required for the experience; from Lu's appearance (which we discussed in the previous blog post), to the user interface (UI), to the augmented reality components themselves, they have been responsible for almost everything that you, the visiting public, will see during your Song: the Future experience. 

The augmented reality items themselves are, in many ways, the centrepieces of the whole performance. It has, therefore, been vital to craft beautiful, yet meaningful tableaus, pulling inspiration from not just the story we are telling through Lu, but from Leeds Industrial Museum itself and the many industries that it represents. I won't spoil the surprise of what exactly will be waiting in each of the rooms of the visit, but having been privy to early and updated versions, I am very excited to see them at full scale on location in October. 

We have also seen from the Human studio a largely finished iteration of the UI itself - the framing through which the visitor will experience the Song: the Future performance, and interact with Avatar Lu. Your interactions will be a mix of video calls and text conversations with Lu, and as such the UI has been designed to resemble any of the myriad messaging platforms with which you are no doubt well acquainted. By keeping the performance simple and intuitive in this way, we are hoping to minimise the experience's barrier for entry, and allow as many people as possible to join in and enjoy it for themselves. 

Having the UI and augmented reality elements is all very well and good, but they will mean nothing without the actual performance from Lu herself. Our team has been working closely with Megaverse to finalise the script, in particular making changes in parts where it became obvious that certain actions wouldn't work with the Lu avatar, or where her questions seemed out of place, or at odds with the narrative and research goals. Indeed, at the end of last week Ben (of Megaverse fame) and I sat down to hash out these final details, and produced a draft of the script that we can, at last, call final. 

This was important and necessary for perhaps the most exciting news from the past few weeks; the recording and capture of Lu's performance at the Human studio in Sheffield! 

What did I say earlier about not burying the lede? Well, I guess it happens to the best of us. 

On Friday of last week, Professor Haili Ma - the primary investigator of the Song: the Future project - headed to Sheffield to carry out the recording. Working with members of the Human and Megaverse team, they recorded her vocals reading the script and, as described in our previous blog post, tracked her facial motions in tandem using an iPhone camera. The tracked data can then be overlaid onto the Avatar Lu model and combined with the vocal track to produce the performance that we are so excited to share with you. By all accounts the recording session was a great success, and now we await the final process of rendering and producing the final outputs with baited breath! 

Of course, there's more to putting on a performance like Song: the Future than the actual performance itself, and our whole team has been working away feverishly behind the scenes to ensure that everything is up to code and ready for the October start date. 

First and foremost, we are working to ensure that the experience is safe and secure for everyone participating. Teaming up with Leeds Industrial Museum itself and others within the University, we have made every effort to make the experience inclusive, and able to be enjoyed by all. 

There is also the need make sure that personal privacy is respected throughout the experience, in particular in the form of data collection. Although we plan to collect very little in the way of personal data - we will only ask for a completely optional email address from visitors if they're interested in hearing about follow-up initiatives such as focus groups and interviews - we are still dedicated to making sure that it is stored safely and securely, to ensure the privacy of all concerned. To this end, we are currently working with the University to finalise the details of our data storage, and double checking that all proper standards are being followed and applied throughout the capture chain. For your own peace of mind, a privacy notice will be available within the experience so that you can review how we store and use any data collected from your side, including details on how to opt out if you so desire. 

All of this would prove a rather wasted effort, however, if nobody knew about the performance in the first place! To spread word of the Song: the Future project we have been working with Leeds Museums and Galleries - Leeds Industrial Museum's parent wing of Leeds city council - to create a publicity website for the upcoming performance, including a sneak preview teaser trailer of Lu! Although the website is not currently public at time of writing, it should be available very soon, for you to plan your own visit! 

As much as the development and completion of the Song: Future performance has dominated our recent workload, it hasn't stopped us from looking ahead and planning the next steps of the project. Towards the end of the performance, in late October, we will be hosting a symposium for partners and invited speakers, to discuss the knowledge gained from the project, and what lessons to take forward with us into the future. We will be joined by our friends and colleagues both here in the UK and at our associated institutes in China, which means navigating the necessary evil of time zones across the globe, and as such terribly early starts for those of us unlucky enough to be living in Europe! 

Alright, I think that's plenty for you to be getting on with, you lucky people! And indeed, plenty for us to be getting on with ourselves. I look forward to talking to you again soon!