Mulling the metaverse: computer scientists weigh in on the internet’s future potential

Ever since Meta (formerly Facebook) announced its plans last October to build a “metaverse,” speculation has run wild about what it might look like, and how it could affect our lives. The company envisions a network of immersive, three-dimensional and interoperable virtual worlds driven by artificial intelligence where we will work, play, shop and socialize.

Among the existing technologies that are supposed to help get us there are virtual reality, augmented reality, user-generated video games and blockchain. Others may emerge, such as computer interfaces that would allow users to control applications with their thoughts via an electroencephalogram or implanted sensor. The idea, at the broadest level, is that these technologies will combine to create graphically rich virtual spaces with a high degree of verisimilitude that will seamlessly blend with, or replace parts of, our lives offline.

To help realize this vision, Meta recently gave out $510,000 in grants to 17 computer science labs at 11 Canadian universities to support research that “advances innovations needed to build for the metaverse.” Researchers may use these $30,000 unrestricted grants as they see fit to support their work in human-computer interaction, artificial intelligence and next-generation digital technologies.

For computer scientists, whether the metaverse lives up to its hype will largely depend on how thoughtfully we design and develop its infrastructure, features and services. Getting it right, they say, will require striking a healthy balance between the commercial interests of Meta, other companies driving this push, and users’ needs. To do that, we must consider factors such as equitable access, respect of privacy, mental and physical wellbeing, and technical functionality, they add. Their expert insights illuminate the opportunities and challenges we face, and the actions we should take, if we want to realize a metaverse that is functional, inclusive, decentralized and healthy.

Advancing AI

Harnessing the full potential of cyberspace will hinge on the progress we make in artificial intelligence, meaning how we get robots to perform tasks typically done by humans, says Richard Sutton, a renowned computer science professor at the University of Alberta. Dr. Sutton is a founder of reinforcement learning, a branch of machine learning in which robots learn optimal behaviour by interacting with their environments through trial and error and earning positive or negative rewards based on their actions. This differs significantly from the dominant paradigm of supervised machine learning in which a machine relies on a labelled dataset programmed by a human to identify patterns and make predictions.

“Reinforcement learning is more naturalistic,” said Dr. Sutton, who is a principal investigator at the university’s Reinforcement Learning and Artificial Intelligence Lab. “It’s learning the way an animal or person would – by interacting with the world and seeing what has better or worse outcomes…. It’s the standard model of how the brain processes rewards.”

After conducting decades of research, Dr. Sutton believes that reinforcement learning-trained robots will perform many important functions we may come to expect in the metaverse, such as giving advice on trading stocks, providing customer service as store avatars, or serving as characters in video games.

“We need intelligent systems that are continually sensitive to our feedback so that they can help us make smart decisions,” Dr. Sutton said. “They should be aware of our goal and try to achieve that mission.”

Working 9 to ?

For many of us, using the metaverse will influence how we make a living. The way computer scientist Joanna McGrenere sees it, digital technology has already blurred the boundaries between our work and personal lives, a trend accelerated by the pandemic-fuelled Zoom boom. The metaverse is set to exacerbate that tendency.

“I can imagine that in the intended immersiveness of the metaverse, that it is going to be very easy to lose track of time, and for it to kind of take over in a way that has the potential to be unhealthy,” said Dr. McGrenere, who is head of the University of British Columbia’s eDAPT research group (which is the recipient of a Meta grant). “I think more and more people are going to need help with reflecting on how their time is being spent at work, and whether it’s consistent with what they need to produce in a classic productivity sense, but also accounting for wellbeing.”

That was a conclusion from a 2020 research paper co-authored by Dr. McGrenere that looked at how the increasing cognitive and time demands of operating in the always-connected digital age requires a rethink of work-productivity measures. The researchers surveyed 40 knowledge workers about their personal productivity, how they defined making good use of time at work, and their feelings about their work time. “We envision highlighting the integration of emotion-tracking and the need for human self-reflection in addition to automatic tracking” of work time in a potential redesign of current productivity tools, the authors wrote.

“If you can input into a lightweight, easy-to-use tool how you’re feeling about how your time is spent, and then can reflect on that at the end of the day,” Dr. McGrenere said, “you can start to see, ‘wow, when I spend 15 hours a day in the metaverse, maybe I don’t feel so great at the end of the week’ or ‘maybe I do feel great’ and sort out what is working for them.”

Promoting well-being

Rita Orji agrees that promoting well-being in the next iteration of the internet is important, not just at work but in all spheres of life, and that will require innovative thinking and methods. The computer science researcher leads Dalhousie University’s Persuasive Computing Lab, which explores how to design interactive, personalized technologies, such as apps and games, that promote health and well-being, particularly among underserved groups. Her recent research has focused on persuasive games for preventing disease, and interventions that use augmented reality and virtual reality for healthy behaviour change.

While every new technological innovation gives rise to both possibilities and problems, Dr. Orji is optimistic about the metaverse’s potential to offer personalized, easy-to-use and accessible health services and supports. Specifically, she says, it could have a positive impact in areas such as monitoring our physical and mental health, incentivizing us to adopt more healthful behaviours, aiding health-care providers in diagnosing and treating patients, and “promoting the greater good,” as she put it.

“What really excites me is determining how to design digital applications that can empower people and improve lives, and are intelligent enough to understand your specific context,” said Dr. Orji, who is the Canada Research Chair in Persuasive Technology and another Meta grant recipient.

Making this happen will require avoiding a common tech industry trap  – creating applications that serve narrow interests or populations, or as Dr. Orji puts it, the “select few who are privileged” with enough resources to access those applications. Having grown up in a remote Nigerian village without access to electricity or pipe-borne water, she would like to see cyberspace serve users of all stripes, including those from marginalized socioeconomic, gender, racial or differently abled groups. To that end, she says, we will need to emphasize participatory design that incorporates the input of all stakeholders to create truly responsive tools and services.

“Deciding that we’re going to get diverse voices and experiences accommodated in the design at the early stage of conception is key to inclusion,” she said.

Hacking the hardware

The inclusion of women in hardware design to build the metaverse is a subject that computer scientist Mark Hancock has explored. He co-authored a systematic review of how virtual reality research considers the gendered aspects of “cybersickness.” A phenomenon more common in women than men, cybersickness occurs when exposure to a virtual environment causes symptoms such as dizziness, nausea, headache and fatigue. Dr. Hancock and his co-authors concluded the issue has not been systematically studied and deserves attention at every stage of virtual reality research.

“There are known gender issues with virtual reality technology that suggest female-identifying people are more likely to experience cybersickness, and the tech industry has a history of ignoring equity and diversity throughout its design processes, perhaps unintentionally,” said Dr. Hancock, who is director of the University of Waterloo’s Touchlab, which focuses on devising methods for interacting with information on novel digital interfaces. “It is also a known issue that it is hard to recruit women to participate in studies involving VR, as they often describe it as not being designed ‘for them.’”

Dr. Hancock also explores embodiment, which is when our awareness of the hardware components fades away and we feel like we truly inhabit our virtual self. Full embodiment will be essential for people to have meaningful metaverse experiences, he said, but the extent to which we can trick our brain into thinking that a virtual body is our actual body depends on further advances in VR technology.

Dr. Hancock and his team have looked at measuring how embodied we feel when interacting with different VR tools – an investigation that proved using our hands, versus using a controller, leads to a more embodied experience. Another study he co-authored found that the transportational quality of a virtual reality headset can mitigate the productivity-hampering distractions that are common in open office settings, improving a worker’s performance and job satisfaction.

“Actually being able to feel the way things are in a virtual world – there are still technical challenges,” said Dr. Hancock. “You can have rich experiences using a VR headset, but it’s not there in the sense that you can’t reach out and grab stuff, you have to use a controller…. Can we adapt to that new way of being? Possibly, yes.”