Evolving interactions
New opportunities to engage and interact
The range of methods for human-computer interaction is expanding, and interactions themselves are growing more immersive and seamless. This is creating new opportunities for organizations to reimagine how they engage with, learn from and delight their customers, employees and other stakeholders.
Many of the trends in this area are easy to overlook as they don’t, at least on the surface, seem revolutionary. In fact, the accuracy and usefulness of these systems has and will continue to grow immensely in the next couple years. For example, users have been able to speak commands into their mobile devices for years, but, until recently, interaction had to be phrased in a way the device would understand.
Products and services such as OpenAI’s ChatGPT, Google’s Bard and Microsoft’s Copilot have leveraged advances in Generative AI to lead this charge, lowering the bar for individuals to interact with systems or directly with AI models. Tech giants such as Apple are also undertaking major overhauls of their voice-to-text systems, using better predictive AI and context-aware models to render technology touch-free. Beyond voice and text, we expect continued advancement in extended reality (XR) technologies that allow users to interact in virtual worlds, though perhaps not at the pace predicted by early champions of the space.
There’s significant potential for these evolving modes of interaction to make technology experiences more inclusive, as demonstrated by systems like Jugalbandi which enables illiterate people to access information about government programs by speaking to their mobile device in their native language. Yet they also present a broader set of risks and accessibility issues. While accessibility approaches for more traditional interactions like mobile are relatively well understood, this is not the case for newer interactions like XR. Making voice- or GenAI-based interactions more available and sophisticated widens the scope for misuse and unintended consequences.
For the near term, there is no shortage of promising use cases for XR in emerging areas like training and data visualization that all enterprises should be ready to explore. Consumer adoption is likely to remain primarily limited to areas like collaboration, gaming and entertainment. So far, the advancements in consumer devices haven’t been sufficient to expand outside these areas, but product development by , and others is expected to continue.
You often see people’s eyes rolling at terms like metaverse or XR, but at the same time, people are using these immersive technologies in their daily lives without even realizing it: just look at how readily people accept blurring or changing their surroundings on video calls
You often see people’s eyes rolling at terms like metaverse or XR, but at the same time, people are using these immersive technologies in their daily lives without even realizing it: just look at how readily people accept blurring or changing their surroundings on video calls
Signals
Tech giants unveiling new XR-enabled devices such as Apple’s ‘spatial computer,’ and Meta’s . These are designed to be more comfortable and less intrusive than earlier examples of such hardware. In Apple’s case it incorporates to reduce the need for physical input devices.
The development of GenAI-enabled apps that enhance accessibility for people previously facing physical and/or linguistic barriers to working with technology systems. Jugalbandi, for example, enables interactions in multiple Indic languages simply through the user’s voice. It does this using a combination of ChatGPT, language translation models and a fixed data set from which to answer questions.
Higher education institutions pushing the frontier of data visualization. Researchers at institutions such as are experimenting with using XR to bring representations of data beyond screens into more immersive hybrid environments.
The release of enhanced libraries for gesture recognition designed to help developers connect physical gestures to application features, such as those developed by and .
Continued advancements in natural language processing, including the development of pre-trained models to address specific tasks like , and a growing focus on . These advancements are being leveraged by forward thinking organizations to improve how users find and use information. For example, has created an interactive assistant that goes far beyond what’s possible with parametric search.
Trends to watch
Adopt
-
Using virtual environments to test and model desired physical outcomes in an industrial context.
Analyze
-
AI-powered virtual assistants and non-playable characters that recreate human interaction within the metaverse.
-
Ultra-high resolution XR with photorealistic rendering over a wide field of view. Currently only available via extremely expensive headsets. An example is .
Anticipate
-
Most terms of service (TOS) or end-user license agreements (EULAs) are impenetrable legalese that make it difficult for people without a law background to understand. Understandable consent seeks to reverse this pattern, with easy-to-understand terms and clear descriptions of how customers' data will be used.
-
A collaboration strategy where, using XR, everyone on a hybrid local/remote team interacts with the same shared artifacts, such as whiteboards and other information radiators. This brings the remote collaborators closer to the in-person team.
Adopt
Analyze
Anticipate
-
An Artificial General Intelligence (AGI) has broad capabilities across a range of intellectual tasks, and is often compared to human-level intelligence. This contrasts with today's "narrow" AI which can be remarkable, but only for very specific tasks.
The opportunities
By getting ahead of the curve on this lens, organizations can:
Lower costs by making interactions more efficient. The ability to communicate with a system with the flick of a finger or through voice commands in natural speech, rather than the more laborious process of typing, has the potential to in the workplace and other environments.
Boost satisfaction by reducing friction and enhancing availability for customers. Voice-based platforms promise to make it easier for customers, especially those with accessibility challenges, to retrieve product information or get answers to queries in a smooth, hassle-free way. Similarly, advances in natural language processing opens up possibilities such as using chatbots to provide a base level of service and support outside working hours.
Derive better insights from richer interactions. As AR and VR make it possible to bring data to life outside the screen or printed page, the foundations are being laid for what’s known as . Immersive environments can help users experience data aurally and tactically as well as visually, enhancing understanding, analysis and, eventually, decision-making.
Test out scenarios to improve responses. XR-enabled simulations can be used by enterprises to run teams that could test the business, giving them an accurate sense of their capacity to react and identify areas for improvement.
What we've done
An immersive fan experience for the FC St. Pauli Museum
The Hamburg-based football club museum’s ambition is to create a dynamic and immersive experience that grows from fan engagements and promotes topics such as diversity, sporting events against racism, and wider conversations around social impact. worked closely with FC St. Pauli Museum on a six-week project to design a new interactive fan experience as part of Tech Lab — a initiative that takes innovative ideas and uses cutting-edge technology to bring them to life.
Actionable advice
Things to do (Adopt)
Actively investigate processes or areas that can be improved or replaced by advances in AI and interaction technologies, such as:
Customer interactions, more aspects of which can be supported by chatbots as they grow more sophisticated and .
- Researching and uncovering market trends. Business analysts at have been able to use ChatGPT and other tools such as for ideation and scenario creation.
Things to consider (Analyze)
Monitor the XR/AR space for possible use cases. The benefits for things like training and crisis management, where realistic physical interactions are particularly important, could be substantial. In addition to enterprise contexts, organizations derive value from industrial use cases ranging across design, manufacturing and maintenance applications. The cutting-edge work taking place on is also likely to prove important for how we consume and manipulate data in future.
Understand the diversity of AI tools and how this can serve the needs of people who may want to interact with AI in different ways. Some developers, for example, are comfortable with tools like GitHub Copilot where the actual interaction is largely the same as it has been for some time with autocomplete functions. Others would rather have a ‘conversation’ with a ChatGPT-like solution and use the resulting learnings in their regular internal development environment.
- Learn, and go, where your customers are. Consider your customer base and how likely your average users are to adopt any new platforms for interaction you offer. For example, retailers could focus on augmented reality (AR) interactions which enable richer interactions for consumers in physical locations.
Things to watch for (Anticipate)
- Consider ‘outside the box’ use cases. There’s no question the expanding nature of interactions will result in potential applications for every business, but these applications may not always be obvious. This means it’s important to think through business processes and examine where the ability to present data or engage with customers in an entirely different way might improve the entire experience.