Lecture
Immersive technologies are also called extended reality technologies. The list includes virtual and augmented reality, as well as 360° video. They create an effect of full or partial presence in an alternative space and thereby change the user experience in a wide range of completely different fields. In this post we’ll talk about where these technologies stand today, how they are implemented and how they are used in business.
First, let’s define the terms and abbreviations we’ll be talking about:
The list of “other realities” can also include:
Visuo-haptic mixed reality ( VHMR ) - a branch of mixed reality that is able to fuse visual and tactile perceptions of both virtual and real objects.
Marketers who apply the term VR to 360-cameras are, at best, being careless with terminology.6
SR (substitutional reality). In one of the laboratories of Japan’s RIKEN institute for physical and chemical research, a “reality substitution system” has been developed that merges fiction and reality in a person’s mind in such a way that, ideally, they become impossible to tell apart.7 At the current stage of technological development this term can be considered a synonym for virtual reality. But in the distant (or not so distant) future it may be created.
Simulated reality. This is a hypothetical reality indistinguishable from real reality. Simulated reality may contain sentient entities that may or may not be aware that they are living inside a simulation. This is quite different from the currently existing, technologically achievable concept of virtual reality. Virtual reality is easy to tell apart from actuality; participants immersed in it never doubt the nature of what they are experiencing. Simulated reality, by contrast, would be difficult or impossible to separate from real reality.
RAC: a two-dimensional reality–virtuality continuum.
Now let’s take a closer look at what lies behind these abbreviations.
Available in cameras for quite a while now, supported by Facebook since 2018. Essentially these are sequential sets of shots stitched together using algorithms. They can be taken with a single camera as well as with dedicated 360° cameras.
How it works. 360° cameras film the surrounding space, after which the resulting footage is stitched together in dedicated software. Seamless solutions also exist, but they are more costly for businesses. Sometimes additional graphics are added to the finished video.
“Panoramic” online broadcasts are also common these days. When you have several points with a panoramic view, it’s enough to set up the equipment and start filming. But this format has fairly demanding hardware requirements, and it also requires careful audio setup.
Where it’s used. This technology is widely used in real estate showings. A simple Google search turns up a host of B2B agencies that create virtual tours for realtors and developers: 360 Real Estate, 360walkthruproperty, Forj and others.
Like many innovative technologies, augmented reality first appeared in the middle of the last century for military purposes. It was planned, and later successfully applied, for projecting important instrument readings onto a pilot’s helmet visor or a fighter jet’s windshield, in order to reduce the time needed to obtain information and increase pilots’ reaction speed.
Today several directions of augmented reality are developing. The most widespread is AR on smartphones. Apple has been developing its own ARKit for several years now. Today there are more than 5 billion smartphones in the world, and that potentially represents a very large reach for AR products. A striking example of success is the game Pokemon Go, which took the whole world by storm. There are marketing projects in which, for example, coins are scattered in AR and the user has to search for and collect them in order to get a discount. The coins are scattered in such a way that, like breadcrumbs, they lead the user to the doorstep of a store. It’s also worth mentioning the recently launched Google Street project in the US, which helps with navigation. Simpler scenarios are also in demand: for example, in 2017 Positive Technologies released merchandise — a calendar and a planner — where the pictures came to life when a phone was pointed at them.
Other devices for working with augmented reality are augmented-reality glasses. Google Glass — a project that hasn’t quite succeeded yet. Hololens, a Microsoft project — is an expensive solution, but its advantage is that it’s immediately integrated with the company’s ecosystem. In addition, there are many other augmented-reality glasses on the market — from Ericsson, Huawei and many other large companies and startups.
Separate solutions are being implemented for industry on a subscription model that includes consulting and equipment repair.
How it works. Projecting a virtual object into the real world can be done via a phone. Markers are often used, as is projection onto the glass of a car, an airplane, or glasses — for example, in Microsoft Hololens glasses. If virtual reality technology is used outdoors, special attention needs to be paid to lighting, since in very bright light AR objects can look dim. Overall, even though it’s not the most polished technology, it still allows for entertainment applications to be built.
The mixed reality continuum (left) and its extended version for remote physical experiments (right)
Where it’s used. Holograms, overlays on live TV broadcasts, the use of interactive screens — these are the embodiments of AR that already feel familiar to us. However, there are also more unexpected examples of the technology’s use.
For example, Pepsi Max used AR technology to create one of its commercials. The company installed special augmented-reality screens at one of London’s bus stops and filmed the reactions of random passers-by.
Another example — interior planning at IKEA.
Augmented reality is, as I already said, one of the secrets behind the success of Pokemon Go. It’s also the standout feature of the Argon4 app built on the AR Browser SDK, which can, for example, be used to provide reference information about landmarks when a smartphone camera is pointed at them.
As for AR in glasses, this area is still less developed, since current models are fairly bulky and expensive. But every year the glasses get lighter and more appealing. Another direction for AR is projection onto contact lenses. Its drawback so far is that a grid is visible on the lens. But this method has plenty of advantages as well.
How it works. Virtual reality is, first and foremost, visual and audio content. Sound is key here: it complements the virtuality and creates a sense of presence in an unreal location by simulating the reflection and direction of sound waves.
You can enter an alternative, virtual reality by, for example, putting on special glasses. By splitting the image in front of your eyes into two parts, they create a stereoscopic effect. With positional tracking for the body, the virtual space will also account for the movements of the head and torso.
There are other ways to enter virtual reality as well:
Almost all the major phone and laptop manufacturers have released their own virtual reality headsets: they are made by HTC, Lenovo, Xiaomi, Samsung, Oculus and other companies. Some are trying to build an ecosystem: for example, HP promotes a lineup of laptops and headsets with controllers and external tracking.
For such solutions a different name is already used — mixed reality. In fact it’s more of a marketing term; it appeared alongside the release of Windows Mixed Reality — a platform for developing applications. In essence it’s a construction kit with 3D models and a certain set of bundled assets. In terms of the quality of the product it can create, the Windows solution is far weaker than the Unreal Engine and Unity engines. Unlike the Windows Mixed Reality platform, they offer not only the ability to develop and create virtual objects, work with textures and lighting, but also to build a certain amount of business logic.
Where is VR used?
The key industries in which VR can be used are
IKEA carried out an interesting project in the field of VR. The company invited its customers to put on special glasses and find themselves in a virtual kitchen. There, using a joystick, they could read a recipe book, prepare meatballs, and send groceries for recycling.
The German carrier Deutsche Bahn is also experimenting with virtual reality. For four years now the company has been inviting its prospective employees to put on VR glasses and demonstrate their abilities in a simulation of a real working day.
The Israeli technology company Actiview has gone even further and developed a platform for remote hiring of employees using VR technologies. “Simulation in virtual reality lets us control what the user sees, hears and feels. We see their behaviour and can collect this data,” comments Roy Elishkov, the company’s vice president for strategy and business development.
Russian companies are also working with VR applications in various fields. For example, Modum Lab creates VR projects for training and other scenarios.
Table 1. Summary of the differences between reality and virtuality.
| Real Environment (RE) | Augmented Reality (AR) | Pure Mixed Reality (PMR) | Augmented Virtuality (AV) | Virtual Environment (VE) | |
|---|---|---|---|---|---|
| The primary environment is either the virtual world (V) or the real (R) world. | R | R | R | V | V |
| Users interact with the virtual (V), real (R), or both (RV) worlds in real time. | R | RV | RV | RV | V |
| Digital content is overlaid onto the real environment. | - | √ | - | - | - |
| Real content is overlaid onto the virtual environment. | - | - | - | √ | - |
| Digital content is merged with the real world, so that both digital and real content can interact in real time. | - | - | √ | - | - |
Virtual content in PMR is not overlaid onto the physical environment (as in AR), but virtual objects are rendered in such a way that they are indistinguishable from the physical world. Visual consistency is the core element of pure mixed reality (
Finally, let me try to sum up the industry’s experience and suggest what comes next. In striving for realistic imagery, we run into the limits of graphics and information-processing speed. Hybrid systems involving the cloud may be the way out. At the same time, the current “cartoonish” image still allows for “immersive” content to be made, since that is enough for the brain. Try walking across a VR plank over a VR chasm.
VR headset penetration is currently small, so we set up VR installations in our offices and invite consumers there. It’s clear that this approach has potential for selling expensive and complex products — real estate, yachts, cars, sightseeing tours. The key here is the transition from ad hoc solutions, that is, applications implementing a specific use case, to full-fledged XR platforms that will let buyers and sellers find each other.
There is still no sign of rapid monetization in the field of immersive technologies; players are acting as if feeling their way in the dark. The reason is that not that many people use extended reality in everyday life. To try it out you need to download an additional app at the very least — and that complicates the path by which the technology reaches the end consumer.
None of these developments involve significant gamification; on the contrary, there is a drive toward realism. Creating tools for effortless interaction is precisely the main task for developers — content development is currently secondary to the development of UX/UI. Image resolution is still low and it isn’t photorealistic yet, so the technology still has plenty of room to grow and develop. Improving the user experience is tied to processor performance and to the speed of data transmission and processing.
Hybrid solutions may appear soon: for example, image processing could be carried out on a server online, after which it would be transmitted over radio, Wi-Fi or Bluetooth to the user’s mobile headset. For now, though, this is only speculation. In any case, we are confident that immersive technologies will find applications across many industries.
Comments