Introduction and motivation
Networking and communications research has resulted in an unprecedented growth in the usage of the mobile Internet, creating an opportunity for companies to develop next-generation smartphones and user experience enhancing devices. The Tactile Internet is the latest evolutionary leap that enables communications between human beings and remote physical and/or virtual objects via the Internet.
The Tactile Internet is expected to create a plethora of new opportunities and applications that will reshape our life and economy. The Tactile Internet may be used in the fields of defense, medicine, education, and many more. It may play a pivotal role in breaking the social and economic barriers of today's society. Fig. 1 shows the evolutionary leap of the Tactile Internet according to a 2014 ITU-T Technology Watch Report. The high availability, security, ultra-fast reaction times, and carrier-grade reliability of the Tactile Internet will add a new dimension to the human-machine interaction by enabling tactile and haptic sensations.
Fig. 1. Evolution of the Tactile Internet [1].
A very low round-trip latency in conjunction with ultra-high reliability and essentially guaranteed availability for control communications has the potential to move today’s mobile broadband experience into the new world of the Tactile Internet for a race with (rather than against) machines. By building on the areas where machines are strong and humans are weak, machines are more likely to complement humans rather than substitute for them. The value of human input will grow, not shrink, as the power of machines increases.
To facilitate a better understanding of the Tactile Internet, it is helpful to compare it to the Internet of Things (IoT) and 5G mobile networks as well as elaborate on their commonalities and subtle differences. There is a significant overlap among IoT, 5G, and the Tactile Internet, though each one of them exhibits unique characteristics. For illustration, Fig. 2 provides a view of the aforementioned commonalities and differences through the three lenses of IoT, 5G, and the Tactile Internet. The major differences may be best expressed in terms of underlying communication paradigms and enabling devices. IoT relies on machine-to-machine (M2M) communications with a focus on smart devices (e.g., sensors and actuators). In co-existence with emerging machine type communication (MTC), 5G will maintain its traditional human-to-human (H2H) communication paradigm for conventional triple-play (voice, video, data) services with a growing focus on the integration with other wireless technologies (most notably WiFi) and decentralization. Conversely, the Tactile Internet will be centered around human-to-machine (H2M) communications leveraging tactile/haptic devices. Despite their differences, IoT, 5G, and the Tactile Internet seem to converge toward the following common set of important design goals:
- Very low latency on the order of 1-10 ms
- Ultra-high reliability with an almost guaranteed availability of 99.999 percent
- H2H/M2M coexistence
- Integration of data-centric technologies with a particular focus on WiFi
- Security
Fig. 2. The three lenses of IoT, 5G, and the Tactile Internet: Commonalities and differences [1].
Once machines become connected to the Internet, the next natural leap is to have them controlled remotely. This will generate a completely new paradigm for control communications to steer/control elements of our physical-virtual surroundings and environment. Physical objects may be no longer disconnected from the virtual world and may be controlled remotely. This development is opening up huge opportunities for both economy and society. While the IoT opened up many opportunities for automation using a wide array of Internet-enabled gadgets, the Tactile Internet aims at reclaiming and increasing the human presence by using robotics and state-of-the-art remote control technologies. To enable haptic feedback, deliver physical sensations in real time, and enable networked control systems that work in such an intertwined and immediate fashion, it is key to make sure that highly dynamic processes can be automated or controlled remotely [2]. In particular, the seamless integration of physical and virtual objects will play a pivotal role in the emerging Tactile Internet.
Extended reality: VR/AR/MR and human-machine interaction
As already pointed out by Bill Gates back in 2007, the robotics industry is developing in much the same way that the PC business did 30 years ago. In his vision, we may be on the verge of a new era, when the PC will get up off the desktop and allow us to see, hear, touch, and manipulate objects in places where we are not physically present. Wirelessly linking domestic robots to PCs and using their fiber broadband connectivity could provide many benefits, e.g., cleaning floors, folding laundry, and physical assistance or even companionship for the elderly, possibly from thousands of miles away. By delegating processing-intensive tasks such as visual recognition and navigation to the high-performance hardware of personal and in-home computing facilities, the robot may be a relatively inexpensive device [3].
A recently emerging trend is the so-called extended reality (XR), which is a combination of real and virtual world and human-machine interaction. In the future, users will be able to seamlessly switch to any form of virtual, augmented, and mixed reality with the help of artificial intelligence. According to Qualcomm, XR will be the next-generation mobile computing platform that brings the different forms of reality together in order to create a reality-virtuality continuum for the extension of human experiences.
Virtual reality (VR) is an immersive 3D environment created by a computer. In VR, 3D is essential for simulating the real world. VR allows users to experience the real world and observe objects according to their demands rather than being restricted by the defined objects. Applications of VR have significantly expanded the remote monitoring domain for human beings [4]. Augmented reality (AR) enables the live view of a physical, real-world environment, whose elements are augmented by computer-generated perceptual information, ideally across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory information. The overlaid sensory information can be constructive (i.e., additive to the natural environment) or destructive (i.e., masking of the natural environment). AR may be perceived as an immersive aspect of the real environment [5, 6]. In this way, AR alters one’s current perception of the real-world environment, as opposed to VR, which replaces the real-world environment with a simulated one. Mixed reality (MR) combines the immersive capabilities of VR with AR to bring data into the physical world. MR is the mix of real and virtual worlds, where physical and digital objects co-exist and interact in real time. This powerful combination makes it a medium suited to both consumer and enterprise domains and marks a new paradigm, in which people interact with an increasingly data-rich world. The ability to blend digital information and objects seamlessly with our physical world creates a huge potential how we use computers [7]. MR has the potential to change how people work, communicate, and relate to the world.
Many enterprises already leverage various types of immersive experiences, including AR, VR, and MR. Out of these three types of reality, MR has become the goal for many enterprises due to its distinctive capabilities compared to other immersive experiences. MR is capable of producing enterprise-focused applications with the best combination of both physical and digital worlds. This is why many enterprises invest in the development of MR applications to deliver digital experiences to their customers. Fig. 3 illustrates how MR helps bridge the gap between real and virtual environments.
Fig. 3. Mixed reality bridges the gap between real and virtual environments [8].
More specifically, in Fig. 3, augmented virtuality describes cases, where a real object is inserted into a computer-generated environment. Conversely, augmented virtuality occurs in a virtual environment. An aircraft maintenance engineer, who is able to visualize a real-time model of an engine in flight is an example of augmented virtuality as it occurs on a computer screen with real-world elements that are physically thousands of kilometers away [9]. Unlike augmented virtuality, augmented reality uses the existing natural environment and simply overlays virtual information on top of it. As both virtual and real worlds harmoniously coexist, users of augmented reality experience a new enhanced natural world, where virtual information is used as a tool to provide assistance in everyday activities. MR includes augmented virtuality, augmented reality, and mixed configurations thereof and blends representations of virtual and real-world elements together in a single user interface. The difference between augmented reality and augmented virtuality boils down to where the user interaction takes place. If the interaction happens in the real world, it is augmented reality. In contrast, if the interaction occurs in a virtual space, it is considered augmented virtuality. For convenience, Table 1 summarizes the major differences between VR, AR, and MR that together constitute XR.
The flagship of MR devices is Microsoft's HoloLens, which is the first self-contained, holographic computer, enabling users to engage with digital content and interact with holograms in the world around them. As illustrated in Fig. 4, possible use cases include but are not limited to:
- Visualizing complex projects, processes, and designs prior to signing off
- Training distributed workforces on complex tasks
- Showing data in situ to facilitate real-time decision-making
Fig. 4. HoloLens: The first self-contained, holographic computer, enabling users to engage with digital content and interact with holograms in the world around them [11].
Allowing customers to touch and feel in the virtual world can be extremely powerful, especially for object manipulation. During the last years, e-commerce and online shopping in particular have embraced emerging VR/AR/MR technologies for tying our online and offline worlds closer together.
Online-to-Offline/Offline-to-Online (O2O) communications
Commerce models have constantly changed with the evolution of technology. O2O is a new electronic commerce mode that has become popular among consumers and profitable for e-vendors. Typically, O2O commerce aims to find customers online and bring them into real-world stores. O2O efficiently integrates and optimizes the application of online resources with offline business opportunities and uses the Internet as the interface of offline transactions. The business process of O2O starts with service providers presenting information about products or services to customers online, followed and searching for a customer’s goods and services online. Users may order as well as process the payment online but experience the services or products offline [12].
O2O commerce is big business, though as already mentioned above it is very hard to launch a brand these days that is online only. According to Sucharita Mulpuru, a retail analyst at Forrester Research, the e-commerce environment becomes increasingly crowded and difficult. In fact, the O2O aspect of businesses is becoming so important that even companies, which are successfully 100% online, now offer offline locations. For instance, Amazon announced in late 2016 that they will open more than 2,000 Amazon Go grocery and convenience stores across the US. The scale of the O2O market has been growing and well-known companies have started to invest in this industry. A good example is again Amazon and their recent $13.7 billion purchase of Whole Foods in 2017, which shows where the leader in online commerce is placing its bets. It should be noted that according to recent studies, only 8% of retail sales are done online while more than 80% of retail sales will still happen in physical locations until 2020 [13].
Beside Amazon, it is interesting to note that all top-10 retailers apply a brick-and-mortar operation. Among others, Alibaba intends to adjust and personalize their product offerings, create their tailored marketing campaigns, and streamline their supply chains. Towards this end, Alibaba has attempted to take advantage of the world’s biggest e-commerce market by investing as much as $8 billion in retail stores in China [14]. In the US, Nordstrom is one of America’s largest fashion retailers that has introduced data-driven, personalized experiences in their retail stores. Their marketing team tracks Pinterest pins to help identify trending products and promote them in their brick-and-mortar stores. Another example is Amazon’s drone delivery, which is a delivery system designed to safely get packages to customers in 30 minutes or less by using unmanned aerial vehicles. Amazon’s drone delivery service is one of a number of promising offline and online resource strategies to create win-win situations [13]. Other similar O2O business strategy examples include Uber, Airbnb, and Groupon.
O2O communications has expanded the scope of businesses and e-commerce. O2O communications integrates the virtual and real worlds and gets support and recognition from all walks of life. O2O communications aims to maximize the use of offline and online resources by promoting each other in order to achieve win-win situations [12]. Fig. 5 illustrates the details of the O2O commerce service model. The model provides information, services, discounts, and ways to store offline messages via an O2O platform that is pushed to online consumers such that they buy goods and services online and then go to an offline store to enjoy the service and experience. More specifically, the top of Fig. 5 represents the real-world marketing service model, where manufacturers produce and deliver their products through various channels and stores. By using location-based services and proximity commerce marketing strategies, retailers may motivate consumers to use their mobile devices to interact with a digital signage or kiosk to get online coupons and subsequently shop in a nearby physical store. The bottom of Fig. 5 represents the online marketing service model. Users may read other customers’ opinions online before making a decision. They click on likes, share, and post comments on social events while retailers may monitor online word-of-mouth (WoM) to adjust their marketing strategies and sustain a relationship with their customers directly and instantly. Moreover, retailers may organize social events to influence and engage consumers’ social network friends to participate and provide referral awards or group purchase discounts.
Fig. 5 O2O commerce service model [15].
In recent years, access to online stores has been offering almost infinite opportunities for customers to purchase products and services. More and more companies go online, thus making it harder for many companies to stand out or even to be seen at all. While in the US, Canada, and the UK O2O commerce is often limited to click-and-collect and deal sites, it is important to note that there exist many innovative ways O2O communications may be used. In fact, in China, it is already integrated into everyday life, ranging from transportation to eating and shopping. Clearly, North America could learn a lot from Asia when it comes to O2O communications and how it is used to integrate technologies and companies into our everyday life [13]. Towards this end, Table 2 compares the advantages and disadvantages in of the online and offline domains in greater detail.
The development of e-commerce is limited since the tangible industry cannot be completely replaced by the service industry. In particular, new electronic products have exploited customer experience as tenet to retain customers through better service experiences. With E-commerce in its business essence it is difficult to achieve good services, which is one important reason why the tangible industry cannot be replaced at the moment. As a consequence, the world’s fastest-growing companies leverage O2O communications. On the other hand, however, it’s interesting to note that 70% of millennials prefer shopping in brick-and-mortar stores, which is surprising given that they spend an average of 7.5 hours a day online. This is due to the fact that many customers want to have an experience that allows them to hold, touch, and in some cases try on products. Hence, multi-modal feedback would allow consumers to physically interact with a product, whereby the offline store does not necessarily need to be a retail shop, but an experience instead.
Research problems
Although there has been considerable growth in the online shopping market in recent years, the largest consumption market is still offline. E-commerce is now moving towards new fields using the O2O model with broad development prospects. Thus, studying XR applications has great significance for the expansion of O2O business and improvement of e-commerce implementation by tackling the following open research problems, among others:
Low customer trust and loyalty
With the development of electronic commerce, customer trust has become a prerequisite and guarantee for successful online transactions. According to recent studies, the trust in O2O e-commerce is still. The nature of products and services and the extent of information disclosure will have some impact on consumer trust. Providing more detailed information about services helps increase a sense of trust among consumers.
Detailed information about products and services
To provide detailed information about products and services, MR systems have been designed such that users can understand digital objects in a space similar to the physical environment and mitigate the user experience problem. In this way, trust in the product increases in terms of quality, thereby resulting in an increased willingness among customers to buy and use it.
Consumer experience
A new interaction dimension that can be incorporated into online virtual worlds is the sense of touch or haptics. In e-commerce, haptic feedback would allow consumers to physically interact with a product. Human hands would be able to test a product by feeling the warm/cold, soft/hard, smooth/rough, and light/heavy properties of surfaces and textures that compose a product, e.g., bed linens and clothes, in order to try them on before purchase. Work has started in the field of haptic-enabled e-commerce. For instance, Costanza creates the online experience of buying a car.
Research directions
The separation of the environment in O2O communications creates uncertainty and risk of the transaction, which is greater than in traditional face-to-face transactions. The research directions related to providing a solution based on XR to increase customer trust via O2O communications are as follows:
- Studying accurate real-time synchronization with the registration of virtual and real elements as a definite feature of XR systems, which poses a difficult technical challenge difficult to be accomplished
- Investigating the high bandwidth, low latency, high stability, and synchronization requirements of haptics that are not met by current state-of-the-art solutions
- Overcoming obstacles with regard to real-time processing and calibration of physical environments
- Investigating networked XR applications, where latency is universally detrimental as it may cause not only a time lag between human operator and feedback, but also system instability due to vibrations of reaction forces
- Combining the physical and digital worlds in ways that amplify positive customer experiences
- Exploiting customers' real-world perceptual and action skills to work with objects in the XR world intuitively, paying particular attention to the limitations of their intuitiveness
- Investigating nudging in advanced XR and O2O communications based technologies to change user behavior and take into account the lack of precision of the human perceptual and motor skills in viewing and interacting with mixed realities
Researchers
Graduate Student
Advisor
Report
- M. Maier and A. Ebrahmizadeh, "Toward the Internet of No Things: The Role of O2O Communications and Extended Reality," June 2019: https://arxiv.org/abs/1906.06738
References
[1]
|
M. Maier, M. Chowdhury, B. P. Rimal, and D. P. Van, "The Tactile Internet: Vision, Recent Progress, and Open Challenges," IEEE Communications Magazine, vol. 54, no. 5, pp. 138-145, May 2016.
|
[2]
|
Z. He, F. Zhu, and K. Perlin, "PhyShare: Sharing Physical Interaction in Virtual Reality," In Proc., ACM Symposium on User Interface Software and Technology, Quebec, Canada, pp. 17-19, Oct 2017.
|
[3]
|
M. Maier, “FiWi Access Networks: Future Research Challenges and Moonshot Perspectives,” In Proc., IEEE Int’l. Conf. Commun. (ICC), Workshop on Fiber-Wireless Integrated Technologies, Systems, and Networks, Sydney, Australia, pp. 371–75, June 2014.
|
[4]
|
G. S. Chen and J. P. Chen, "Applying Virtual Reality to Remote Control of Mobile Robot," In Proc., International Conference on Intelligent Technologies and Engineering Systems (ICITES), Kaohsiung, Taiwan, pp. 383-390, April 2014.
|
[5]
|
S. Chi. Y. Yuen, G. Aoyuneyong, and E. Johnson, "Augmented Reality: An Overview and Five Directions for AR in Education," Journal of Educational Technology Development and Exchange (JETDE), vol. 4, no. 1, pp. 119-140, Oct 2011.
|
[6]
|
P. Parvinen, J. Hamari, and E. Poyry, "Introduction to the Minitrack on Mixed, Augmented and Virtual Reality," In Proc., International Conference on System Sciences, Hawaii, USA, pp. 1395-1396, Jan 2018.
|
[7]
|
M. K. Bekele, R. Pierdicca, E. Frontoni, E. S. Malinverni, and J. Gain, "A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage," Journal on Computing and Cultural Heritage (JOCCH), vol. 11, no. 2, pp. 1 - 36, March 2018.
|
[8]
|
P. Chen, X. Liu, W. Cheng, and R. Huang, "A Review of Using Augmented Reality in Education from 2011 to 2016," In Proc., ACM/IEEE, Innovations in Smart Learning, Chicago, USA, pp. 13-18, Sep 2016.
|
[9]
|
T. Williams, D. Szafir, T. Chakraborti, and H. Ben Amor, "Virtual, Augmented, and Mixed Reality for Human-Robot Interaction. HRI companion of the 2018", In Proc., ACM/IEEE International Conference on Human-Robot Interaction, Chicago, USA, pp. 403-404, Mar 2018.
|
[10]
|
A. Ricci, M. Piunti, L. Tummolini, and C. Castelfranchi, "The Mirror World: Preparing for Mixed-Reality Living," IEEE Pervasive Computing, vol. 14, no. 2, pp. 60-63, April- June 2015.
|
[11]
|
https://www.microsoft.com/en-us/hololens
|
[12]
|
J. Wan, Q. Jiang, and G. Pan, “Research on Performance Influence Factors of O2O Website with Interpretative Structure Model,” In Proc., International Conference on e-Business (WHICEB), Wuhan, China, pp. 31-41, May 2017.
|
[13]
|
Z. He, T. C. E. Cheng, J. Dong, and S. Wang, "Evolutionary Location and Pricing Strategies for Service Merchants in Competitive O2O Markets," European Journal of Operational Research, vol. 254, no. 2, pp. 595-609, Oct 2016.
|
[14]
|
J. Zhang, "Customer Loyalty Forming Mechanism of O2O E-commerce," International Journal of Business and Social Science, vol. 5, no. 1, pp. 164-169, April 2014.
|
[15]
|
T. M. Tsai, W. N. Wang, Y. T. Lin, and S. C. Choub, “An O2O Commerce Service Framework and its Effectiveness Analysis with Application to Proximity Commerce,” Procedia Manufacturing, vol. 3, pp. 3498-505, Jan 2015.
|
|