Looking through the Jio Glass
The promise of Augmented Reality hardware has been immense, but so have been the number of misses. Can Jio’s next hardware play be any different?
Reliance has been on a tear in the last few months. Jio Platforms, has raised nearly $16B in four months. A company that was built on the core of oil & gas did not have those terms being uttered through their Annual General Meeting (AGM) in the first 60 minutes. The next bet, looking forward, among many things is a pair of mixed reality glasses called Jio Glass.
The Basics: What is Jio Glass?
Jio Glass launched as mixed-reality glasses that serve primarily for education & entertainment. The core tech specs and details about how the glass works are scarce during the launch. However, some of the underlying technologies, principles and potential paint for a very interesting picture on how a country with over a billion people could have access to consumer grade wearable computing.
Reliance Jio’s foray into mixed reality started in 2019 when they acquired majority stake in deep-tech startup Tesseract which has a few hardware products in XR space. At their 42nd AGM in 2019, the company had showcased a ‘Holoboard AR headset’ made by Tesseract, but did not ship any to consumers; and this seems to be an iteration by them, over the last year.
The Anatomy of Consumer Grade Augmented Reality
For Jio Glass to be successful, when it launches, it has to not just be a good product but also build an ecosystem around it. Broadly, the important tiers are:
- Infrastructure: The data the connects your glass
- Hardware: Does the glass work?
- Operating System: What does ambient computing look like?
- Apps & services: Third-party developer adoption
- Content: The things that keep you coming back for more
How close are we to realizing this? We are not living in this future yet, but there is a strong case to be made that we are accelerating towards it, in 2020, faster than ever before.
The Jio Glass Hardware
Before we get into the weeds about the device itself, it’s useful to rehash the spectrum of Augmented Reality / Mixed Reality. Hardware devices come in multiple shapes and forms, and can be categorized broadly into the following:
Jio Glass can be categorized into the Smart Glasses category, given what we know right now. During the launch, the following details were laid out by Jio:
- Focus on collaboration
- Product weight of 75 grams
- ‘Personalized’ audio features
- Support for over 25 applications
A few other details emerged during the course of the event too.
It is connected to your phone via USB-C
Smartphones have become processing powerhouses. Your average smartphone has a generous amount of computational headroom. Jio Glass and an emerging number of ‘XR Viewers’ tend to take advantage of tethering to your smartphone using a cable, and offload processing requirements to the phone. If you want to learn more about how this approach became popular over the others, I had previously written in-depth about the growing potential for smartphone tethered wearable computing you can read about here.
Possible display technology:
While details are light (ha), I suspect the device has a type of display that’s now fairly common in AR hardware, known as a BirdBath display. The birdbath combines two main optical components, a spherical mirror/combiner (part-mirror) and a beam splitter. The name “birdbath” comes from the spherical mirror/combiner looking like a typical birdbath. It is used because it generally is comparatively inexpensive while also being relatively small/compact while having good overall image quality.
This design has a medium FOV around 40-50 degree Field of View, so you’re still going to be seeing most of the mixed reality content through a narrow box in front of your vision.
The Problem? Light loss: In a 50/50 beamsplitter, there is a 50% light loss on the first bounce, additional loss when the light hits the combiner, and another 50% loss as the light rays travel back through the beamsplitter and are eventually projected onto the eye. To compensate for the significant loss of light the lenses are very dark, similar to wearing sunglasses indoors.
That’s not all. For a good Mixed Reality experience, the degree of immersion is key in making you believe what’s in front of you is the real deal. This has 3 main components to it, dictated by hardware:
Wait, how is this different from things like Microsoft’s Hololens?
Devices such as the Magic Leap and the Hololens rely on Waveguide optics. Waveguides represent a new form of optics technology, while still under development offers significant advantages in terms of form factor, clarity and ghosting. The display source is usually side mounted, using a Liquid Crystal on Silicon (LCOS) or DLP display. Both LCOS and DLP shoot collimated light rays through a diffraction grating that redirects light and eventually forms an expanded image, which is then projected it into the eye.
While there is a lot of excitement to waveguide displays and the high-resolution AR experience, they are also the most expensive to manufacture and have significant reject rate. The prices of waveguides are expected to come down with better manufacturing processes, but today they are still a significant amount of cost for a high-end headset. Headsets with waveguides are retailing in the $3000 to $5000 range. However, current birdbath display XR viewers are in the $400 to $500 range.
So, Jio Glass is the first of it’s kind?
Qualcomm announced a strategic investment in Jio not too long ago, and have been shepherding the XR space with similar devices broadly labeled as ‘XR Viewers’
There have been a robust line of devices that have launched under the Qualcomm XR Viewer program and operate under the same principles that Jio Glass aims to. Connect to your smartphone, with split rendering of visuals via an affordable, light weight wearable glass that’s on you.
The XR visual processing pipeline is both compute intensive and latency sensitive. Splitting the processing correctly requires a system approach. To solve that, Qualcomm has a dedicated line of silicon to power these devices in conjunction to the smartphones in your pocket. They are known as the XR line of processors and the XR2 currently powers a few smart glasses in other markets such as China.
The tiny chip has to perform a number of tasks parallelly with the smartphone to allow a smooth mixed reality experience. All of this, without thermal issues or rapid power drain as you don’t want your glasses getting hot or drain your phone through 30 minutes of usage. It’s one of the core reasons why the world of Augmented and Mixed Reality is as notoriously hard to crack, and some of these functions are:
What went wrong with the previous AR hardware?
We’re looking at you, Google Glass.
Besides Google Glass, there have been plenty of venture backed companies promising to make consumer grade Augmented Reality come to life, only to realize the scale of the problem is larger than originally anticipated. The technology was almost there in a lot of cases, but the real issue was that the stakes to beat the major players to market were so high that many entrants pushed out boring, general consumer products. In a race to be everything for everybody, the industry relied on nascent developer platforms to do the dirty work of building their early use cases, which contributed heavily to nonexistent user adoption.
A key error of this batch was thinking that an AR glasses company was hardware-first, when the reality is that the missing value is almost entirely centered on missing first-party software experiences. To succeed, the next generation of consumer AR glasses will have to nail this.
Operating System
While we know very little about Jio Glass or it’s underlying operating system, we can once again look towards similar XR Viewers and borrow possibilities on how Jio Glass might serve content to users. Specifically, Nreal.ai has been one of the most anticipated devices to launch in the recent past. It currently sports an Android fork that they call Nebula UI. Nebula transforms your current environment into a user interface, enabling you to place persistent app screens within a physical space and interact with them at will. You can see it in action below:
Why build on Android? Besides the announced partnership with Google, this provides for easy developer support and backward compatibility for 2D apps to be ‘pinned’ to your peripheral vision. In addition, a tethered Android smartphone serves as a 3DoF controller to laser-point at icons and reposition app windows as you want them in 3D space.
Going back to Jio Glass’s promotional material talking about placing 2D applications into your world, only strengthens the case for this approach.
Critically, Jio Glass can potentially serve as a bridge between the host device’s OS (Android) and the glasses, enabling traditional Android apps to be accessed while wearing the Jio Glass— including tablet-class versions of Facebook, YouTube, and other apps that are displayed in wide rather than tall windows.
However, the biggest advantage with this approach might just be the fact that developers need to do no work / minimal work to make their applications 2D compatible with Jio Glass, and that brings me to the most important part of this category.
Applications & Services
What if I told you that everything that went into making the Jio Glass or any other Augmetned Reality device till this step were actually the easy parts?
The existence of first party software experiences, applications and services will single handedly decide the success of the Jio Glass or any other smart glass that becomes a consumer market success.
The AR Content Chicken & Egg
Building augmented reality or mixed reality content is hard. Very hard. It takes plenty of time, trial and working on game engines such as Unity & Unreal to get close to the fidelity that consumers have come to demand from even the most rudimentary smartphone apps. However, developers need to be incentivized to build atop new platforms because if there aren't any users for your cool new application in mixed reality, it doesn’t matter and vice-versa. If there are no applications for you to try out when your purchase your Jio Glass, it’s going to be stowed away and forgotten (remember Windows Phone?).
Developer adoption is absolutely critical in making this new platform successful, and a step in the right direction is to seed applications and use cases out of the box when the device ships. Out of 25 such claimed applications, one such that was shown on stage was around collaboration.
There are a million things that are unclear from these visuals, such as the full-passthrough backgrounds. However, an interesting part of the collaboration demo was the utilization of 3D avatars to help better showcase immersion and virtual embodiment. The visual language is very similar to a popular XR collaboration service called Spatial.io. How similar? see below:
Disruptive technologies, though they initially can only be used in small markets remote from the mainstream, are disruptive because they subsequently can become fully performance-competitive within the mainstream market against established products. This happens because the pace of technological progress in products frequently exceeds the rate of performance improvement that mainstream customers demand or can absorb. As a consequence, products whose features and functionality closely match market needs today often follow a trajectory of improvement by which they overshoot mainstream market needs tomorrow. And products that seriously underperform today, relative to customer expectations in mainstream markets, may become directly performance-competitive tomorrow
Hence, seeding content and selecting education and entertainment as beachheads can prove to be effective for Jio Glass to wedge it’s growth into markets that prize content at first, build a library of content before scaling. Whilst they attempt to solve for India, things are moving fast back in Silicon Valley.
What about Apple Glass?
Apple’s aggressiveness in areas like wearables and, at least from a software perspective, augmented reality, suggest the company will press its hardware advantage to get to the future before its rivals, establishing a beachhead that will be that much more difficult for superior services offerings to dislodge. The best way to think about Apple has always been as a personal computer company; the only difference over time is that computers have grown ever more personal, moving from the desk to the lap to the pocket and today to the wrist (and ears). The face is a logical next step, and no company has proven itself better at the sort of hardware engineering necessary to make it happen.
Critically, Apple also has the right business model: it can sell barely good-enough devices at a premium to a userbase that will buy simply because they are from Apple, and from there figure out a use case without the need to reach everyone. I was very critical of this approach with the Apple Watch — it was clear from the launch keynote that Apple had no idea what this cool piece of hardware engineering would be used for — but, as the Apple Watch has settled into its niche as a health and fitness device and slowly expanded from there, I am more appreciative of the value of simply shipping a great piece of hardware and letting the real world figure it out.
Moreover, there is evidence that Jio sees the value in Apple’s approach: the company’s push into hardware may in part be an attempt to find a new business model for Jio Platforms, but establishing the capabilities to compete in hardware beyond the smartphone is surely a goal as well. It is increasingly becoming evident that sheets of smartphone glass are simply no longer the most fertile ground for innovation.
Other Jio patterners such as Facebook too, have been deeply invested in realizing VR/AR as the next computing platform. To quote Mark Zuckerberg from one of his keynotes:
We experience the world through this feeling of presence and the interactions that we get with other people, which is why Facebook’s technology vision has always been about putting people at the center of your computing experience. We’ve mostly done that so far through building apps. I don’t think it’s an accident that a lot of the top-used and biggest apps that are out there are social experiences that put people at the center of the experience, because that’s how we process things.
But there is only so much you can do with apps, without also shaping and improving the underlying platform. I find it shocking that we’re here in 2019 and our phones and our computers are still organized around apps and tasks and not people that we are actually present with. I feel like we can help all of us together deliver a unique contribution to this field by helping to ensure that the next platform changes this.
Infrastructure
What’s in it for Jio in the long term?
During the AGM event, Jio’s 5G plans were outlined too, and this fits well with their bullish view on wearable computing and Jio Glass. Much like the headroom with smartphone processing, 5G creates infrastructure level headroom for data, throughput and lowers latency. AR/VR devices are one of the first new categories to leverage on the additional advantages 5G brings with it.
This helps set the stage for Jio’s network to expand aggressively. With this, Jio effectively looks to own the entire stack from the data that connects your devices, the very devices you use, the applications and services that power them, and now further into commerce and content as well.
This in crease in wireless capacity will be facilitated with Jio’s India-made 5G equipment and Edge capabilities. In the world of XR, one of the most talked about terms is called M2P or Motion to Photon latency. Motion-to-Photon latency is the time needed for a user movement to be fully reflected on a display screen. The smaller the M2P, the more immersive your experience.
Combine this with the larger Jio Platform’s goal of enabling Industrial IoT applications, critical services, SMB data connectivity and wearable computing — you now witness an ambitious 5G infrastructure that can power the next wave of devices in the country in the coming decade.
The curious case of two industries that face the same existential question: “what’s the killer application?”. We do not yet know the killer application for XR or mixed reality. We do not yet know the killer application for 5G. Now, does putting these two industries together for a country that has typically not been a very early adopter of new technologies going to work? We are yet to see. However, if we cease to take a short term view and rather view into the sunset to see how the decade might unfurl, it is compelling to see why these industries might lead to profound impact, if they work out. VR/AR have probably had more false starts as an industry than most other spaces; and anybody who’s been around long enough breeds cautious optimism on if/when it is the right time to consider it akin to the next smartphone revolution. We might very well have to wait for Apple to reveal what thousands of engineers have been working on over years in their Augmented Reality plans. Till then, the Jio Glass serves as a glimpse into a future that promises much.
Conclusion
AR is the next mobile computing platform. It is here today, but still in fancy. We need to make a lot of progress before AR can be optimally immersive, intelligent and connected. The Jio Glass is perhaps a bold attempt in the right direction to help kickstart the consumer adoption of smart glasses in India. That sentence comes in the backdrop of other significantly sized AR hardware companies who’ve taken some ambitious swings at making consumer AR happen before.
I am unsure if at all the Jio Glass will ship to consumers in the near future. Even if it does, there is much for the first time user of Augmented Reality or Mixed Reality to come to terms with. Tempering expectations is an inevitable part of the XR space now, as we wait for hardware and content to catch up with sci-fi. Jio might rather be playing the long term game with Glass. A decade’s view into the future and what makes wearable computing an important part of their plans moving forward.
Would consumer Augmented Reality be different this time around? There is but one way to know, and that is to try. Technology, much like it always has, will iterate itself to the ideal format that consumers want it to be. We lose the inherent magic if we stop taking the swing. Put those goggles back on, the next few years promise to have plenty in store for the space.
Build. Learn. Repeat. 🥽
