So Mel, can you explain what is the definition of immersion, which is something often used to describe VR experiences and also in particular, nowadays, we have all these fascinating VR displays that enables us to really see things in 3D with stereo vision? But how does other aspect of VR, which enable us to interact with it, and how does that link back to the concept of immersion? Okay. So I'm going to talk about an ideal system rather than the systems that we have today. So in an ideal system, it would display in all sensory systems. So what I mean by that, of course, we are most familiar with vision, but, of course, there's also sound which is pretty feasible. But even if you think about sound, there's many different ways that sound itself can be portrayed. So it could just be sound not coming from any particular direction. It could be specialized sound, so that when in virtual reality, an event is happening over there, I hear it from over there. Or it could represent how sound is reflecting through an environment a particular room. So it's properly modeled sound. So just as with vision, there's many different levels of vision that you could have from very simple to realistic simulation of how light flows in an environment. So the same is true of sound, but all of these things, more or less, is pretty well today. So there's also haptics. Haptics is two aspects, the sense of touch when you touch something, you feel it and different surfaces have different feelings. And the other aspect of haptics is force feedback. If something touches you or pushes you, you feel the force. So a true ultimate virtual reality system would support both of those. And today, it does to some extent. So, for example, there are haptic devices where you can feel particular kinds of touch feedback, and there are haptic devices, quite complicated ones, where you can feel forced feedback. But, unlike vision, so one visual display device can represent any kind of visual input. For haptics, it's not like that. For each kind of haptics, there's a special device. So you can have a really good device that gives you the feeling of pushing a needle through flesh, for example, if you're using it for training in surgery, or you can have a haptic device where you're touching materials and it feels realistic. But there's no generalized haptic device in the sense of, in real life, I can be walking along and my elbow happens to brush against the wall and I feel it. There's no generalized haptic device here in virtual reality, which makes this possible, and there's certainly no force feedback device. For example, if a virtual character in virtual reality pushes my shoulder, I'm typically never going to feel it because there's no device. There's certainly no general device that does that. So haptics is an area of which requires a lot of development to get towards, let's say, what was Ivan Sutherland's dream of the ultimate display. And another one is smell. So, of course, there again are particular systems that can deliver smell in virtual reality, but there is again no generalized smell system. One of the problems with smell is that once it's in a place, it doesn't go away very easily. So you can make a smell, let's suppose you're in the virtual reality, you're going into a place where there's been a fire, and you smell the fire. So something has to come in real life that makes you smell the fire. But then when you go out of that place, that smell of fire is still going to linger because it's going to be in the real world. So if we go through the various sensory operator's vision and sound, they're pretty well-cared for in today's systems. But, of course, there's lots of room for improvement even in those. Haptics is very, very good particular haptic devices but no generalized haptics. The same is also true of smell. Can I just add something about smell? Because that just reminded me, I visited a lab in Switzerland where, basically, in order to simulate a smell and to solve that problem you've mentioned to get the smell actually out of the way after you change to a different environment is that they put a little pipe into my nose which pumps oxygen constantly. But then when I get close to a particular object in VR, I can just smell that object, but when I move away, obviously, they start pumping oxygen instead. So that's probably one way to do it. So it's really fascinating to think about all this potentials of VR, different VR systems we could develop in the future. And so my question here is, is there a way to actually measure immersion to kind of have a way to compare different VR systems? Is one more immersive than the other? It's very important to understand what we mean by immersion in virtual reality because it's a term overused. So we might say, "Oh, I felt very immersed," or "I felt this was immersive," and so on. But what do we really mean by immersion? To me, immersion is the description of a system. It's a technical description of what a system can deliver. So for example, if we take two head-mounted displays, one that tracks in four degrees of freedom, six degrees of freedom, which means I can turn my head in any direction and I can translate my head like this, and I can turn in any direction, and always, the feedback will be correct. The visual feedback and the sound feedback will be correct for those movements because my head is tracked in all six degrees of freedom. And then compare that with another head-mounted display, which only tracks in terms of rotation. So if I turn my head, it's okay, but if I translate, nothing happens. So these are two different types of immersion, and I would say, the way I think about it is the first one that tracks in six degrees of freedom is more immersive. I mean, immersive in this technical sense because you can use the first one to simulate the second because the second is a subset of the first. The second one, we can only track rotations but not translations. It's a subset of the one where you can track in all six degrees of freedom. And this is actually really important because it gives you qualitatively different experiences and different information. Because if I go like this and nothing happens, well actually, two things will happen. One is I'm likely to get quite sick because I'm moving my head but the world is not updating or my visual display is not updating accordingly. And second, it means, well, I can't look close to an object. I can't look further away an object. I can't look behind an object by moving my head. And this is something very real. For example, in today's head-mounted displays, typically the ones based on phones where you slot your smartphone into a casing, they only have the rotational kind of head-tracking because they're based on the inertial system of the phone. Whereas a head-mounted display, which has a head-mounted display in itself, not just a phone, typically, they have the full six degrees of freedom tracking. So if we generalize this idea, I would say one system is more immersive than another if the first can be used to simulate the second. Should I give another example? Actually, because we spent lots of time discussing in this course the difference between CAVE and HMD, so which one of these two systems you think is more immersive? Remember, I'm talking about a technical definition. I'm talking about specification of the system, the hardware and software. At this, moment I'm not talking about the effect of those on you though I gave that as an example that they do have different effects. But I'm only talking about the technical specification. So a head-mounted display with full six degrees of freedom tracking is more immersive than the CAVE because I could use a head-mounted display to simulate the whole process of going in a CAVE and being in a CAVE. But I can't use a CAVE to simulate the process of picking up a head-mounted display and putting it on. That's just not possible. Or if we take another example, for many years, people talked about desktop virtual reality. So desktop virtual reality was that you'd sit in front of a monitor and maybe with stereo vision, you wear glasses and so on and you see stereo on the screen. Some people call this virtual reality or desktop virtual reality. Virtual reality through a full head-tracked head-mounted display is more immersive than desktop virtual reality because I can use the full head-mounted display system to simulate a desktop system. So this is what I mean that there's various levels of immersion that when you have system A and System B, then you can use A to simulate B, then A is more immersive than B. The last point on this, this doesn't apply to all systems, so I could have two systems X and Y, where X can't be used to simulate Y, and Y can't be used to simulate X. This immersion is not like a full ordering of all possible systems, is what's called in mathematics, a partial order. So one example you can think of is probably when you are in the CAVE comparing to when you're in a sort of IMAX cinema, where they come very sort of simulate each other. Yeah, they're different. Yeah. Because in the cave, for instance, when you turn back, normally, you don't have the back wall. So it doesn't actually 100% simulate experience you can have in IMAX and definitely not the other way. Yeah. You couldn't use either of those to simulate the other. Yeah. And I'm talking about simulation in principle. I mean as an ideal, not like an actual hardware and so on.
