Mixed Reality for Wargaming - Very Initial Experiments

 

In October I upgraded from my trusty Oculus/Meta Quest 1 to a Quest 3. Whilst the VR support of the Quest3 is only a marginal improvement (on the already very good) Quest1, the big step forward with the Quest3 is its Mixed Reality (MR) capability. By Mixed Reality we mean the ability to overlay virtual objects and information onto the physical world - but in such a way that the virtual objects are aware of, and respond to, the physical world layout. So for instance if you threw virtual dice onto a physical table they would bounce and roll across the table, and then probably fall off the table edge and onto the floor! 

Mixed Reality

The Quest uses a process called video pass-through. Existing Mixed Reality headsets such as the Microsoft Hololens and Magic Leap deliver Mixed Reality through projecting digital information onto a visor which the users looks through to see the physical world with the virtual overlay. By contrast, with the Quest the user is looking at a stereoscopic video screen which uses camera on the front of the headset to show the physical world, and then then displays the digital information on thew same screen. The Hololens and Magic Leap suffered from two big problems - they had a very narrow field of view and they either needed a relatively dark room or had to darken the physical world view in order to get enough contrast to see the digital information. The Quest3 solves both of these, the FOV is the same as in VR, so about 120 degrees, and it actually prefers more brightly lit rooms.

So what does it look like. This photo at the top of the page is the introductory game in Quest3 where a spaceship falls through a hole in your room's ceiling, lands on the table below it, and then starts spewing little furry aliens which start bouncing around the room and which you need to shoot - and each time you miss and hit your rooms (physical) walls the (virtual) walls begin to disappear and reveal an alien landscape behind!

Mixed Reality Wargaming

But what about wargaming? If you've played Tabletop Simulator you'll know that that already delivers a 3D wargames table - complete with models, terrain, cards and dice - but you only "see" your opponents as icons - there is a limited sense of being in the same room as them. With MR, you should be able to have that same virtual tabletop experience, but you can put the table top on your own (empty) physical table in your gaming room, and your opponents can be a mix of people physically in the same room as you (but wearing their own MR headsets) and people in remote locations coming in via the Internet (and wearing their own MR headsets and seeing the virtual table set up on their physical tables).

The nice things about MR wargaming, just as with Tabletop Simulator, is that you don't have to code any rules, you just need the various 3D assets - models, terrain, dice, cards etc - and then let players manipulate them just as they would in the physical world, and consulting physical rules (or digital copies) as and when needed.

My initial experimentations in this area are all about proving the concept, so I'm not trying to bring in a full game, just proving that the technology can do what is needed. There are likely to be 4 stages:

• Proving it for solo play - i.e. only I am playing - but I get a fully configurable virtual wargames table (ideal for urban wargaming experiments)
• Proving it for remote play - I'm at home and my opponent is in their home
• Proving it for local play - I'm at home and my opponent is in the same room as me
• Proving it for hybrid play - ideally with 3 locations and 4-6 people spread between them (if I can find enough Quest3 owning wargamers!)

First Experiments

Seeing as no-one has yet built (to my knowledge) a wargaming MR app, and I didn't want to have to write the code myself, I first needed to find an app that would let me import, place and move objects in MR. I drew up a long list of potential games/apps that I might be able to co-opt to my needs, but luckily the second one I tried - Arkio - did exactly what I needed. Arkio is designed for architects and  lets them import 3D models and share and explore them in MR, and crucially also has simple in-world build and annotation/measurement tools. So using Arkio and one of their demo cityscapes I brought in a T72 model (bought for $7 on Turbosquid) and moved it around the table, both using the controllers and just with my hands. For good measure I could move the model yacht in the model too.

And here's moving the objects by hands:

A very basic demo, but enough to prove that the concept of solo wargaming in MR is completely viable.

One issue I did have though was that when I first brough the T72 into MR I was expecting it to be something like a 1:35 scale model, whereas it came in as a 1:1 model and completely filled my (relatively large) wargames room!

If you're going to play a game in MR then its good to be able to use the physical rules as it saves you having to work out how to import them. The video quality on the Quest was not only good enough to do that, but also good enough to use your smartphone or tablet - so PDF rules would be OK. 

Of course its useful to have things like QRS sheets in world - so I converted a PDF document into an image and brought that in-world. Since this is MR you can of course put things anywhere, even hanging in space, and resize them as you need them.

A key capability is to be able to improvise within the game/application so you aren't limited to what's been already imported - this is vital to provide a flexible environment for game prototyping. Luckily Arkio has the tools to support this - the simple buildings in the video below were built using the in-world tools.

Another really neat feature of Arkio is that you point your controller at a point in the model and then be teleported into that place at "human" scale so you can view the whole model in VR. Ideal for sorting line-of-sight issues!

And finally, this need not just be for miniatures gaming but can also be used for boardgames - in this case Littoral Commander. If the cards were moved form the physical to the MR space you'd then be able to place them anywhere around you - no more running out of physical table space!

So that's it for the initial look-see. Next task is to try and get the stage 2 test sorted, and maybe bring some more assets in start working up a playable game.