Mixed Reality for Wargaming - Very Initial Experiments

 

In October I upgraded from my trusty Oculus/Meta Quest 1 to a Quest 3. Whilst the VR support of the Quest3 is only a marginal improvement (on the already very good) Quest1, the big step forward with the Quest3 is its Mixed Reality (MR) capability. By Mixed Reality we mean the ability to overlay virtual objects and information onto the physical world - but in such a way that the virtual objects are aware of, and respond to, the physical world layout. So for instance if you threw virtual dice onto a physical table they would bounce and roll across the table, and then probably fall off the table edge and onto the floor! 

Mixed Reality

The Quest uses a process called video pass-through. Existing Mixed Reality headsets such as the Microsoft Hololens and Magic Leap deliver Mixed Reality through projecting digital information onto a visor which the users looks through to see the physical world with the virtual overlay. By contrast, with the Quest the user is looking at a stereoscopic video screen which uses camera on the front of the headset to show the physical world, and then then displays the digital information on thew same screen. The Hololens and Magic Leap suffered from two big problems - they had a very narrow field of view and they either needed a relatively dark room or had to darken the physical world view in order to get enough contrast to see the digital information. The Quest3 solves both of these, the FOV is the same as in VR, so about 120 degrees, and it actually prefers more brightly lit rooms.

So what does it look like. This photo at the top of the page is the introductory game in Quest3 where a spaceship falls through a hole in your room's ceiling, lands on the table below it, and then starts spewing little furry aliens which start bouncing around the room and which you need to shoot - and each time you miss and hit your rooms (physical) walls the (virtual) walls begin to disappear and reveal an alien landscape behind!

Mixed Reality Wargaming

But what about wargaming? If you've played Tabletop Simulator you'll know that that already delivers a 3D wargames table - complete with models, terrain, cards and dice - but you only "see" your opponents as icons - there is a limited sense of being in the same room as them. With MR, you should be able to have that same virtual tabletop experience, but you can put the table top on your own (empty) physical table in your gaming room, and your opponents can be a mix of people physically in the same room as you (but wearing their own MR headsets) and people in remote locations coming in via the Internet (and wearing their own MR headsets and seeing the virtual table set up on their physical tables).

The nice things about MR wargaming, just as with Tabletop Simulator, is that you don't have to code any rules, you just need the various 3D assets - models, terrain, dice, cards etc - and then let players manipulate them just as they would in the physical world, and consulting physical rules (or digital copies) as and when needed.

My initial experimentations in this area are all about proving the concept, so I'm not trying to bring in a full game, just proving that the technology can do what is needed. There are likely to be 4 stages:

• Proving it for solo play - i.e. only I am playing - but I get a fully configurable virtual wargames table (ideal for urban wargaming experiments)
• Proving it for remote play - I'm at home and my opponent is in their home
• Proving it for local play - I'm at home and my opponent is in the same room as me
• Proving it for hybrid play - ideally with 3 locations and 4-6 people spread between them (if I can find enough Quest3 owning wargamers!)

First Experiments

Seeing as no-one has yet built (to my knowledge) a wargaming MR app, and I didn't want to have to write the code myself, I first needed to find an app that would let me import, place and move objects in MR. I drew up a long list of potential games/apps that I might be able to co-opt to my needs, but luckily the second one I tried - Arkio - did exactly what I needed. Arkio is designed for architects and  lets them import 3D models and share and explore them in MR, and crucially also has simple in-world build and annotation/measurement tools. So using Arkio and one of their demo cityscapes I brought in a T72 model (bought for $7 on Turbosquid) and moved it around the table, both using the controllers and just with my hands. For good measure I could move the model yacht in the model too.

And here's moving the objects by hands:

A very basic demo, but enough to prove that the concept of solo wargaming in MR is completely viable.

One issue I did have though was that when I first brough the T72 into MR I was expecting it to be something like a 1:35 scale model, whereas it came in as a 1:1 model and completely filled my (relatively large) wargames room!

If you're going to play a game in MR then its good to be able to use the physical rules as it saves you having to work out how to import them. The video quality on the Quest was not only good enough to do that, but also good enough to use your smartphone or tablet - so PDF rules would be OK. 

Of course its useful to have things like QRS sheets in world - so I converted a PDF document into an image and brought that in-world. Since this is MR you can of course put things anywhere, even hanging in space, and resize them as you need them.

A key capability is to be able to improvise within the game/application so you aren't limited to what's been already imported - this is vital to provide a flexible environment for game prototyping. Luckily Arkio has the tools to support this - the simple buildings in the video below were built using the in-world tools.

Another really neat feature of Arkio is that you point your controller at a point in the model and then be teleported into that place at "human" scale so you can view the whole model in VR. Ideal for sorting line-of-sight issues!

And finally, this need not just be for miniatures gaming but can also be used for boardgames - in this case Littoral Commander. If the cards were moved form the physical to the MR space you'd then be able to place them anywhere around you - no more running out of physical table space!

So that's it for the initial look-see. Next task is to try and get the stage 2 test sorted, and maybe bring some more assets in start working up a playable game.

Empress Marava in Mozilla Hubs - v1

 

About 10 years or more ago I built a full scale model of an Empress Marava class Far Trader in the Second Life virtual world (see image below). Whilst SL is still a great place it doesn't have the traction and mindshare of some of the more recent Social Virtual Worlds, and crucially you can't really use a VR headset with it. 

Empress Marava bridge in Second Life

So I've started on a project to try building Traveller environments in Mozilla Hubs, which seems to offer the best combination of capabilities and accessibilities of the current crop of worlds. Vitally  it used WebXR - so everything runs in your 2D browser and is accessible from an ordinary URL, and in a VR headset you just point the browser at the URL to get in - no software to download and install. In fact I started this project using the custom WebXR framework I developed for my 3D Traveller Map, but then realised that for something that doesn't need scripting Hubs was a better bet as multi-user, avatar design and terrain follow movement all came out the box.

You can access the Empress Marava at: https://hubs.mozilla.com/a3Bqj7N/empressmaravav1

You don't need any account or sign in. Click on Join Room, and you'' be given a random name (which you can change) and a choice of (very blocky) avatars. It wil ask you for microphone permissions as you'll be able to see and chat with anyone else in there (see them gaming possibilities...). Setting up private spaces is an option I'll talk about in another post.

You start on the bridge. If this is your first time in Hubs there's a quick tutorial on movement. It's basically WASD or cursor keys to move, hold left-button and drag to move around.

Now this build is VERY rough. It was a "disposable prototype" just to see what is possible. Even as it is it's still TWICE the recommended size of a Hubs space, so performance may be laggy. Not all exterior walls are in place, so you can walk out into space! There is also no "outside". Just walk through the doors (they don't slide open!), and there are just some random bits of furniture. I've left the deck plan on so you get some idea of where you are. The model is probably about 2x "real" size, but that makes VR movement easier.

This is the cargo deck:

And here's the Owners Quarters:

You should notice a background hum, and this gets louder as you move towards the engine room I've only done the main deck, not the upper deck.

I should note that the build only took a few hours and needed no special 3D skills - the Hubs "Spoke" editor is also free and on-line and accessed through your browser - you only need to create an account. of course the whole build and build process could all be made a lot slicker and better looking of course - I just used their basic architectural blocks and textures. And one nice feature is that you can allow others to "remix" a build, so if I enabled it anyone else could pick up the Marava and customise it to their own needs.

Now on the starboard side of the ship is the big cargo airlock. You can spot it by the image of the planet in the window.

Now if you click on the image Hubs will load up a new space - in this case a Mars like desert world.

Now feel free to wander round (and adventure) on this world. If you want to get back to the ship just head back the rocks you started by and click on the ship bridge image.

So my next plans are:

• Do a v2 build with customised "starship" walls and the second deck
• Build out a suite of different planetary surfaces

I hope you can see all the adventure (and even STEM education) possibilities that this sort of easy access 3D/VR environment opens up. I'd love to hear how you get on and your thoughts about how the Traveller community might be able to use this sort of platform in the future.

Perhaps I'll meet you on the Marava some day!

GDW Traveller Map in VR/3D

 I've finally finished the 1st release of my VR Traveller Map web application. It dynamically pulls data from Traveller Map and you can browse 25 sectors with it.  

YouTube video walkthrough below and at https://www.youtube.com/watch?v=NfnAVowrgug. 

Instructions and link to the app at http://taunoyen.com/wiki/doku.php?id=traveller_3d_map. 

You can switch some data on and off (gas giants, zones, allegiance etc), click on worlds for UWPs, and set height and size to reflect information such as population, importance, tech level and starport. You'll need a WebXR compatible VR headset to use it (e.g. Oculus Quest). The current 2D/3D interface for ordinary users is more limited and lets you walk around Regina subsector but no more - will add fuller functionality next year.

From a technical standpoint the neat thing is that this is in WebXR so you don't need to download any software onto the VR headset, you just access through the built-in browser - but get a full VR experience.

Suggestions for v2 welcome - and of course any bug reports (only tested on Quest and PC).

Virtual Reality Wargaming

Isn't it great when day job and hobby come together. I knew I was at the right place when one of the first slides someone put up (actually Steven Bowns from Wargames Research Group) was of the three editions of WRG's Modern Warfare rules, the same rules I cut my modern wargaming teeth on back in the 70s/80s, and the three of which still sit proudly on my bookshelf!

The event was the Defence Science and Technology Lab's show-and-tell on Defence Wargaming. We (my company Daden) were luck enough to get a demo and talking slot.

For the occasion I updated the Virtual Wargames Room (our 3D Virtual Tabletop - 3D VTT) we built in Trainingscapes a year or so ago. The new version was optimised for the Oculus Quest (which meant the hex board and model soldiers had to go).

SPI BAOR at grand scale

A 6x4 type hex table

Within the space we had:

• Three big floor maps for wargames, BAOR 1984 (an old SP game), Waterloo and the Modern War's Crisis in the Baltic. These have a real sense of scale, the BAOR map in particular looks like it's filling a whole sports hall and you just want to walk all over it!
• Example counters (done as cubes for ease of handling) for each game
• A more conventional "wargames table", in this case with a hex grid and some example troop blocks and terrain features, and even an example simple 3D sculpt tank and figure (think Airfix 1/32nd!)
• A dice for random number generation
• Some static information panels (one on Soviet tactics, the other a ruleset aide-memoire)
• A couple of dynamic information panels/web browsers to bring in rule or period related data or even live Google Docs so you can log (or read) activity and someone on the web can read/update it.

About a "28mm" figure - looks great when you kneel down in VR!

One of the whole aims was to keep it very simple. Nothing is automated, it works just the same way as though you got a physical wargame out on a real table (and we even had people knocking things off the table as in real life, and picking them up to keep the place tidy afterwards).

A visitor moving virtual pieces on a virtual wargames table!

That simplicity really worked and we got some great feedback about the whole concept. We also got some great feedback to our lightening talk, linked both to current projects we have with MOD and some of the peripheral (e.g. the BattlefieldAR system we are working up with Battlefields Trust) and hobby projects I am are working on.

The whole day gave a good insight into some of the companies working in the professional wargaming space (or with aspirations to work there), running the whole gamut from machine-learning AI based systems to good old paper and card wargames and PIP dice for unit activation. There was also a good presentation on the current MOD needs in this area, so hopefully we'll get the chance to progress some of the ideas we presented a bit further.

This is the second "professional wargaming" event I've been two in the last 2 months, and I'm missing a third at the end of the month. It's fascinating how there is a huge resurgence in wargaming in the military community (it was a dirty word when I was in), and I can't wait to see how the recreational and professional sides (continue to) interact.

I'm hoping to actually get the 3D VTT to a state when it can be used for a real game over the next few months. If you've got access to an Oculus Quest or Rift and might like to take part then let me know. We might also do a simple 3D session as well where anyone with a decent PC can play - again let me know if you're interested. I might even take it out to COW, or possibly one of the local shows.

OculusGo

Work colleague using OculusGo

OculusGo came out today. We got our last week to play with and certainly impressed - most usable VR option so far - but miss 6 DOF (Degrees of Freedom - i.e. ability to move laterally). Will see how well it works with my 3D/VR VTT concept - probably not well without 6DOF!

A 3D/VR Virtual Table Top/Wargames Room

As mentioned in the last post my physical space for gaming has taken a dive for a month or so, and that proved to be the kick I needed to look at a 3D/VR virtual table top or virtual wargaming room. It's a project that's been in the offing for ages, ever since my Second Life days. I've used on-line 2D tools to do tabletop map movement (but not graduated yet to Roll20), but I want something that really feels like I'm using a physical tabletop.

Now it just so happens that in the day job we've created something called Fieldscapes - www.fieldscapesvr.com, a 3D/VR authoring tool for learning and training, so I decided to see if I could press this into service as a 3D/VR VTT.

Initially I made a virtual, modular, Hexon table in Unity, along with some suitably sized (10 & 20cm frontage) blocks, brought them into Fieldscapes, laid them out and dropped in. And yes, I could pick up a block with the hand controller and move it on the table!

So I next made/bought/rescaled some simple terrain pieces (woods, hills, bridge, houses), and found a nice simple 3D "quality street" type soldier and cavalry man that I could shrink down to about 20mm and base up. In these came, and not only could I move them but just as in real life I could crouch down by the table top to look at them more closely.

On the next iteration I added some shelves to put example soldiers on (so no need to crouch), added separate tables for Horse & Musket and WW2 (found something to use as ~ 1/32nd tanks and men), and also created two map tables (using Google Maps images), and one table with a 3D sculpted terrain. I also put in a web browser screen so you can bring rules up in world!

In theory all these bits are multi-user, so someone else should be able to come in world and play with them with me - that's the next test! Then after that we see if we can scale up to a reasonable force on the table.

There is no doubt it seems a very real experience (right down to now having a floor littered with discarded soldier - and some just hanging in space!), but there is also no doubt that I wouldn't want to wear the current generation of VR headsets for a 1 hr+ gaming session. But things will evolve.

Finally, here's a current long shot of the room. I'm currently camped out in our Daden Campus building, but may get something a bit grander in due course!