Last Updated: Jun 4, 2017
Tutorial Accomplishments
- Add occlusion to the scene
- Fix Voice command to swap between grid and occlusion
Every time I show this demo to people I highlight the problems that are caused by not having the real world occlude your holograms. The picture above shows an example of this problem, can you tell which sphere is covered by the wall and which isn’t? Although it appears that the one on the right should be occluded, they both should be. This tutorial shows how to fix this.
Occlusion for Spatial Understanding
This became very easy to accomplish due to PR#663 on the HoloToolkit done by Rafał Legiędź. Take a look at the code changes here:
These changes allow the Material to be updated while spatial understanding is running. This allows us to switch the material to the any other material and have the surface update in real time. Open ObjectPlacer.cs in Visual studio and change this method to have the following code:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
public Material OccludedMaterial; | |
private void HideGridEnableOcclulsion() | |
{ | |
SpatialUnderstandingMesh.MeshMaterial = OccludedMaterial; | |
} |
Open your scene in Unity and Select Placement in the Hierarchy. Set Occluded Material in Object Placer to the Material “Occlusion”. The properties should look like this:
That’s it! Build and Deploy the project. Notice that after spatial understanding is complete and the world is generated there is now occlusion with the real world. Stand behind a real world object that blocks your view of a hologram. Try having a hologram partially blocked. This seems like it totally solves the problem, but be warned, this level of occlusion highlights every flaw in your spatial mesh, so use this technique carefully.
Updating our Voice Command
In a previous tutorial we created a voice command that looked like this:
Due to the addition of occlusion this no longer works, lets correct the “Toggle” command to only work after we have finalized the spatial mesh and to swap between occlusion and the grid visualizing the spatial mesh. Edit the Speech.cs file to look like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
using HoloToolkit.Unity; | |
using UnityEngine; | |
public class Speech : MonoBehaviour | |
{ | |
public SpatialUnderstandingCustomMesh SpatialUnderstandingMesh; | |
private Material _swapMaterial; | |
public void Start() | |
{ | |
_swapMaterial = SpatialUnderstandingMesh.MeshMaterial; | |
} | |
public void ToggleMesh() | |
{ | |
if (SpatialUnderstanding.Instance.ScanState != SpatialUnderstanding.ScanStates.Done) return; | |
var otherMaterial = SpatialUnderstandingMesh.MeshMaterial; | |
SpatialUnderstandingMesh.MeshMaterial = _swapMaterial; | |
_swapMaterial = otherMaterial; | |
} | |
} |
On Start we store a copy of the Spatial Understanding Mesh Material which is the standard Spatial Understanding Grid. When the voice command for toggle is received we first make sure that spatial understanding is completed before we do anything. If it is complete we swap the currently displayed material for the material we have stored. This allows us to toggle between the occluded material and the grid.
Build and Deploy the application. You should be to speak the word “Toggle” after you have finalized the spatial mesh to switch between the grid and occlusion.
Tutorial Index
Versions: Unity 2017.1.0p5 | MixedRealityToolkit-Unity v1.2017.1.0 | Visual Studio 2017 15.3.2Unity 3D Project Creation | How to create a HoloLens project in Unity 3D |
Source Control | Configure git for HoloLens / Unity work |
Spatial Mapping | How to spatial map a Room |
Object Surface Observer | Set up fake spatial map data for the Unity editor |
TagAlongs and Billboarding | Tag along instructions to the user to force text on screen |
Spatial Understanding | Add spatial understanding to get play space detail |
Finalizing Spatial Understanding | Complete Spatial Understanding and Input Manager |
Object Placement and Scaling | Find valid locations for holograms in the play space |
Hologram Management | Manage the holograms you want to place in the world |
Cursor and Voice | Add a cursor and voice commands |
Occlusion | Add occlusion with the real world to your scene |
Colliders and Rigidbodys | Add Colliders and RigidBodys to your holograms |
- Download the completed app Western Town in the Windows Store!
- Completed Source code from the entire tutorial available on GitHub.
You missed to also specify a variable declaration, in the first code snippet of ObjectPlacer.cs:
public Material OccludedMaterial;
Great tutorials, by the way, thank you very much! 😉
Good catch! I corrected the code snippet.