HoloLens Tutorial – Occlusion

Last Updated: Jun 4, 2017

Tutorial Accomplishments

  1. Add occlusion to the scene
  2. Fix Voice command to swap between grid and occlusion

 

Every time I show this demo to people I highlight the problems that are caused by not having the real world occlude your holograms.  The picture above shows an example of this problem, can you tell which sphere is covered by the wall and which isn’t? Although it appears that the one on the right should be occluded, they both should be. This tutorial shows how to fix this.

 

Occlusion for Spatial Understanding

This became very easy to accomplish due to PR#663 on the HoloToolkit done by Rafał Legiędź. Take a look at the code changes here:

https://github.com/Microsoft/HoloToolkit-Unity/pull/663/commits/22d1d94b54d64e19fcfbf3e157990eefb04b8f99

These changes allow the Material to be updated while spatial understanding is running.  This allows us to switch the material to the any other material and have the surface update in real time. Open ObjectPlacer.cs in Visual studio and change this method to have the following code:


public Material OccludedMaterial;
private void HideGridEnableOcclulsion()
{
SpatialUnderstandingMesh.MeshMaterial = OccludedMaterial;
}

view raw

ObjectPlacer.cs

hosted with ❤ by GitHub

Open your scene in Unity and Select Placement in the Hierarchy.  Set Occluded Material in Object Placer to the Material “Occlusion”.  The properties should look like this:

That’s it! Build and Deploy the project.  Notice that after spatial understanding is complete and the world is generated there is now occlusion with the  real world.  Stand behind a real world object that blocks your view of a hologram.  Try having a hologram partially blocked. This seems like it totally solves the problem, but be warned, this level of occlusion highlights every flaw in your spatial mesh, so use this technique carefully.

 

 

Updating our Voice Command

In a previous tutorial we created a  voice command that looked like this:

 

Due to the addition of occlusion this no longer works, lets correct the “Toggle” command to only work after we have finalized the spatial mesh and to swap between occlusion and the grid visualizing the spatial mesh.  Edit the Speech.cs file to look like this:

 


using HoloToolkit.Unity;
using UnityEngine;
public class Speech : MonoBehaviour
{
public SpatialUnderstandingCustomMesh SpatialUnderstandingMesh;
private Material _swapMaterial;
public void Start()
{
_swapMaterial = SpatialUnderstandingMesh.MeshMaterial;
}
public void ToggleMesh()
{
if (SpatialUnderstanding.Instance.ScanState != SpatialUnderstanding.ScanStates.Done) return;
var otherMaterial = SpatialUnderstandingMesh.MeshMaterial;
SpatialUnderstandingMesh.MeshMaterial = _swapMaterial;
_swapMaterial = otherMaterial;
}
}

view raw

Speech.cs

hosted with ❤ by GitHub

On Start we store a copy of the Spatial Understanding Mesh Material which is the standard Spatial Understanding Grid.  When the voice command for toggle is received we first make sure that spatial understanding is completed before we do anything.  If it is complete we swap the currently displayed material for the material we have stored.  This allows us to toggle between the occluded material and the grid.

Build and Deploy the application. You should be to speak the word “Toggle” after you have finalized the spatial mesh to switch between the grid and occlusion.

 

 

Tutorial Index

  Versions: Unity 2017.1.0p5 | MixedRealityToolkit-Unity v1.2017.1.0Visual Studio 2017 15.3.2
Unity 3D Project Creation How to create a HoloLens project in Unity 3D
Source Control Configure git for HoloLens / Unity work
Spatial Mapping How to spatial map a Room
Object Surface Observer Set up fake spatial map data for the Unity editor
TagAlongs and Billboarding   Tag along instructions to the user to force text on screen
Spatial Understanding Add spatial understanding to get play space detail
Finalizing Spatial Understanding Complete Spatial Understanding and Input Manager
Object Placement and Scaling Find valid locations for holograms in the play space
Hologram Management Manage the holograms you want to place in the world
Cursor and Voice Add a cursor and voice commands
Occlusion Add occlusion with the real world to your scene
Colliders and Rigidbodys Add Colliders and RigidBodys to your holograms

2 thoughts on “HoloLens Tutorial – Occlusion

  • Posted on September 6, 2017 at 6:00 am

    You missed to also specify a variable declaration, in the first code snippet of ObjectPlacer.cs:

    public Material OccludedMaterial;

    Great tutorials, by the way, thank you very much! 😉

    Reply
    • Posted on September 10, 2017 at 3:55 pm

      Good catch! I corrected the code snippet.

      Reply

Leave a Reply