HoloLens Tutorial – Cursor and Voice

Last Updated: Apr 30, 2017

Tutorial Accomplishments

  1. Add a cursor to the scene
  2. Add A Voice Command to the scene

 

This section of the tutorial will add user control to our application. We will add a cursor which can be used in conjunction with the input manager we saw in a previous tutorial to allow the user to click on objects. We will also add a voice command to the scene that will allow us to turn the visualization of the spatial map on and off as desired.

 

The Cursor

Open the project in Unity.  Open Assets -> HoloToolkit -> Input -> Prefabs -> Cursor.  Add the prefab Cursor under the Control folder.  Your project hierarchy should now look like this:

Build and Deploy the project. Notice how the cursor aligns to the spatial map as you are mapping the room. This is because each triangle of the spatial map has a collider attached when it is created. You will also notice that the cursor tracks the billboard, this is because of the attached Box Collider. Finish the spatial map, notice how the cursor goes through the holograms. This is because these objects don’t have colliders. Let’s fix these two issues. Edit ObjectCollectionManager and change the contents to this:

This is a very minor change adding a line into CreateTree that adds a collider to each object as it is being instantiated, and a new method that adds a collider to each gameobject and all of its child objects which we use to add colliders to the buildings.

Open Unity 3D.  Select the Spatial Status Billboard GameObject in the Hierarchy and disable the box collider.  The Components should look like this:

Build and Deploy your application.  After you finish deploying notice that the cursor tracks exactly to the real world objects and to the holograms and does not track the billboard.

The cursor is a fundamental component of any HoloLens application and I recommend that you add it at the beginning of your project.  I added it late in this tutorial to demonstrate that although the Cursor is a prefab that just works, you need to be aware of what objects you add colliders to and which ones you do not in order to get the desired behavior for your app.

 

Voice Commands

Open the project in Unity.  Edit the Player Settings and add the microphone permission:

 

Create a new Script called Speech.cs in your scripts folder.  Open it in Visual Studio and add the following contents:

This script provides one method that toggles the drawing of the spaial mapping mesh on and off.

Create a new Empty Gameobject called Speech under the GameObject Control.  Add the HoloToolkit Component Keyword Manager to the Speech GameObject. Add the Speech.cs script to the Speech GameObject.  Configure the properties to look like this:

The Keyword Manager is configured to start automatically, and will response to the keyword “Toggle”.  When Toggle is detected by the keyword manager, it calls the ToggleMesh method of the instance of the Speech script also attached to this gameobject.  The Speech Component gets a reference to the Spatial Understanding Custom Mesh so that it can manipulate the visibility of the Mesh.

Build and Deploy the application. You should be to speak the word “Toggle” at any time to show/hide the current spatial mesh.  Notice that even after you finalize the spatial mesh you can use “Toggle” to turn the mesh back on.

 

 

Tutorial Index

  Target Versions: Unity 5.6.1f1 | HoloToolkit Jun-04-2017Visual Studio 2017 15.2
Unity 3D Project Creation How to create a HoloLens project in Unity 3D
Source Control Configure git for HoloLens / Unity work
Spatial Mapping How to spatial map a Room
Object Surface Observer Set up fake spatial map data for the Unity editor
TagAlongs and Billboarding   Tag along instructions to the user to force text on screen
Spatial Understanding Add spatial understanding to get play space detail
Finalizing Spatial Understanding Complete Spatial Understanding and Input Manager
Object Placement and Scaling Find valid locations for holograms in the play space
Hologram Management Manage the holograms you want to place in the world
Cursor and Voice Add a cursor and voice commands
Occlusion Add occlusion with the real world to your scene
Colliders and Rigidbodys Add Colliders and RigidBodys to your holograms

One thought on “HoloLens Tutorial – Cursor and Voice

  • Posted on May 8, 2017 at 8:55 pm

    Update for this tutorial. You must add the Microphone permission for the speech recognition to work. The article has been updated to correct this mistake.

    Reply

Leave a Reply