HoloLens Tutorial – Cursor and Voice

Last Updated: Apr 30, 2017

Tutorial Accomplishments

  1. Add a cursor to the scene
  2. Add A Voice Command to the scene

 

This section of the tutorial will add user control to our application. We will add a cursor which can be used in conjunction with the input manager we saw in a previous tutorial to allow the user to click on objects. We will also add a voice command to the scene that will allow us to turn the visualization of the spatial map on and off as desired.

 

The Cursor

Open the project in Unity.  Open Assets -> HoloToolkit -> Input -> Prefabs -> Cursor.  Add the prefab Cursor under the Control folder.  Your project hierarchy should now look like this:

Build and Deploy the project. Notice how the cursor aligns to the spatial map as you are mapping the room. This is because each triangle of the spatial map has a collider attached when it is created. You will also notice that the cursor tracks the billboard, this is because of the attached Box Collider. Finish the spatial map, notice how the cursor goes through the holograms. This is because these objects don’t have colliders. Let’s fix these two issues. Edit ObjectCollectionManager and change the contents to this:

This is a very minor change adding a line into CreateTree that adds a collider to each object as it is being instantiated, and a new method that adds a collider to each gameobject and all of its child objects which we use to add colliders to the buildings.

Open Unity 3D.  Select the Spatial Status Billboard GameObject in the Hierarchy and disable the box collider.  The Components should look like this:

Build and Deploy your application.  After you finish deploying notice that the cursor tracks exactly to the real world objects and to the holograms and does not track the billboard.

The cursor is a fundamental component of any HoloLens application and I recommend that you add it at the beginning of your project.  I added it late in this tutorial to demonstrate that although the Cursor is a prefab that just works, you need to be aware of what objects you add colliders to and which ones you do not in order to get the desired behavior for your app.

 

Voice Commands

Open the project in Unity.  Edit the Player Settings and add the microphone permission:

 

Create a new Script called Speech.cs in your scripts folder.  Open it in Visual Studio and add the following contents:

This script provides one method that toggles the drawing of the spaial mapping mesh on and off.

Create a new Empty Gameobject called Speech under the GameObject Control.  Add the HoloToolkit Component Keyword Manager to the Speech GameObject. Add the Speech.cs script to the Speech GameObject.  Configure the properties to look like this:

The Keyword Manager is configured to start automatically, and will response to the keyword “Toggle”.  When Toggle is detected by the keyword manager, it calls the ToggleMesh method of the instance of the Speech script also attached to this gameobject.  The Speech Component gets a reference to the Spatial Understanding Custom Mesh so that it can manipulate the visibility of the Mesh.

Build and Deploy the application. You should be to speak the word “Toggle” at any time to show/hide the current spatial mesh.  Notice that even after you finalize the spatial mesh you can use “Toggle” to turn the mesh back on.

 

 

Tutorial Index

  Versions: Unity 2017.1.0p5 | MixedRealityToolkit-Unity v1.2017.1.0Visual Studio 2017 15.3.2
Unity 3D Project Creation How to create a HoloLens project in Unity 3D
Source Control Configure git for HoloLens / Unity work
Spatial Mapping How to spatial map a Room
Object Surface Observer Set up fake spatial map data for the Unity editor
TagAlongs and Billboarding   Tag along instructions to the user to force text on screen
Spatial Understanding Add spatial understanding to get play space detail
Finalizing Spatial Understanding Complete Spatial Understanding and Input Manager
Object Placement and Scaling Find valid locations for holograms in the play space
Hologram Management Manage the holograms you want to place in the world
Cursor and Voice Add a cursor and voice commands
Occlusion Add occlusion with the real world to your scene
Colliders and Rigidbodys Add Colliders and RigidBodys to your holograms

4 thoughts on “HoloLens Tutorial – Cursor and Voice

  • Posted on May 8, 2017 at 8:55 pm

    Update for this tutorial. You must add the Microphone permission for the speech recognition to work. The article has been updated to correct this mistake.

    Reply
    • Posted on October 25, 2017 at 10:14 am

      Yes that is correct, it has been obsolete for many months now. I will address that in the next update of the tutorial.

      Reply
  • Posted on October 25, 2017 at 2:39 pm

    One other issue that I have is that the mesh does not toggle. I only get a box with the ‘trying to map your surroundings’ and the contoured hill (the one that comes on when you first start up the HoloLens). It does toggle though. I have only tried this tutorial using the emulator until now. Could that be the reason?

    Reply

Leave a Reply