HoloLens Tutorial – Object Placement and Scaling

Last Updated: Apr 23, 2017

Tutorial Accomplishments

  1. Use Spatial Understanding to find the location for a hologram
  2. Instantiate the hologram in the world
  3. Scale the hologram to the size of the available space

This section of the tutorial will focus on object placement and scaling of Holograms. For this section I am using a pack of 3D objects from the Unity Store, but you should be able to use any objects that have been optimized for Unity and aren’t too complex for the 3D processing power of the HoloLens.

 

Add 3D Objects to the Project

Start by adding a 3D Objects pack form the Unity Asset Store.  I’m using the Pro Wild West Pack, there is also a free version called the Pro Western Starter Pack that should get the job done as well.

After purchasing, use the download manager in unity to download and import the pack.  Unity will display an import screen that looks like this:

With everything selected click the Import button. A new folder is created in your Assets folder called “Pro Wild West Pack”. Browse through the folders and you will find all the parts that make up the 3D objects. For this project we will use the contents of the Prefabs folder.

 

 

Disable the Spatial map and the Billboard

Before we display a hologram, we need to disable the spatial map visualization and the billboard containing text. Although this is only tangentially related to placement we will do this as the beginning of the script for object placement. Create an empty GameObject under Holograms and name it “Placement”.  In the project Pane, select Scripts.  Create a new script called “ObjectPlacer.cs”.   Add the component Object Placer to the Placement node in the Hierarchy.  Your project should now look like this:

Add the following contents to ObjectPlacer.cs:

Also modify the method Update in SpatialUnderstandingState.cs to contain the following code:

In unity Select Spatial Status Billboard in the Hierarchy. Drag the Placement GameObject into the Spatial Understanding State Component into the Placer property.

These changes allow the application to call the CreateScene method of the ObjectPlacer.cs script on the specific instance that is attached to the Placement GameObject.  Next select the Placement GameObject in the Hierarchy.  Drag the Spatial Understanding GameObject onto the Object Placer component in the Spatial Understanding property.  This will allow the Object Placer script to control the state of the Spatial Understanding by giving it a reference to the instance of the script.

Build your project and run it on either the emulator or the HoloLens.  Now when you finalize the spatial understanding the text is disabled by setting SpatialUnderstandingState.Instance.HideText to true in ObjectPlacer.cs and the material showing the spatial understanding map is disabled by setting SpatialUnderstandingMesh.DrawProcessedMesh to false.  This sets us up with a clean environment to display Holograms in as well as giving us an easy place to instantiate holograms that occurs immediately after spatial understanding is finalized.

 

Place the Hologram

Next we will use Spatial Understanding to find a location to place city_Saloon.prefab.  This prefab is found here and looks like this:

 

Before we place the actual hologram lets draw the outline of a box where spatial understanding tells us to place our hologram.   This is useful to have as an option for troubleshooting and debugging and will help demonstrate exactly what is happening. We will again borrow code from the Spatial Understanding example included with the HoloToolkit, with some small tweaks to decouple it from that example.  I’m not going to go into detail explaining this code since I did not write it, but the basic idea is that every frame it draws lines representing the boxes that are passed to it.  Create a new script caller “BoxDrawer.cs” and add these contents to it:

In your Scripts folder also create a new script called “PlacementQuery.cs”.  Add the following code to it:

This script creates a data object that is used to keep track of a spatial understanding request. Why not just make the request right away and skip storing the request in a data structure? Right now that would work fine, but eventually we want to create many holograms, and the most efficient way to do this is to feed all the requests to spatial understanding at once in a separate thread so that the UI thread doesn’t lock up for the user which would make the HoloLens appear to lock up.  For the same reason, we also will need to create the script “PlacementResult.cs”:

Next make these changes to ObjectPlacer.cs:

We have created some public properties on ObjectPlacer.cs, these properties keep track of the desired size of the hologram we are creating and a reference to the hologram.  These are done in public properties so that changes to the hologram or its size can be made in Unity with having to change the code.  In the next tutorial we will expand this concept further to allow us to create any number or holograms at different sizes.  Start and Update are both updated to support boxdrawer.cs to draw boxes if the public debug boxes property is checked.  We again make this a public property so that we can change it in unity without touching any code.

The CreateScene method first checks to make sure that spatial understanding is ready, if it is not it exits immediately.  (In a real application you would want to do something to attempt to recover at this point)  Next the Spatial Understanding solver is initialized and we update the billboard to let the user know we are generating the world.  The next call creates a list of work for the solver to do containing 1 item that is trying to solve for the size of the building and then we make a call to start doing the work in GetLocationsFromSolver.

GetLocationFromSolver starts a separate thread and attempts to complete each work item in the list that was given to it by making a call to PlaceObject.  The Result of that call is added to the results queue for drawing.  PlaceObject makes the actual call to Spatial Understanding taking all of the information about the work to be completed as parameters.

The Update method also makes a call to ProcessPlacementResults on every frame.  This method checks for any work in the work queue, and if there is any it takes one item off of the queue and draws a box for it.  This method only does one item of work in the queue per frame to make sure the frame rate is not impacted.

In Unity select the Placement GameObject in the Hierarchy.  Find the City_Saloon.prefab in the Assets folder in your project pane.  Drag the prefab to the Saloon Building Prefab property in the Object Placer Component.  Your project should now look like this:

Why did we need to reference the Saloon prefab if we are only drawing a box?  The reason is because we are using the size and shape of the prefab that we eventually want to draw for the space request that we make to spatial understanding.  We will also use this later when we instantiate the hologram into the world.  Build the app and run the app in the emulator or on the device.

 

Instantiate and scale the Hologram in the world

Next make the following changes to ObjectPlacer.cs:

First we change the default for debug boxes to false.  We still want the ability to turn them on and off through a check box but normally we will want them to be disabled.  The method ProcessPlacementResults has been changed to calculate the correct rotation for the hologram and create the building with a call to CreateWideBuilding.

CreateWideBuilding starts by calculating the correct position for placement of the hologram.  The position that spatial understanding gives us is the exact center of the area in three dimensional space, but the position that unity wants is the bottom center of the area.  In other words we want the center of the desired location for the hologram in the X and Z planes, but the start of the hologram in the Y plane.  Next we call instantiate passing the hologram, position and rotation desired.  If the hologram is instantiated correctly, we parent the hologram to the Placement GameObject in the hierarchy and set the local scale to the desired size for the hologram.

The scaling relies on a series of utility methods that I have created called GetBoundsForAllChildren, CalcScaleFactorHelper, and RescaleToDesiredSizeProportional. GetBoundsForAllChildren examines the hologram and all its children and creates a bounding box for that hologram and its children. The bounding box is then used in CalcScaleFactorHelper to find the largest scaling factor that will allow the hologram to fit in the space requested from Spatial Understanding (desiredSize). RescaleToDesiredSizeProportional uses the results of CalcScaleFactorHelper to create  a vector3 that is the local scale transform that should be applied to the hologram to get the desired size.

Build the app and run the app in the emulator or on the device.  The application now displays the saloon scaled to the correct size based on the results from Spatial Understanding.

 

 

Tutorial Index

  Versions: Unity 2017.1.0p5 | MixedRealityToolkit-Unity v1.2017.1.0Visual Studio 2017 15.3.2
Unity 3D Project Creation How to create a HoloLens project in Unity 3D
Source Control Configure git for HoloLens / Unity work
Spatial Mapping How to spatial map a Room
Object Surface Observer Set up fake spatial map data for the Unity editor
TagAlongs and Billboarding   Tag along instructions to the user to force text on screen
Spatial Understanding Add spatial understanding to get play space detail
Finalizing Spatial Understanding Complete Spatial Understanding and Input Manager
Object Placement and Scaling Find valid locations for holograms in the play space
Hologram Management Manage the holograms you want to place in the world
Cursor and Voice Add a cursor and voice commands
Occlusion Add occlusion with the real world to your scene
Colliders and Rigidbodys Add Colliders and RigidBodys to your holograms

21 thoughts on “HoloLens Tutorial – Object Placement and Scaling

  • Posted on February 12, 2017 at 12:55 pm

    Hi, Cameron. How would we adapt this code to make Shape and Topology queries instead? Do you plan to cover that on another tutorial?

    Reply
  • Posted on March 25, 2017 at 2:23 pm

    Hi guys! I am getting the error Placer does not exist in the current context. Is their meant to be a Placer function in Spatial Understanding State? Love the tutorials btw such a great help

    Reply
    • Posted on March 27, 2017 at 7:27 pm

      I did not run into that issue, if you want to share some code that has this issue I’d be happy to take a look.

      Reply
  • Posted on March 28, 2017 at 9:20 am

    Hi Cameron! Thanks for you reply. I manged to sort the issue was a silly mistake in my code. I am currently working on my dissertation at Heriot Watt University where i am utilizing the HoloLens in order to create a companion for children. Would be awesome to sit down and pick your brains about spatial awareness. I have a working companion just need to let him use his environment to make him more life like.

    Reply
    • Posted on April 4, 2017 at 8:52 pm

      That sounds really interesting Jay, I’ll shoot you an email.

      Reply
  • Posted on March 29, 2017 at 9:24 am

    Hi, I have managed to place my box in my room. The size of my debug box is 1*2*0.5 and my hologram is well inside, but it is not very large. I expected it to fill the debug box. How can I make my hologram fill the entire box ?
    Thanks for your tutorials by the way.

    Reply
    • Posted on April 4, 2017 at 9:27 pm

      The method RescaleToDesiredSizeProportional finds the largest possible scaling factor that fits within the bounding box of the space found by the solver. You could easily scale it to fill the box, but then your model would be stretched out of shape and most likely this isn’t desirable. The easiest path would be to request a space from the solver that has the same proportions as the model that you are rendering as a hologram. For example if your model’s actual size is 100x200x300, you could request a space that is 1x2x3 and the model would exactly fill the box. (or 2x4x6 or 1.5x3x4.5, etc) I hope that helps!

      Reply
      • Posted on April 5, 2017 at 7:01 am

        Thanks. That helps a lot. When do you think the other parts will be available ?

        Reply
  • Posted on May 8, 2017 at 7:09 pm

    Great Series, thank you very much. Does SpatialUnderstanding restrict the Holograms after placement? I added a the HandDraggable class to the prefab and it seems like the building is about to move but doesn’t. That’s understandable if so, but how is done or can it be undone with worldanchor, etc.etc.

    Reply
    • Posted on May 8, 2017 at 8:52 pm

      No, the spatial mapping does not place any restrictions on the holograms. It tells you where to place holograms and tracks where they have been placed, but you can choose to ignore what spatial understanding tells you. Handdraggable uses hit tests on colliders to find valid locations to move the hologram. Those colliders are created by spatial mapping and not spatial understanding, so the surfaces will not line up with the visualization of the spatial understanding. If they aren’t moving at all there is probably some other issue.

      Reply
      • Posted on May 24, 2017 at 2:54 pm

        You were right, the problem was on my end. So, I’m trying to test the other SpatialUnderstandingDllObjectPlacement methods and trying to place the objects on the wall.
        So I modified the placementDefinition in the CreateLocationQueriesForSolver method of the ObjectPlacer from

        SpatialUnderstandingDllObjectPlacement.ObjectPlacementDefinition.Create_OnFloor(halfBoxDims);

        to this:

        SpatialUnderstandingDllObjectPlacement.ObjectPlacementDefinition.Create_OnWall(halfBoxDims, 0.0f, 50.0f, SpatialUnderstandingDllObjectPlacement.ObjectPlacementDefinition.WallTypeFlags.Normal, 0.0f, 0.0f);

        But my custom objects (basic cubes) do not appear at all. When I switch it back to Create_OnFloor the objects do appear. What am I missing? It it the min and max height of the wall? Do I estimate those values? Or am I missing something somewhere else?

        Reply
    • Posted on May 20, 2017 at 12:37 pm

      I haven’t gone through and recreated this demo yet, but do the holograms have a WorldAnchor component? If they do, it will prevent movement. In that case, you’ll need to remove the WorldAnchor component, then move the hologram to the desired location and re-add a WorldAnchor component with the new location.

      Reply
  • Posted on May 23, 2017 at 8:59 am

    Hi,
    I just started this tutorial and in the section “Disable the Spatial map and the Billboard” , there are three things I noticed in a Hierarchy Panel ,
    1- Basic,
    2- Spatial
    3- Control.
    Can somebody guide me from where it come from? Is this article the continuty of another previous articles?
    Or is it just prefabs of HoloToolkit?

    Reply
    • Posted on May 23, 2017 at 9:46 am

      This is a tutorial series that builds on each tutorial, this is the 8th tutorial. At the end of the article there is an index that shows them in order.

      Reply
  • Posted on May 26, 2017 at 2:31 am

    Hi, thank you for your great tutorial, now I understand more about spatial understanding.
    I have a question is that after I place the holograms on the ground mesh, and if I move the camera, the holograms move with the camera, instead of staying where they are placed.
    Is there some setting I have missed? thank you.

    Reply
    • Posted on May 26, 2017 at 3:38 am

      it was my mistake, I attached SimpleTagAlong on the Hologram object instead of only on billboard.
      After moving SimpleTagAlong to billboard object the holograms stay where they are. thanks.

      Reply
  • Posted on July 21, 2017 at 8:39 am

    I’ve been trying to place some objects around a room, but for now this code only works with placing objects on the floor, and doesn’t work with the wall. I don’t really know why since from what I understand, the query should work with different surfaces. Do you know how can do it with walls? Thanks a lot!

    Reply
    • Posted on July 30, 2017 at 10:39 am

      In ObjectPlacer.CS in the method CreateLocationQueriesForSolver there is a call to SpatialUnderstandingDllObjectPlacement.ObjectPlacementDefinition.Create_OnFloor. If you change this to Create_OnWall you should be able to create the objects on the wall. Most likely you would want to extend some of the patterns to make this configurable per object instead of being hard coded. As with most demos/examples like this I have oversimplified these classes for the sake of making the points, but you will likely need to extend and enhance these patterns to handle the needs of your application.

      Reply
  • Posted on August 1, 2017 at 5:50 am

    Hi, i am currently following the tutorials, and using the hololens emulator.
    Is there a way to make the mesh stay after I scan the room so I can place the holograms?

    Reply
    • Posted on August 1, 2017 at 9:05 am

      Sure, you can comment out the line of code that turns the mesh off, it is in objectplacer.cs in method update, there is a call to HideGridEnableOcclulsion, that would be the one to comment out.

      Reply

Leave a Reply