At work I’ve been doing some research into private cloud options. We hope to launch a private cloud (focused on development, not production) to host our build servers, test systems, tools like TFS and Git, and anything else we think of. Digging into Microsoft’s Private Cloud offering turned out to be challenging. There are many articles and videos explaining how to use it once it is up and running, but I could not find a single guide on how to create one from scratch, so I thought I would document what I have done to save someone else the trouble.
The basic components needed for a Private Cloud is:
- An Active Directory Domain Server with working DNS
- At least one Hyper V Server (more is better) with Multipath I/O installed on each
- A SQL Server installation
- A System Center Virtual Machine Manager Installation (SCVMM)
- An SCVMM library Server
- A System Center Application Manager Installation
This can all be easily accomplished on 1 machine (which is what I have done) or could be spread across multiple servers which is what you want in a production environment. This installation walk-through assumes that you want to do it all on one box. If you do I recommend that you have some pretty decent specs, 4+ cores, 16+ gigs of ram, with multiple network connections and at least RAID 5 class storage at a minimum. Anything less will make it difficult to experiment with due to performance problems.
Step 1 – Install Windows 2012 R2
Start by performing a normal installation of data center edition if available or standard in a pinch. Licensing gets a bit hairy if you are using the standard edition, check out this Microsoft Licensing White Paper as a starting point to get a bit more information. Once you have a working installation with all the proper drivers and fully updated, you can start building your cloud.
Step 2 – Configure Your Networking
For the server the ideal configuration is to have one network connection dedicated to the bare metal for administration tasks and the rest to servicing the Virtual machines. Open your Network Connection Adapter Settings, and rename all of them based on their functionality, this will make life easier later. For my server I renamed the bare metal network connection as “Internet and LAN” and the rest of my network adapters to “Hyper V1”, “Hyper V2”, etc.
Next create a Network Team containing all of your “Hyper V?” network adapters. I named this “Hyper V Team”. In Windows 2012 it isn’t really obvious where to create a network team. Open Server Manager, Select Local Server, and look for NIC Teaming, you can open the dialog from here. Make sure that the team you create is configured as follows:
- Teaming Mode: Switch Independent
- Load Balancing Mode: Hyper-V Port
This assumes that you do not have network switches capable of teaming; if you do, these are most likely not the right settings, but they should still work.
Step 3 – Configure an AD Domain and DNS
Installing an Active Directory domain and DNS is well documented. There are many step by step guides available like this Guide. Two important tips many of them miss:
- Before starting, configure your “Internet and LAN” network adapter with a static IP and make sure the DNS servers are also static and correct.
- Select a domain name that is not already in use anywhere this server will connect to. I recommend the best practice of adding “.local” to the end which should keep it from conflicting with any internet addresses.
Once AD and DNS are functional continue to the next step.
Step 4 – Install the Hyper V Role
Open up Server Manager and “Add Roles and Features”. Add the Hyper-V role to the server.
Once Hyper V has installed, open up Hyper V Manager and configure a virtual switch that will give network access to your virtual machines. This should be an “External” virtual switch, and should be configured for the connection type of “External network” and point at the “Microsoft Network Adapter Multiplexor Driver” if you created a team earlier, or if you did not, it should point to the network adapter used to connect to the LAN and Internet. If this network adapter/team is dedicated to only serving the virtual machines (which is highly recommended), uncheck the “allow management operating system to share this network adapter”.
Step 5 – Test your Hyper V Server
This is a good chance for a sanity check to make sure everything is working together. Create a Virtual Machine based on a Windows operating system. Once the machine is installed, install the integration services, then add the computer to the domain. After the reboot finishes verify that you can ping the server by its short windows name and its full name (machinename.domainname). If these are all working you have all of the important foundational pieces installed.
Step 6 – Install SQL Server
I installed SQL Server 2012 R2 standard to support the databases needed by the System Center tools. Install a default instance and configure the services to use a domain account for the SQL services, this makes life easier later. This should be installed on the bare metal server, not in a VM. SQL Server does work fine in a VM, but this just incurs additional overhead on something you want to be as fast as possible, especially when you use sql server for other things later. I suspect you could also get away with SQL Express here but did not try.
Step 7 – Install SCVMM
Create a Virtual machine running Windows Server 2012 and join it to the domain. Install SCVMM 2012 R2 into the virtual machine, it will ask some prerequisites along the way, but it gives you links to download them. Once SCVMM is installed click “Add Hyper-V Hosts and Clusters” and select:
- Windows Server computers in a trusted Active Direct domain
- Manually enter the credentials and enter credentials that have admin access on your hyper-v server
- Specify Windows Server computer by names and type the name of your hyper-v server
- Select your server name from the list and add it.
At this point you should be able to see the VM you created earlier and it’s status.
Step 8 – Add the SCVMM Library
A library in SCVMM, is used to hold ISO images, virtual machine templates, virtual machine drives, and service templates. This can exist pretty much anywhere on your network. Sticking with our concept of the single machine install, I recommend creating a share on the server itself. A standard practice for this share is to call it “MSSCVMMLibrary”. Make sure the domain account you want the SCVMM virtual machine to access it with has full control in security settings and share settings. After this share is created you can populate it with all the items you want to have in your library, such as ISO’s and virtual machine appliances, or even any VHD files you might already have. I recommend creating an organized file structure. Next in SCVMM:
- Right click Library Servers and click “Add Library Server”
- Enter the credentials used to access the server and click next
- Enter the computer name of the computer you are using as the library server and click search
- Select the computer from the selected servers list and click next
- Check the checkbox next to the MSSCVMLibrary share you have created and click next
- Click Add Library Servers
Open up the jobs and you will see a job called “Add Library Server” and make sure this completes without error. Once it completes you can select library and you should now see the server listed. Anything you had in the share will be listed here. Be careful when you add items to the share, by default the library is updated every hour, so it can take an hour for something to appear in the library. The Library can be updated manually if needed in the SCVMM application.
Step 9 – Install System Center Application Controller 2012 R2
On the same server you have installed SCVVM, perform the install process for System Center Controller Manager. This is a straightforward installation, if you have any problems check out the Technet guide. The tricky part is the actual configuration. Make sure that if you have a firewall installed on this machine that you have port 443 open so that you can access the web site created by SCAM from the rest of your network. Open up the app controller web site (the installer has created a shortcut in your start menu), and log in with your administrator account.
Step 10 – Connect SCAM to SCVMM (and Azure)
Open the settings section and select Connect SCVMM. For connection name you can use whatever you want, server name should be the full computer name of the virtual machine you have install SCVMM on, and port can be left at its default value of 8100.
Step 10.5 – Optionally connect to Azure if you are building a hybrid cloud
Open the Subscription page under settings, click add, and enter the following:
- Name: enter any name that will help you remember which subscription you are connecting to.
- Subscription ID: enter the subscription id found in the Azure portal under Subscriptions, Manage Subscriptions.
- Management Certificate: Upload the private key of your management certificate
- Management Certificate Password: Enter the password used to unlock your private key
If you do not have a management certificate associated with your Azure subscription, create one at the Visual Studio command line with the command:
makecert -sky exchange -r -n "CN=" -pe -a sha1 -len 2048 -ss My ".cer"
This will result in a public key getting generated in the .cer file created and a private key getting add to your local machine’s certificate manager. Run “certmgr” on the machine used to generate the certificate, find the private key and export it, during this process you will associate a password with the certificate and generate a PFX file which is your private key. These are the items that get inserted into SCAC.
If you did generate a new certificate you also need to upload the public key to your azure account. In the portal, select Subscriptions, Manage Subscriptions, Select Management Certificates, and click upload. This must be done before you can connect SCAC to Azure.
At this point you should have a fully functional private cloud!
In my next post I will cover how to deploy a service into your private cloud as well as VM’s.