Tag Archives: VMware

Deploying vCenter Server Appliance 5.1 with AD auth

3 Replies

In theory using the vCenter Server Appliance (hereafter vCSA) offers a number of big advantages over using vCenter. Firstly you don’t need to commit a Windows Server OS licence, secondly to manage your VMs you can use a Flash-enabled browser on any operating system (including on Surface RT!), and thirdly it should be a lot quicker to deploy.

On that last point, a number of configuration steps in the setup of the vCSA are counter-intuitive and can waste a substantial amount of time. This is because the vCSA defaults to a hostname of localhost.yourdomain.com. The various web services that the appliance runs (SSO, Lookup Service, Inventory Service, vSphere Web Client) interact with each other using SSL sessions and, although there’s a built-in method to make vCSA regenerate its self-signed certificates at boot time, at the time of writing this does not work once these web services have been configured.

If you do not edit the hostname, you will not be able to enable AD authentication, and will likely encounter “Failed to connect to VMware Lookup Service https://yourhostname:7444/lookupservice/sdk – SSL certificate verification failed” when attempting to use the vSphere Web Client. However if after completing the initial setup wizard you configure the hostname to something meaningful and try to regenerate the SSL certificates, the vCSA will hang at boot time, displaying:

Waiting for network to come up (attempt 1 of 10)...
Appliance Name - VMware_vCenter_Server_Appliance
Configuration for eth0 found
The network for interface eth0 is managed internally, network properties ignored
DNS : x.x.x.x
Hostname or IP has changed. Regenerating the self-signed certificates.
Starting VMware vPostgres: ok
Waiting for the embedded database to start up: .[OK]

Furthermore, most of the potentially useful VMware KB articles assume a dedicated vCenter, rather than a vCSA. There seems to be no way out of this boot hang, and you will have to start deploying the vCSA all over again. The trick is to cancel out of the Setup wizard and change the hostname at the very start, before the various web services have been initialized at all.

I have outlined the successful setup process here as a reminder for future deployments:

  • When downloading the vCSA, download the .ova file. Ignore the other .vmdk and .ovf downloads, the .ova one will allow you to set up the IP address details at the time of deployment rather than having to change them later.
  • It is recommended to use Thick Provisioning for the disk, despite 120GB being overkill for typical small deployments.
  • Enter the chosen static IP address, network and DNS settings. Sadly they left out a hostname field here which would have saved a lot of grief.
  • Do not start the VM on completion of the wizard, then reduce the vCSA virtual machine RAM from 8GB down to 4GB – the supported minimum configuration for small deployments of less than 10 hosts.
  • As per the instructions displayed on the vCSA VM console, connect a web browser to port 5480 (the default credentials are root and vmware), and accept the EULA.
  • At this point quit the Setup wizard. Failure to do this will cause SSL certificate issues later.
  • In the Network tab change the hostname, in the Admin tab change the root password and enable SSL certificate regeneration, and finally in the System tab set the correct time zone then reboot.
  • Create a DNS A record for your vCSA.
  • After the reboot when you connect back to the admin console, the hostname in the certificate will remain as localhost.yourdomain.com – this is apparently normal.
  • Disable SSL certificate regeneration.
  • Start the Setup wizard again from the main vCenter Server tab Summary screen.
  • Select “Configure with default settings”.
  • Enable AD authentication (an Active Directory Computer account object will be created for your vCSA).
  • Despite having enabled Active Directory authentication, this will not work until the AD domain SSO Identity Source has been configured – you’ll see the error “A general system error occurred: Authorize Exception”. The VMware documentation mentions using the SSO Admin account to do this (admin@System-Domain) but this password is not defined by the user when deploying a vCSA. By default the root account is also an SSO admin, so log in to vSphere Web Client on port 9443 as root. Navigate to Administration > Sign-On and Discovery > Configuration > Add Identity Source:
    vCSA - Add SSO Identity Source
  • Enter the server URLs in the format ldap://dc1.mydomain.com
  • The DNs will probably both be cn=Users,dc=yourdomain,dc=com since you are only likely to need the Domain Admin user or Domain Admins group.
  • You need to set the Domain Alias to the NETBIOS domain name or else the vSphere Web Client plugin’s “use windows credentials” option will not work.
  • Set Authentication Type to Password and use a basic AD user account that you might use for photocopier scan-to-email directory lookups. Test the settings and save once working ok.
  • vCSA does not implicitly trust Domain Admins like a standard vCenter installation would, and the permissions are somewhat difficult to find in the vSphere Web Client. Navigate Home > vCenter > vCenter Servers > your vCSA > Manage tab > Permissions:
    vCSA edit vCenter permissions
  • Add your required AD groups to the role Administrator.
  • I had a further issue after all this, where connecting using vSphere Client with the “use windows credentials” option would result in the error “A General System error occured: Cannot get user info”. This is due to an omission in the vCSA which has been fixed in the latest download version (currently 5.1.0.10100-1123965). You can get around this by editing a config file on the vCSA.
  • One final reboot of the appliance is necessary to avoid permissions errors from the Inventory Service when logging into the vSphere Web Client with Windows session authentication. Be aware that the port 9443 web service can take several minutes to start, even after the console of the appliance has apparently finished booting.
  • In environments where you can’t afford the 120GB of disk space for the vCSA, you could use VMware Converter to V2V the appliance, resizing to say 40GB in the process.
 
 

Shortcut to testing VMware Auto-Deploy

Leave a reply

It’s very useful to have a VMware lab environment particularly for training needs, and outside of larger enterprises this is generally difficult to achieve owing to the high time cost of setting one up. VMware Auto Deploy remedies this problem since it allows very rapid provisioning of ESXi hosts via PXE network boot.

I wrote these notes for myself since I’m getting some hands-on experience of it to upgrade my VCP4 qualification to VCP5. EDIT – I passed the exam :). I got the information about the Auto Deploy commands from this concise guide. My post here covers how to deploy this if you already use Windows DHCP and WDS, and in such a way that uses but does not impact your production VMware infrastructure. The key to this is using the vCenter Server Appliance to create (with just a few clicks) a separate vCenter on a different VLAN, hosted on your existing ESXi hosts. I think some of the required file downloads may need you to have a proper paid for vSphere licence.

  • Create a new VLAN, add it to your ESXi hosts’ VM Network trunk ports. Define a gateway for the new subnet on your router. In a Cisco environment you would also need to configure your DHCP server as an IP-helper. That’s beyond the scope of this document, and I assume that you probably already run a DHCP server that manages multiple subnets.
  • Obtain and install the VMware vCenter Server Appliance. Log in to the vmware.com site, go as if to download vCenter server, then after the EULA you’ll see multiple files offered (the .iso you would normally download, then further down the .ovf and .vmdk files for the appliance).
  • Install this appliance on your existing ESXi environment. Follow the instructions on screen and select Embedded Database. The default credentials are root and vmware. I reduced the VM’s RAM from 8GB down to 4GB. Ideally put this VM in the new VLAN you created.
  • This appliance already has the Auto Deploy option installed, but it’s disabled (in the Appliance WebUI go to Services > Auto Deploy). Impressively, the appliance also hosts the web services for the vSphere Webclient by default, which you can access from a browser on https://YourApplianceIP:9443/vsphere-client. Given the hassle-free setup for all this, I think it’s highly likely I will transition the production environment over to this instead of running vCenter on Windows.
  • If you’re already using WDS and Microsoft DHCP, make sure you’re defining options 66 and 67 (boot server & boot filename) on a per-scope basis (not in Server Options), that way we can configure different behaviour for the new subnet.
  • Launch vSphere Client and target it at your vCenter Server Appliance. From Home, select Auto Deploy. Select Download TFTP Boot Zip. Extract these files to the root level of your REMINST share on your WDS Server. These bootstrap files are designed to live at the TFTPRoot, not in the boot folder with the other WDS boot files (file paths that are loaded after the initial boot are hard-coded, not relative).
  • By default, the WDS TFTP server only allows read access to \tmp and \boot. You need to use Regedit to edit HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WDSServer\Providers\WDSTFTP\ReadFilter. Append “\*” to the existing value. Where I work the WDS server only serves boot images since I favour unattended install rather than image-based deployment, so I’m ok with this relaxing of the default security.
  • Create a DHCP scope for your new subnet (including a reservation for your vCenter Server Appliance). Define option 67 as undionly.kpxe.vmw-hardwired
  • Now you can connect some test physical machines with Intel VT support to the new VLAN you created and set them to boot to PXE first in the boot order. Test this – you should get a warning that there are no associated ESXi images for this machine. Note down the displayed model name (in my case OptiPlex 745).
  • Download an ESXi offline bundle (go as if to download ESXi from vmware.com, accept the EULA, and it’s further down on the download page).
  • Use a Windows 7 machine (which already has PowerShell 2.0) to install vSphere PowerCLI.
  • Connect the PowerCLI to the vCenter Server Appliance (help on the cmdlets is available here):
    Connect-VIServer 172.16.10.100
  • Add the offline bundle as a “depot”:
    Add-EsxSoftwareDepot C:\users\me\Downloads\ESXi500-201111001.zip
  • Within this depot there are several images, and we need their names:
    Get-EsxImageProfile | fl
  • Create a deployment rule:
    New-DeployRule -Name "FirstTimeBoot" -Item "ESXi-5.0.0-20111104001-standard" -Pattern "model=OptiPlex 745"
  • Activate it:
    Add-DeployRule -DeployRule FirstTimeBoot

Now your ESXi test machines can be woken with WOL, will boot ESXi and will automatically bind to vCenter Server Appliance, where they can be managed. Perfect for home study using a VPN connection. Here is the official VMware Auto Deploy Administrator’s Guide.

Upgrading to vCenter 4.1 with bundled SQL Express Edition – database migration fails

5 Replies

My infrastructure uses a bundled SQL Express Edition database because it isn’t hugely complex, and I didn’t want too much dependency on other servers (themselves VMs). I encountered problems upgrading the vCenter database while moving from Windows Server 2003 R2 SP2 x86 to Windows 2008 R2. I was migrating from vSphere 4.0U1 to 4.1. Perhaps this precise combination of versions was the problem, or perhaps it was that my database started life as VirtualCenter 3.5.

The process seems simple enough – unzip and run the Data Migration Tool from the installation media on the source vCenter server, move this folder (now with \data added) to the destination server and launch the install.bat script. However, there seem to be two major snags. The first is that the backup process will fail:

DB logs: HResult 0x2, Level 16, State 1 Named Pipes Provider: Could not open a connection to SQL Server

VMware has a knowledgbase article about this. They claim it’s caused by a misconfiguration of the SQL 2005 Express Edition instance which is pretty rich considering it was set up by their own installer. Make the named pipe change they suggest and it will work. Now unplug this machine from the LAN or disconnect its network adapter in vSphere if it’s a VM – remember you can connect the VI Client directly to the hypervisor which is hosting it.

The destination server needs to be configured with the same hostname as the source. I had then assumed that the restore tool would need to be run after a new install of vCenter 4.1 was placed on the destination server, so I installed vCenter. Then I discovered that the install.bat script in the datamigration folder refuses to run if it detects the product is already present. So naturally I uninstalled it and tried again. Perhaps this is what messed things up, or perhaps it’s because I’m using Windows 2008 R2.

Anyway, the datamigration\install.bat script kicks off the main product installer, supposedly importing all your backed up settings.

According to the vCenter 4.1 Upgrade Guide page 40 item 10 you are supposed to:

Select Install SQL Server 2005 Express instance (for small-scale deployments) and click Next.

Item 16 on the same page states that:

When the vCenter Server installation finishes, click Finish. The data migration tool restores the backed up configuration data.

If you do this you may, like me, discover that it actually doesn’t work and you end up with a completely blank database instance. Consulting datamigration\logs\restore.log I found no reference at all to any database restore.

My workaround

Go to your original vCenter server. Open the Data Sources MMC snap-in. In System DSN you should see an entry like so:

Note down the details, then create the same entry on your destination server (this will create a 64bit DSN). Notice how on page 38 of the Upgrade Guide it specifically states that:

If you use the data migration tool to migrate a SQL Server Express database located on the vCenter Server system to a new system, you do not need to create the 64-bit DSN. The data migration tool creates the DSN as part of the installation process

Apparently sometimes you do need to create the DSN.

On the destination server, copy the file \datamigration\data\vc\vc_upgraded_db and paste it in C:\temp. Rename it to vc_upgraded_db.bak.

Still on the destination server download, install and run the 64bit SQL Management Studio Express. Even if you’ve uninstalled vCenter, the SQL Express Edition instance will be left behind. If there’s not already one from a failed install, create a local database called VIM_VCDB. Restore the backup in C:\temp\vc_upgraded_db.bak over the top, paying attention to select Options -> Overwrite Existing Database and browsing to the target file locations of both the mdf and ldf files – the old database was found in C:\Program Files\Microsoft SQL Server\MSSQL.1\Data but on the 64bit system it’s C:\Program Files (x86)\…

Right-click your database, and select Properties. In Options, make sure your database recovery model is set to Simple. If you don’t do this your transaction log will fill up in a couple of days and the vCenter services will stop. In my case it seemed to have defaulted to Bulk-Logged.

Once that’s done you’ll need to update your newly created DSN to set the default database to VIM_VCDB. Now run datamigration\install.bat once again but this time opt to use an existing database as shown below:

Strangely, you will find that the you cannot use the SYSTEM account for the vCenter services however this can easily be changed in the Services MMC snap-in later.

And that’s it. You should end up with a working install with your data intact. One final tweak is to delay launch of the vCenter services so they don’t fail to start up at boot time.

UPDATE – after wasting a number of hours today with this, I’ve done some more searching and found this VMware KB article which basically just admits that the Data Migration Tool sometimes doesn’t work, and won’t even report errors in the log! When I download something as important as this, I sort of take it for granted that I won’t have to Google “vcenter data management tool does not migrate database” the moment I try using it (can’t believe I didn’t try that).