Musings of a sys admin
Uncategorized
VMware High Availability
Nov 9th
High availability or HA as I will call it from now on, is a feature of Virtual Center which allows for the automatic restart of VM’s in the event of a host failure.
For example if you had 4 ESX servers running 40 VM’s (10 on each). if one host goes pop then HA would detect the failure and restart the VM’s on the 3 remaining hosts. However of course whilst there is not a great deal of options to fiddle about with (most of them follow the same pattern) you do have an important decision to make, which is if a host fails do you want to restart your VM’s or would you rather they stay down. This is basically is it more important that all the vm’s are up and running but possibly slower than normal, or would you rather some or all of the VM’s stay down until you have dragged yourself out of bed and into the office to fix the issue.
HA can be enabled once you have created a cluster by right clicking on the cluster and selecting “Edit Settings”. The first screen you will see consists of 2 check boxes, one for enabling/disabling HA and one for enabling/disabling DRS. The choice here is self-explanatory but you might want to spend a minute reading the couple of paragraphs on that page.
The next tab worth looking at is the Vmware HA tab there are 3-4 options here that you will need to consider.
The first option is Admission Control, within that setting is the options to set the number of host failures the cluster can tolerate this can be any number between 1-4. This by default is set to 2 and of course if you suddenly find yourself loosing 4 hosts in your cluster then you have a rather large problem on your hands. The next option is to prevent or allow the powering on of vm’s if they violate availability constraints. This means basically do you want to allow VM to be powered on even if the total number of configured memory resources exceeds the actual resources that the cluster provides.
Maths bit: You can work out your availability constraints by taking the amount of ram provided by your smallest ESX host (I.E, the one with the least amount of physical memory) and then find your vm with the most amount of configured memory and divide the ESX memory by the vm ram which will give you your figure of the amount of guest vm’s each host can have, any more than that and your availability constraints have been violated!
Example:
6 ESX Hosts smallest has 24 GB of ram largest amount of guest ram is 2GB and host failure is set to 1.
24/2×5=60
So if one host fails the total amount of virtual machines that can be powered on with violating availability constraints is 60. If you need any more than that then you will need to allow the vm’s to power on even though they violate the constraints.
The next setting is the default cluster setting, within the settings is the vm restart priority which is by default set to medium This setting is cluster wide but can be overridden at individual vm level. The configurable options for the restart policy are high, medium low and disabled. Clusters set to high have their vm’s restarted first, medium next and low last and disabled.. well you get the idea.
The next option down is the host isolation response setting, I’ve blogged about this before so I wont go into too much detail but it basically means when a host loses its network connection to other hosts in the cluster. Network connectivity is checked regularly by a heartbeat (ping). It is possible in vmware to have a situation where the host has lost its network connection to the other hosts in the cluster but the vm’s on that server are perfectly happy and working away normally. In previous updates of esx the default behaviour was to power off the vm which would leave HA to restart the vm’s on another server, However from U2 onwards this changed to leave vm powered on because you will find 9 times out of 10 just the service console is having issues and the vm’s are still working away quite happily.
The next option is “virtual machine monitoring” this again is an experimental feature which allows virtualcenter to monitor the tools installed on the vm’s (by using heartbeats in a very similar way to HA). You an enable this option and use the slider to adjust the sensitivity. Virtual machine monitoring does know when a vm has been purposefully shutdown or powered off so it will not be constantly restarting machines you are trying to shutdown.
The only button on this page is the Advanced Options which you should only make changes to this once you know what you are doing.
The sub tab of the HA options is “Virtual Machine Options” this allows you too specify different restart priority and host isolation response behaviours for individual vm’s. So you can effectively prioritize/delay or disable the restarting of individual vm’s should you deem appropriate.
Vmware common myths
Oct 28th
I think this is a post that i will keep updating as i think of things, but i thought i would start out with nice easy ones to get going with.
Something that quite often happens with a new esx farm is the admins want to tentively vm a ‘low risk’ server thinking that its not the end of the world if it goes skyward.
This normally translates as a server that does not do much and has been sat in the corner of the server room for years banging away doing its thing. Now of course when its virtualized its given a whole new set of hardware that is years away from what its used too. This gets admins and users very excited as whatever that server used to do is now been given a massive boost in performance.
Virtualization is not about speed its about consolidation. As admins start virtualizing other boxes the old server may very well go back to about the speed it was before.
More to follow
VMware NTP Weirdness
Oct 26th
Well today I discovered that one of the ESX hosts in our cluster did not have its NTP settings correctly configured, although it did take me a little while to figure it out.
We have a 2003 guest box on the host which when I logged onto it to do some work was displaying a totally wrong date and time. So I set it back manually in the OS (we dont use DHCP for our server subnet). Did not think much of it until 10 minutes later all of a sudden the correct date and time reverted back to the wrong date and time. I checked the 2003 boxes settings again and it wasnt synching its time with any internet NTP server which is how we like it. So I had a look at the vmware tools installed on the box and sure enough that was not set to synch time with the host so again I ruled that out.
So I merrily set the date and time again and went on my way and sure enough in another 10 minutes the date/time had reverted. so I checked the host and found that NTP was not enabled and was displaying the same date and time as the 2003 guest OS. I set up NTP on the host and this cured the issue.
I do find it somewhat strange that even though the guest was not set to sync date/time with the host it still did it, presumably after the w32 service failed to sync with an online ntp service the VMware tools took over and synched it anyway.
Weird, I feel a bit of googling coming on!
Transcender sale 20-21 oct 2009
Oct 20th
I have just had an email come in saying that transcender are have a 40% off sale on all their items for 2 days only.
You may remember that I reviewed a transcender product not too long ago and it came out very well against the product I had normally used. So if you have not used transcender yet now is a good time to give them a go.
www.transcender.com
Virtualizing Linux OS
Oct 14th
Just a quick observation considering vmware has a linux based service console and lots of options for new linux builds, its really crap at virtualizing physical boxes!
Learning Java the easy and fun way
Oct 1st
I was flicking through the usual linux mags I buy the other and came across a link to a java learning tool called Robocode. Its basically a java game where you can pit your tank against others (either the default’s or the ones that other people have written). There are a good number of tanks to choose from to start with and then you edit the code to improve your tanks abilities.
HA Host Isolation Response
Sep 28th
Host isolation is basically a state that a host in an HA cluster can reside in should it detect a loss of network connection. However there are a good few things to know about this feature. When a host is in a HA cluster it sends out a heartbeat to other hosts in the same HA cluster. This enables the cluster to detect when a host has failed and power on the VM’s that it was running should that be the setting you choose.
Firstly the Automated Availability Manager (AAM) controls the heartbeat process through the Service Console and its configured address.
However this alone presents itself with a problem. If you only have one Service Console installed on a host it is possible that AAM will think a host has failed when in actual fact a failure has only occurred somewhere on the path from the Service Console mapping to the physical NIC (uplink) and outwards. With a host that has multiple uplinks (a full blown ESX Production server will need at least 3 or more) It could mean that the Virtual Machine port group on another physical NIC could be happily working away.
After 15 seconds of missing a heartbeat response each node will ping the default gateway for its service console (this is called the Isolation Response Address). This is basically the hosts way of saying “is this problem my fault”. If the host receives a response from its default gateway it carries on as normal as the fault is not with the host. If the host however does not receive a response it will go into isolation mode.
The host(s) that enters isolation mode would then read their isolation configuration which will tell the host either to power off the VM’s or leave them running.
The correct configuration for this setting relies pretty much on you network. If for example you have a vswitch configured with a Virtual Machine port group and the Service Console port group then as you’ve lost your Service Console you’ve also lost your VM network too so you will want to power the VM’s off and let HA bring them all back up again on other hosts. However if your network is redundant enough loosing a service console isnt really a big thing if your Virtual Machine and Vmotion networks are still working.
It is considered best practice to have 2 Service Console’s configured per host on different vswitches,vmnic’s and physical networks. this sounds a bit complicated but when you consider that Vmotion should really have its own dedicated physical switch and subnet then the choice on where to put the Service Console suddenly becomes very clear.
In the Next blog I will talk about the actual steps you need to take to configure Host Isolation and a few extra switches that you may need.
VMware Install & config Course
Sep 22nd
Hi all,
Well today was the first day of my install & config course. So far its nothing particularly new although it is good to meet with other IT bods and hear there views and experiences of vmware. I suspect that a few vmware releated posts will pop up.
Vmware install and configure course
Sep 2nd
Hi all,
Work has kindly agreed to pay for my vmware course for me which is very appriciated. I wanted to do the course so I can take the vcp exam and become a vcp.
There is alot of cool stuff to learn in order to pass the exam and I am quite sure that virtualization is the way of the future so it will help me imensely if something untoward happens to my job (I doubt it but you never know).
Also I discovered that if i get my skates on I can also take the vsphere exam before the end of the year without having to go on another course. I doubt I will get round to it as my open university course also starts soon so I think I will be a busy boy for the next few months. Still strings to bows.. Strings to bows!
ESX 3.5 Evaluation
Aug 19th
Hi all,
As stated in my previous posts im looking to set up a lab for ESX which involve using the vmware server for windows xp to virtualize ESX boxes plus setting up some ISCSI Luns on freenas for storage. However looking at the vmware site I thought my plans were dashed before they had even started. Any mention of ESX 3.5 eval versions had disappeared and naturally been replaced by vsphere evals. Bugger thinks I, so I put up a post on VMware communities and some bright spark told me to sign up for an evaluation of vmware view and because it only works on on ESX 3.5 they provide you with trials of ESX and virtual center as well. So I hot footed it to the viewer page and signed myself on to an evaluation, sure enough there were the ISO’s for ESX too.
Marvelous isnt it!