Hello all,
I'm preparing myself for VCP5-DCA exam, and I'm building my personal testlab, to try some features where I need to improve.
At the moment I have following configuration:
3x ESXi host: 10.0.0.20 - 10.0.0.22
1x NFS server(Linux) - 10.0.0.30
1x AD(Windows) - 10.0.0.10
1x VCC(Windows) - 10.0.0.11
Network works perfectly between all machines. All machines are VMs on VMware Worstation 10.
All 3 ESXi hosts are 5.5(Build 1331820). I have exported 3 NFS shares on NFS server, and mounted them on all 3 ESXi hosts, they are writable:
C:\Program Files (x86)\VMware\VMware vSphere CLI\bin>esxcli -s ESXi1 -u root storage nfs list
Enter password:
Volume Name Host Share Accessible Mounted Read-Only Hardware Acceleration
----------- --------------- ------ ---------- ------- --------- ---------------------
NFS1 nas.tntwrk.info /nfs/1 true true false Not Supported
NFS2 nas.tntwrk.info /nfs/2 true true false Not Supported
NFS3 nas.tntwrk.info /nfs/3 true true false Not Supported
C:\Program Files (x86)\VMware\VMware vSphere CLI\bin>esxcli -s ESXi2 -u root storage nfs list
Enter password:
Volume Name Host Share Accessible Mounted Read-Only Hardware Acceleration
----------- --------------- ------ ---------- ------- --------- ---------------------
NFS1 nas.tntwrk.info /nfs/1 true true false Not Supported
NFS2 nas.tntwrk.info /nfs/2 true true false Not Supported
NFS3 nas.tntwrk.info /nfs/3 true true false Not Supported
C:\Program Files (x86)\VMware\VMware vSphere CLI\bin>esxcli -s ESXi3 -u root storage nfs list
Enter password:
Volume Name Host Share Accessible Mounted Read-Only Hardware Acceleration
----------- --------------- ------ ---------- ------- --------- ---------------------
NFS1 nas.tntwrk.info /nfs/1 true true false Not Supported
NFS2 nas.tntwrk.info /nfs/2 true true false Not Supported
NFS3 nas.tntwrk.info /nfs/3 true true false Not Supported
On Linux NFS server I exported all NFS as following:
| /nfs/1 | 10.0.0.0/24(rw,no_root_squash,no_subtree_check,async) |
| /nfs/2 | 10.0.0.0/24(rw,no_root_squash,no_subtree_check,async) |
| /nfs/3 | 10.0.0.0/24(rw,no_root_squash,no_subtree_check,async) |
There is enough disk space on all hard disk partitions that are exported via NFS(I've also checked IO-nodes usage, and it's around 1%):
| Filesystem | Size Used Avail Use% Mounted on |
| /dev/sdb | 99G 1.1G 93G 2% /nfs/1 |
| /dev/sdc | 99G 62M 94G 1% /nfs/2 |
| /dev/sdd | 99G 62M 94G 1% /nfs/3 |
Now I created VM1 and wanted to power it on, but I get following error:
An error was received from the ESX host while powering on VM VM1.
Failed to start the virtual machine.
Failed to power on VM.
Could not power on VM : msg.vmk.status.VMK_NOT_FOUND.
Failed to create swap file '/VM1/./VM1-fcddfe96.vswp' : Not found
VM1 have no memory reservation and is configured to have 512MB, so I expect to have 1 swap file with 512MB size.
When I inspect VM1 log file I can see following:
2013-12-21T19:26:11.578Z| vmx| I120: HV Settings: virtual exec = 'hardware'; virtual mmu = 'hardware'
2013-12-21T19:26:11.588Z| vmx| I120: OvhdMem_PowerOn: lib/overheadmem: paged 54983 nonpaged 12473 anonymous 10025
2013-12-21T19:26:11.588Z| vmx| I120: OvhdMem_PowerOn: initial admission: paged 27889 nonpaged 3199 anonymous 3222
2013-12-21T19:26:11.588Z| vmx| I120: VMMEM: Initial Reservation: 134MB (MainMem=512MB)
2013-12-21T19:26:11.588Z| vmx| I120: MemSched_PowerOn: balloon minGuestSize 26214 (80% of min required size 32768)
2013-12-21T19:26:11.588Z| vmx| I120: MemSched: properties ovhd: 0 mainmem: 0 pinned: 0 fpt: 0 ls: 0 prealloc: 0
2013-12-21T19:26:11.589Z| vmx| I120: numa: VCPU 0: VPD 0 (PPD 0)
2013-12-21T19:26:11.589Z| vmx| I120: numaHost: 1 virtual nodes, 1 virtual sockets, 1 physical domains
2013-12-21T19:26:11.589Z| vmx| I120: CreateVM: Swap: generating normal swap file name.
2013-12-21T19:26:11.593Z| vmx| I120: Swap file path: '/VM1/./VM1-fcddfe96.vswp'
2013-12-21T19:26:11.593Z| vmx| I120: VMXVmdb_GetDigestDiskCount: numDigestDisks = 0
2013-12-21T19:26:11.594Z| vmx| I120: Msg_Post: Error
2013-12-21T19:26:11.594Z| vmx| I120: [vob.swap.poweron.createfailure.status] Failed to create swap file '/VM1/./VM1-fcddfe96.vswp' : Not found
2013-12-21T19:26:11.594Z| vmx| I120: [msg.vmmonVMK.creatVMFailed] Could not power on VM : Not found.
2013-12-21T19:26:11.594Z| vmx| I120: [msg.monitorLoop.createVMFailed.vmk] Failed to power on VM.
2013-12-21T19:26:11.594Z| vmx| I120: ----------------------------------------
Cluster is configured to store swap files together with VMs(default settings), so I assume it's having problem to create swap file on NFS.When I login via SSH to ESXi host directly and try to create SWAP file manually I get following(from my perspective) correct output:
~ # ls -lah /vmfs/volumes/NFS1/VM1/
total 124
drwxr-xr-x 1 root root 4.0K Dec 24 2013 .
drwxr-xr-x 1 root root 4.0K Dec 24 2013 ..
-rw------- 1 root root 8.0G Dec 24 2013 VM1-flat.vmdk
-rw------- 1 root root 490 Dec 24 2013 VM1.vmdk
-rw-r--r-- 1 root root 0 Dec 24 2013 VM1.vmsd
-rwxr-xr-x 1 root root 1.7K Dec 24 2013 VM1.vmx
-rw-r--r-- 1 root root 258 Dec 24 2013 VM1.vmxf
-rw-r--r-- 1 root root 48.9K Dec 24 2013 vmware-1.log
-rw-r--r-- 1 root root 48.9K Dec 24 2013 vmware.log
~ # touch /vmfs/volumes/NFS1/VM1/./VM1-fcddfe96.vswp
~ # ls -lah /vmfs/volumes/NFS1/VM1/
total 124
drwxr-xr-x 1 root root 4.0K Dec 24 2013 .
drwxr-xr-x 1 root root 4.0K Dec 24 2013 ..
-rw-r--r-- 1 root root 0 Dec 24 2013 VM1-fcddfe96.vswp
-rw------- 1 root root 8.0G Dec 24 2013 VM1-flat.vmdk
-rw------- 1 root root 490 Dec 24 2013 VM1.vmdk
-rw-r--r-- 1 root root 0 Dec 24 2013 VM1.vmsd
-rwxr-xr-x 1 root root 1.7K Dec 24 2013 VM1.vmx
-rw-r--r-- 1 root root 258 Dec 24 2013 VM1.vmxf
-rw-r--r-- 1 root root 48.9K Dec 24 2013 vmware-1.log
-rw-r--r-- 1 root root 48.9K Dec 24 2013 vmware.log
~ #
Cluster is configured for HA, HA datastores are: NFS1 and NFS3 (I think this information is not relevant, but wanted to highlight it).
Do you have any ideas where problem can be?