How to release locked Files of a VM?

Sometimes a file or set of files in a VMFS become locked and any attempts to edit them or delete will give a device or resource busy error, even though the vm associated with the files is not running.  If the vm is running then you would need to stop the vm to manipulate the files.  If you know that the vm is stopped then you need to find the ESX server that has the files locked and then stop the process that is locking the file(s).

  1. Logon to the ESX host where the VM was last known to be running.
  2. vmkfstools -D /vmfs/volumes/path/to/file to dump information on the file into /var/log/vmkernel
  3. less /var/log/vmkerneland scroll to the bottom, you will see output like below:
    1. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)FS3: 130: <START vmware-16.log>
    2. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)Lock [type 10c00001 offset 30439424 v 21, hb offset 4154368
    3. Nov 29 15:49:17 vm22 vmkernel: gen 66493, mode 1, owner 46c60a7c-94813bcf-4273-0017a44c7727 mtime 8781867]  ß Bold type added to number for emphasis.
    4. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)Addr <4, 588, 7>, gen 20, links 1, type reg, flags 0x0, uid 0, gid 0, mode 644
    5. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)len 23973, nb 1 tbz 0, zla 2, bs 65536
    6. Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)FS3: 132: <END vmware-16.log>
  4. The owner of the lock is on line 3c, the last part is all you need, in this case 0017a44c7727

  5. esxcfg-info | grep -i ‘system uuid’ | awk -F ‘-‘ ‘{print $NF}’ will display the system uuid of the esx server.  You need to run the esxcfg-info command on each esx server in the cluster to discover the owner.
  6. When you find the ESX server that matches the uuid owner, logon to that ESX server and run the command: ps -elf|grep vmnamewhere vmname is the problem vm.  Example output below:
    1. 4 S root      7570     1  0  65 -10    –   435 schedu Nov27 ?        00:00:02 /usr/lib/vmware/bin/vmkload_app /usr/lib/vmware/bin/vmware-vmx -ssched.group=host/user/pool2 -@ pipe=/tmp/vmhsdaemon-0/vmxf7fb85ef5d8b3522;vm=f7fb85ef5d8b3522 /vmfs/volumes/470e25b6-37016b37-a2b3-001b78bedd4c/iu-lsps-vstest/iu-lsps-vstest.vmx0
  7. Since there is a process running, pid 7570 in the example, you need to kill it by following steps 5-12 on stopping a VM above

Once the kill is complete the files should be released.

2390 Total Views 2 Views Today

Leave a Reply