Understanding Datastore Inaccessibility in VMware

A datastore, as the name implies, is a repository that stores all the files and data of a VMFS virtual machine. It also stores information about the server components, such like the vSwitches. VMFS datastores are saved as VMDK files on the VMFS partition of a physical hard drive formatted for virtual machine hosting.

But just like every other file type, datastore files can get corrupted or damaged, making them inaccessible for the attached VM. When this happens, all VMs attached to the datastore won’t be able to launch. If you’re facing datastore inaccessibility issues, this article explains the workarounds you can try out.

Why is My VM Inaccessible?

There are many reasons why a virtual machine could be inaccessible; one of these reasons is datastore problems and other can include, server problems, corrupt VM files, corrupt VMDK file, or failed RAID. If your VM storage is still showing online, then the most likely reason why the VM is inaccessible is due to network failure. Of course, your reaction to the situation depends on the actual cause of the issue – VM inaccessibility.

1. Network Failure:

This is one of the commonest causes of VM inaccessibility, but then, it also has the simplest solution. If network connection is the reason you can’t access your VM, then you should check the network cables and storage.

2. Disk Failure:

For VMs connected to a RAID storage, if one drive in the array fails, the VM may not start. In this case, you need to replace the failed drive with a new one and rebuild the array (if it doesn’t support hot-swapping).

3. Corrupt or Damaged Files

Files stored on virtual drives are not immune to data corruption, they can equally get corrupt and inaccessible for the same reasons files on a physical drive would get damaged. That said, one of the reasons a VM could be inaccessible is because the datastore file(s) are corrupted, missing, or damaged.

In this scenario, you need to restore the original datastore file(s) and repair it – using a third-party software. Also, if the VM files are not properly configured, your VM could be pointing to an empty datastore with no host on it, so, you should re-check the VM configuration files.

How to Resolve VMFS Datastore Inaccessibility

There are a couple of other ways to resolve VMFS datastore inaccessibility concerns; these are not-so-technical methods, but you still have to follow them carefully.

1. Reload Inaccessible VMs

Virtual machines might be grayed out or marked as “inaccessible” after a permanent device loss (PDL) situation or after you restored a previously failed datastore. To fix this, you have to reload the VMs via CLI using the commands below.

• First, log in to your ESXi host via SSH client and run these commands:
• View all VMs to ascertain which ones need to be reloaded: vim-cmd vmsvc/getallvms
• VMs with the “Skipping Invalid VM” tag are the inaccessible ones that need to be reloaded. Take note of their IDs because that’s what you will use in reloading them.
• Run this code to reload the invalid VMs, one after another: vim-cmd vmsvc/reload 12
• “12” is the ID of the inaccessible VM that needs reloading; replace this number with your actual VM’s ID.
• Run the reload command for each of the invalid VM, when done, use the “getallvms” command to confirm that all your VMs are now accessible and no longer shows “invalid”.

2. Move VMs from Inaccessible Datastores

Another way to resolve VMFS datastore problems is to move all connected VMs to another datastore or host.

• Create a new datastore and clone the virtual machine you want to migrate on vSphere Client – right-click on VM to clone on vSphere Client and select “Clone to Virtual Machine.”
• When the Cloning Wizard prompts, use a unique name for the clone VM and choose a different location in the inventory.
• Leave the rest of the configuration without making any changes and complete the cloning wizard prompts.
• Once complete, right-click the clone VM in the inventory panel and select “Edit Settings.”
• Go to the “Hardware” tab and review the attached virtual disks
• Now, point out the virtual disks to migrate to the new datastore.
• In a bid avoid file conflicts, remove the cloned VM; right-click the cloned VM and select “Remove from Inventory.”
• Enable SSH on the ESXi host where the virtual disks are stored; connect to the host via SSH.

• Go to the VM’s directory on the datastore, the directory should be /vmfs/volumes/datastore_name/vm_name/

• Now, use the vmkfstools command to copy the virtual disks from the current datastore to the new datastore: vmkfstools -i source_disk_path -d thin destination_disk_path
• (repeat this command for each disk to migrate and ensure to replace the “source disk path” and “destination disk path” with the actual directories of the VM data you’re migrating to a new datastore.)
• When the migration is done, you need to register the VM to the new datastore using the vSphere Client. For this, right-click the .vmx file of the original VM (not the clone) and choose “Add to Inventory.”
• Follow the on-screen instructions add the VM back to the inventory.
• Power the VM from the new datastore and verify if the VM is using the disks on the new datastore.

3. Handling Volume Status Changes.

If your VMFS volumes suddenly start showing “offline” status, you can get them back online manually. Usually, this is caused when the volume gets unmounted and made offline; thus, causing the junction path to be removed; so, even if the volume is made online again, the junction path has been removed already and needs to be re-mounted.

The solution is, after bringing the online, you have to manually mount the volume with the same junction path it was connected to.

Conclusion

Datastore inaccessibility can be resolved in many ways, depending on what triggered the situation. Regardless, it is advisable to always back up your VM data to an external storage, and if you experience any data loss, DiskInternals VMFS Recovery can help to get back your files.