Hyper-V Missing VMs

Over the past 6 months I installed 4 Server 2019 Hyper-V hosts for various clients. After several months with no problems, following a reboot, all running VM’s completely disappeared from the Hyper-V management console and were not accessible from the network using management tools, file shares, remote desktop, or even pings. Oddly, shut down or saved VM’s were present.

When this first happened I was shocked. The VHDX files were all present so I could create a new VM, but that didn’t seem practical. Googling showed that this can happen if the Hyper-V Virtual Machine Management service did not start, but in my case it had. I tried restarting the service, the VM’s instantly reappeared, and were in a running state with boot up almost complete.

This issue over the coming months started happening on other 2019 servers and after every reboot, planned or due to a power outage, I had to connect to the host and restart the Hyper-V Virtual Machine Management service.

Further Googling this issue brings up suggestions of corrupt VM configuration files, granting “NT Virtual Machine\Virtual Machines” the “logon as a service right”, doing the same with group policy, and other suggestions, but where restarting the service would resolve in every case I assumed there was not a configuration issue.

In the end setting the Hyper-V Virtual Machine Management service start up type to “Automatic (delayed start)” resolved the problem on all machines, though it resulted in a slightly longer boot time for the VMs.

All of thee servers worked fine for a few months so I assume the problem was due to a Windows update but to date I have found no actual cause. Also, I can confirm this only occurs on my 2019 Hyper-V hosts. There are no issues with Server 2016 or earlier servers.

Hyper-V restart

Update: Oct 2020. I had another server with the same issue. O/S had been installed 8 months prior and no updates applied in recent months. Setting the service as delayed start did not resolve. I had to create a scheduled task to run 10 minutes after boot up. 5 minutes did not work. The scheduled task simply pointed to a batch file with the following. (the ping command just delays the process to be sure Net Stop completes before the next line).

Net Stop VMMS
ping -n 10 127.0.0.1
Net Start VMMS
Exit

Comments on: "Hyper-V Missing VMs" (15)

  1. Gday, I can confirm I’ve run into this issue recently, no idea what is the cause. Delayed start for VMMS didn’t make a difference. At the moment the only solution I have is a scheduled task on boot which will stop the VMMS and then start it again, once that happens, presto VM’s are visible and all start up.
    Did you ever get any further to finding a cause?
    Regards,
    Steve

    • Hi Steve. The delayed start seems to have resolved for me but I can see a scheduled task working as you have done. It is very odd. I set up about 5 2019 servers over a 8 month period and everyone started doing the same thing after 3-4 months. I initially thought it was a Windows update but I apply all updates when setting up a server so the latter ones would have had the previous updates installed initially. I haven’t set any up since the pandemic to see if the same issue persists on new servers. I don’t know if it is relevant, but all servers were HP, though not all the same model. No firmware updates were applied after the initial installation.

      • glimmerman101 said:

        Hi Rob, this server is a HP also, Proliant ML350 G10. I thought it was a Windows updates as we have a few more updates installed on this particular host compared to the other host machine in this environment (both deployed at the same time, same model etc) – at this stage no other 2019 machines exhibit this.
        Thanks for your response, I might have a look further at the HP side of things and see if any drivers or BIOS etc list issues with HYPER-V.

      • glimmerman101 said:

        I’m also considering rebuilding one of these hosts – did you ever go through this and find it was a solution or the delayed restart wasn’t enough of a hinderance to warrant it?

        What we have found the only error in the event logs:

        Cannot load a virtual machine configuration: The request is not supported. (0x80070032). (Virtual machine ID 6C3CC124-1C92-4206-9F45-D1992AE05312)

        – This has led to a bunch of dead ends at this stage looking up the above error.

        I may contact MS support and lodge a case with them also.

  2. I haven’t rebuilt any of the servers. Reboots are done during off hours so the slightly longer boot hasn’t been a problem. Tempted to remove the delayed start and see if the problem persists or maybe a more recent update has resolved.

    I just realized I installed 2019 as a host on an older *Dell* server in my office the end of February and it hasn’t been a problem……yet. Anxious to hear if you learn anything new. Thanks for your feedback.

  3. Steve, just an update. The Dell server I mentioned which didn’t have any issues started exhibiting the same problem of “No virtual machines found on this server” this morning, after a forced reboot (power outages), but on a UPS. This is 8 months after built and no updates applied for more that 12 weeks except antivirus. I confess I do not apply updates regularly on host servers.

  4. Dave Wain said:

    Hi Y’all,

    Came across this post while troubleshooting the same scenerio. We are an MSP and have quite a few 2016/2019 Hyper-V servers (all HP ProLient, mix of Gen8-10) out there. Only one in particular is experiencing this behaviour. Every Monday morning at 3:00am the host is restarted by N-Able for patching and the vms do not appear in Hyper-V management and are not started.
    I tried the delayed start on vmms to no avail.
    Have just implemented the task scheduler stop/start and will see if that works 7 days from now.
    Also added iSNS as the vm disks are on a SAN and some event errors mention this.

    Cheers.

    • Dave Wain said:

      I might add that a manual restart of the host does prove successful. Go figure!
      Also, there is a lot of mention about the symbolic links of the vms. This server only has 1 symbolic link although there are 3 vms. Can someone please explain that?

      Dave

    • Thanks for the feedback Dave. I am curious to hear if the scheduled task works. I have 5 2019 hosts and all exhibit the same behavior. They were installed over an 18 month period and it seems about 2-5 months later started doing this. Therefore, I can’t see it being related to an update. I was working on one of these today and the delayed start still seems to work fine on it.

  5. solmssen said:

    This was very helpful – many thanks. Windows 10 Pro 20H2 – I had one VM and it was missing when I started Hyper-V Manager. Restarting the Hyper-V Virtual Machine Management service instantly brought it back to the Manager – I didn’t even have to close and reopen it. Certainly gave me a little jolt when it was missing!

  6. I just had this happen this morning as well. First time I ever saw that. Little jolt? I had full cardiac arrest! 🙂 however as restarting the service brought it back, I am in much better shape now, whew!
    One thing this did force me to do was to make a copy of the VM files to another drive/folder. I also see that there is a separate location where the config files reside. Just wondering if backing all of these up is sufficient? Would an export be more appropriate?

    Regards,

    Mike.

    • Though you can never have too many backups there is no real advantage. That is not to say you should not have backup procedures. The service doesn’t start properly but the vhd/vhdx files are fully in tact, You can copy the vhd’s but you cannot copy config files or snapshots. An export will not work either as it doesn’t make a copy but rather prepares the VM to be restored elsewhere. If concerned you can take a screen image of the VM’s settings with everything expanded. Should you ever have to move or rebuild you can build a new VM with the same settings and attach those vhd files or those from your backups.

      • Thank you for the information, very insightful and helpful. It seems there is some work to do in order to properly protect a Hyper-V environment. Most likely there are 3rd party products that exist for this very purpose however as I only have 1 VM and it’s running on Windows 10; seems likely just a backup of the disk files is sufficient and then as you mentioned, create a new VM in the Manager and re-attach the disk files..

        Regards,

        Mike.

  7. If the VM is a Windows server version you can us Windows server back up, or Veeam has a free version alot of people use. If copying VHD/VHDX files as your recovery process you can’t have snapshots. You need to merge any before copying.

Leave a Reply to glimmerman101 Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Tag Cloud