Surya Susarla

Dev log - About

Migrating a Hyper-V VM to KVM — What Actually Goes Wrong

I recently moved my dev box from a Hyper-V Gen 2 VM on Windows to KVM/QEMU on Linux. On paper it sounds simple: convert the disk, define a VM in libvirt, boot it. In practice it took three days of debugging. This is a writeup of what went wrong and why, so the next person who tries this doesn't have to figure it out from scratch.

Environment: Pop!_OS host, KVM/QEMU with libvirt 8.0.0, Arch Linux guest, 40GB VHDX converted to qcow2.

The Three Things Nobody Tells You

1. There Is No Bootloader on the Disk

This one took a while to accept because it goes against how you expect bootable disks to work.

Hyper-V Gen 2 uses direct kernel boot. It doesn't look for a bootloader on the disk — it reads the kernel and initramfs directly from a partition and boots them itself. The boot configuration (which kernel, kernel flags, root partition) lives inside Hyper-V's own settings, not anywhere on the disk.

So the disk I had was legitimately bootable — just not in any way that QEMU or a normal BIOS/UEFI understands.

You can see this when you mount the disk:

sudo modprobe nbd max_part=8
sudo qemu-nbd --connect=/dev/nbd0 Arch.vhdx
sudo fdisk -l /dev/nbd0
# Disklabel type: dos  ← MBR, not GPT. No EFI System Partition.

sudo hexdump -C /dev/nbd0 | head -5
# 00000000  00 00 00 00 ...  ← MBR is all zeros. No bootloader code.

OVMF (UEFI) falls through its entire boot list — PXE, HTTP boot — and lands in the EFI interactive shell. SeaBIOS finds an empty MBR and also fails. Both are correct: there's just nothing there.

The fix: use QEMU's direct kernel boot, which is exactly what Hyper-V was doing internally, just made explicit:

# Extract the kernel and initramfs from the disk
sudo mount /dev/nbd0p1 /mnt/boot
cp /mnt/boot/vmlinuz-linux ~/vm/
cp /mnt/boot/initramfs-linux.img ~/vm/
sudo umount /mnt/boot
sudo qemu-nbd --disconnect /dev/nbd0

# Pass them directly to QEMU
qemu-system-x86_64 \
  -kernel ~/vm/vmlinuz-linux \
  -initrd ~/vm/initramfs-linux.img \
  -append "root=/dev/vda2 rw console=ttyS0" \
  ...

This is what got the VM running — and once it's up, you can treat it like any other Linux box.

2. The initramfs Has the Wrong Drivers

Even with direct kernel boot working, the system hung at startup:

A start job is running for /dev/vda2 (30s / 1min 30s)
...
Timed out waiting for device /dev/vda2

The root disk never appeared.

Here's why: the initramfs is a small RAM filesystem that loads before the real root is mounted. It has to contain the drivers needed to find and access that disk. Arch's mkinitcpio builds the initramfs using an autodetect hook that scans currently-running hardware and includes only the drivers it sees — so on Hyper-V, it baked in hv_storvsc and hv_vmbus. KVM presents storage via virtio_blk, which wasn't there.

You can verify this:

cpio -t < initramfs-linux.img | grep -i "virtio\|hv_"
# hv_vmbus.ko.zst    ← present
# hv_storvsc.ko.zst  ← present
# virtio_blk         ← missing
# virtio_pci         ← missing

The fix: rebuild the initramfs inside a chroot with the right drivers explicitly listed. You can't copy an initramfs from another machine — the module paths are kernel-version-specific.

# Mount the disk
sudo qemu-nbd --connect=/dev/nbd0 disk.vhdx
sudo mount /dev/nbd0p2 /mnt/vm-root
sudo mount /dev/nbd0p1 /mnt/vm-root/boot
sudo mount --bind /proc /mnt/vm-root/proc
sudo mount --bind /dev /mnt/vm-root/dev

# In /mnt/vm-root/etc/mkinitcpio.conf, change:
# MODULES=()
# to:
# MODULES=(virtio_pci virtio_blk)

sudo chroot /mnt/vm-root mkinitcpio -p linux
cp /mnt/vm-root/boot/initramfs-linux.img ~/vm/

sudo umount /mnt/vm-root/proc /mnt/vm-root/dev /mnt/vm-root/boot
sudo umount /mnt/vm-root
sudo qemu-nbd --disconnect /dev/nbd0

The autodetect hook is the hidden landmine here. It makes the initramfs hardware-dependent in a way that isn't obvious until you move to different hardware. Explicitly listing drivers in MODULES overrides it and makes the initramfs portable.

3. Guest Config References Hardware by Name

Even after the system booted, there was no network. This one's more of a collection of smaller issues than a single root cause.

Interface naming: Hyper-V's NIC was named eth0. KVM's virtio NIC gets a predictable name based on the slot (ens3, enp2s0, etc.). The systemd-networkd config was matching on the old name and doing nothing.

Fix: match on MAC address instead, and set a fixed MAC in libvirt so it doesn't change across reboots:

# /etc/systemd/network/20-wired.network
[Match]
MACAddress=52:54:00:12:34:56

[Network]
DHCP=yes

Avahi: /etc/avahi/avahi-daemon.conf had allow-interfaces=eth0. Avahi started, found no interface named eth0, bound to nothing, and mDNS appeared completely broken. Fix: remove the allow-interfaces line entirely.

systemd-resolved conflict: Avahi and systemd-resolved both try to own port 5353 for mDNS. Avahi backs off with a warning. If you're using avahi for mDNS, disable it in resolved:

mkdir -p /etc/systemd/resolved.conf.d
echo -e "[Resolve]\nMulticastDNS=no" > /etc/systemd/resolved.conf.d/no-mdns.conf
systemctl restart systemd-resolved avahi-daemon

Debugging Without a Network

When the VM has no network and can't be SSH'd into, the serial console is the only path. QEMU can expose it as a Unix socket:

-serial unix:/tmp/vm-serial.sock,server,nowait

The kernel needs console=ttyS0 in its cmdline, otherwise output goes to VGA only and the socket stays silent. Once that's set, connect with Python (socat and nc weren't reliable in my testing):

import socket, time
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect('/tmp/vm-serial.sock')
s.settimeout(3)
s.send(b'\r\n')
buf = b''
for _ in range(30):
    try: buf += s.recv(4096)
    except: break
    time.sleep(0.2)
print(buf.decode('utf-8', errors='replace'))
s.close()

One more gotcha: if you try to read the guest's journal from the host (journalctl --file=...), it fails if host and guest run different systemd versions:

Failed to open files: Protocol not supported

Workaround: chroot into the guest and use its own journalctl:

sudo chroot /mnt/vm-root journalctl --no-pager -b -1 -u systemd-networkd

The Disk Format

QEMU can boot VHDX directly with format=vhdx, but libvirt doesn't accept vhdx as a valid driver name. Converting to qcow2 is the clean solution:

qemu-img convert -f vhdx -O qcow2 Arch.vhdx Arch.qcow2

Do this after fixing the initramfs and network config inside the guest. The first time I tried it I converted before fixing anything, which just gave me a broken qcow2 instead of a broken VHDX.

Wrapping Up: Installing GRUB and Cleaning Up

Once the VM is running, the host-side kernel and initramfs copies can be retired. Install GRUB from inside the guest:

sudo pacman -S grub
sudo grub-install --target=i386-pc /dev/vda
sudo grub-mkconfig -o /boot/grub/grub.cfg

The MBR partition table already has a 1MB gap before the first partition (sectors 0–2047) — that's exactly where GRUB embeds its core image. grub-mkconfig picks up the kernel and initramfs from /boot automatically.

Also a good time to clean up mkinitcpio.conf. The explicit MODULES=(virtio_pci virtio_blk) added during the chroot phase can go back to MODULES=() — autodetect now runs inside the actual KVM environment and includes the right drivers on its own. Regenerate after:

sudo mkinitcpio -p linux

On the host, remove the <kernel>, <initrd>, and <cmdline> lines from the libvirt XML and reboot. SeaBIOS → GRUB → kernel. The VM is fully self-contained.

Final libvirt Setup

The final VM definition, for reference:

<domain type="kvm">
  <name>supraa-virt</name>
  <memory unit="GiB">18</memory>
  <vcpu>10</vcpu>

  <os>
    <type arch="x86_64">hvm</type>
  </os>

  <features><acpi/><apic/></features>
  <cpu mode="host-passthrough"/>

  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2" cache="none" io="native"/>
      <source file="/home/surya/vm/Arch.qcow2"/>
      <target dev="vda" bus="virtio"/>
    </disk>
    <interface type="bridge">
      <mac address="52:54:00:12:34:56"/>
      <source bridge="br0"/>
      <model type="virtio"/>
    </interface>
    <serial type="unix">
      <source mode="bind" path="/tmp/vm-serial.sock"/>
      <target type="isa-serial" port="0"/>
    </serial>
    <console type="unix">
      <source mode="bind" path="/tmp/vm-serial.sock"/>
      <target type="serial" port="0"/>
    </console>
  </devices>
</domain>

virsh define supraa-virt.xml && virsh start supraa-virt to apply. virsh autostart supraa-virt handles restarts on host reboot.

Checklist

If you're doing this migration, the order matters:

From the host (disk offline):

  1. Mount disk via NBD, inspect partition table (fdisk -l)
  2. Extract kernel and initramfs from boot partition — copy to a host directory
  3. Edit mkinitcpio.conf in the chroot: add virtio_pci virtio_blk to MODULES
  4. Chroot and regenerate initramfs: mkinitcpio -p linux, copy result to host
  5. Update network config: match by MAC, not interface name
  6. Fix avahi if present: remove allow-interfaces restriction
  7. Disable mDNS in systemd-resolved if avahi owns it
  8. Convert VHDX to qcow2: qemu-img convert -f vhdx -O qcow2
  9. Write libvirt XML with direct kernel boot (kernel/initrd/cmdline pointing at host copies)
  10. virsh define, virsh start, virsh autostart

From inside the guest (once running): 11. Install GRUB: pacman -S grub && grub-install --target=i386-pc /dev/vda && grub-mkconfig -o /boot/grub/grub.cfg 12. Clear MODULES=() in mkinitcpio.conf, regenerate initramfs: mkinitcpio -p linux 13. On the host: remove <kernel>, <initrd>, <cmdline> from the libvirt XML 14. Reboot — SeaBIOS → GRUB → kernel, fully self-contained