The driver vanishes if I install other devices like GPUs etc.

I suppose the driver has vanished because of a rollback not because of devices installation (as subject says).
I also suppose that VirtualMIDISynth.dll libs & files were deleted too.

In my opinion the better way to fix it is a reinstall (it preserves all of your settings if VirtualMIDISynth.conf file still exists).

Anyway, Windows stores installed multimedia devices under these 2 registry keys (the first one only exists on x64 systems):
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Drivers32
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Drivers32

You could export these keys content to a .reg file using regedit.exe, then double click that file to restore all of your multimedia devices.