[fedora-virt] Cannot start 2 VMs on F21

[Jerry James]

Greetings,

I recently upgraded my work machine from F20 to F21, via fedup. Now I
cannot run two VMs at the same time, as I used to be able to do, at
least with virt-manager. If I try to start a second VM via
virt-manager, I get an error message:

------------------------------------------------------------------------------
Error starting domain: Unable to read from monitor: Connection reset by peer

Details:
Error starting domain: Unable to read from monitor: Connection reset by peer

Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 89, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 125, in tmpcb
callback(*args, **kwargs)
File "/usr/share/virt-manager/virtManager/domain.py", line 1388, in startup
self._backend.create()
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 999, in create
if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: Unable to read from monitor: Connection reset by peer

Close
------------------------------------------------------------------------------

I can start any of my VMs, if there is no other VM running. After
starting any VM, attempting to start any other VM as a second instance
triggers this failure.

I will paste the contents of /var/log/messages from the latest
attempt. In this case, qemu-Rawhide32 is the VM that was already
running, and had been running for some time, so I don't understand the
"New machine" message. I'm trying to start the qemu-CentOS7 machine.

Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): carrier is OFF
Jan 14 11:02:36 diannao kernel: [94268.464579] device vnet1 entered
promiscuous mode
Jan 14 11:02:36 diannao kernel: [94268.467198] virbr0: port 3(vnet1)
entered listening state
Jan 14 11:02:36 diannao kernel: [94268.467211] virbr0: port 3(vnet1)
entered listening state
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): new Tun
device (driver: 'unknown' ifindex: 7)
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): exported
as /org/freedesktop/NetworkManager/Devices/6
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (virbr0): bridge
port vnet1 was attached
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): enslaved to virbr0
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: unmanaged -> unavailable (reason 'connection-assumed')
[10 20 41]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): link connected
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: unavailable -> disconnected (reason
'connection-assumed') [20 30 41]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) starting connection vnet1'
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 1 of 5 (Device Prepare) scheduled...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 1 of 5 (Device Prepare) started...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: disconnected -> prepare (reason 'none') [30 40 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) scheduled...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 1 of 5 (Device Prepare) complete.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) starting...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: prepare -> config (reason 'none') [40 50 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) successful.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) complete.
Jan 14 11:02:36 diannao dbus[838]: [system] Activating via systemd:
service name='org.freedesktop.machine1'
unit='dbus-org.freedesktop.machine1.service'
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 3 of 5 (IP Configure Start) scheduled.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 3 of 5 (IP Configure Start) started...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: config -> ip-config (reason 'none') [50 70 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 3 of 5 (IP Configure Start) complete.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: ip-config -> secondaries (reason 'none') [70 90 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: secondaries -> activated (reason 'none') [90 100 0]
Jan 14 11:02:36 diannao dbus[838]: [system] Successfully activated
service 'org.freedesktop.machine1'
Jan 14 11:02:36 diannao kernel: device vnet1 entered promiscuous mode
Jan 14 11:02:36 diannao kernel: virbr0: port 3(vnet1) entered listening state
Jan 14 11:02:36 diannao kernel: virbr0: port 3(vnet1) entered listening state
Jan 14 11:02:37 diannao systemd-machined: New machine qemu-Rawhide32.
Jan 14 11:02:37 diannao systemd-machined: New machine qemu-CentOS7.
Jan 14 11:02:37 diannao NetworkManager[946]: <info> Activation
(vnet1) successful, device activated.
Jan 14 11:02:37 diannao dbus[838]: [system] Activating via systemd:
service name='org.freedesktop.nm_dispatcher'
unit='dbus-org.freedesktop.nm-dispatcher.service'
Jan 14 11:02:37 diannao dbus[838]: [system] Successfully activated
service 'org.freedesktop.nm_dispatcher'
Jan 14 11:02:37 diannao nm-dispatcher: Dispatching action 'up' for vnet1
Jan 14 11:02:37 diannao systemd: Unit iscsi.service cannot be reloaded
because it is inactive.
Jan 14 11:02:37 diannao systemd: Unit named.service cannot be reloaded
because it is inactive.
Jan 14 11:02:37 diannao kernel: [94269.394707] qemu-system-x86:
sending ioctl 5326 to a partition!
Jan 14 11:02:37 diannao kernel: qemu-system-x86: sending ioctl 5326 to
a partition!
Jan 14 11:02:37 diannao kernel: qemu-system-x86: sending ioctl
80200204 to a partition!
Jan 14 11:02:37 diannao kernel: [94269.394737] qemu-system-x86:
sending ioctl 80200204 to a partition!
Jan 14 11:02:37 diannao kernel: [94269.448193] virbr0: port 3(vnet1)
entered disabled state
Jan 14 11:02:37 diannao kernel: virbr0: port 3(vnet1) entered disabled state
Jan 14 11:02:37 diannao kernel: device vnet1 left promiscuous mode
Jan 14 11:02:37 diannao kernel: [94269.448871] device vnet1 left
promiscuous mode
Jan 14 11:02:37 diannao kernel: [94269.448894] virbr0: port 3(vnet1)
entered disabled state
Jan 14 11:02:37 diannao kernel: virbr0: port 3(vnet1) entered disabled state
Jan 14 11:02:37 diannao NetworkManager[946]: <info> (vnet1): device
state change: activated -> unmanaged (reason 'removed') [100 10 36]
Jan 14 11:02:37 diannao NetworkManager[946]: <info> (vnet1):
deactivating device (reason 'removed') [36]
Jan 14 11:02:37 diannao journal: Unable to read from monitor:
Connection reset by peer
Jan 14 11:02:37 diannao NetworkManager[946]: <warn> (virbr0): failed
to detach bridge port vnet1
Jan 14 11:02:37 diannao NetworkManager[946]: nm_device_get_iface:
assertion 'self != NULL' failed
Jan 14 11:02:37 diannao NetworkManager[946]: <info> (vnet1): released
from master (null)
Jan 14 11:02:37 diannao systemd-machined: Machine qemu-CentOS7 terminated.
Jan 14 11:02:37 diannao journal: error from service: TerminateMachine:
No machine 'qemu-CentOS7' known
Jan 14 11:02:38 diannao nm-dispatcher: Dispatching action 'down' for vnet1
Jan 14 11:02:38 diannao gnome-session: Gjs-Message: JS LOG: Removing a
network device that was not added
Jan 14 11:02:38 diannao systemd: Unit named.service cannot be reloaded
because it is inactive.


It looks like there is some kind of trouble setting up the network for
the second VM. Does anybody have any suggestions for things I could
try to resolve this? Are there known issues with upgrading from F20
to F21? Thank you,

[Cole Robinson]

I asked the question below on the virt mailing list about 2 weeks ago.
There have been a grand total of zero responses so far. I'll ask the
same question here, in hopes that somebody knows something that could
help.

Post by Jerry James
Greetings,
I recently upgraded my work machine from F20 to F21, via fedup. Now I
cannot run two VMs at the same time, as I used to be able to do, at
least with virt-manager. If I try to start a second VM via
------------------------------------------------------------------------------
Error starting domain: Unable to read from monitor: Connection reset by peer
Error starting domain: Unable to read from monitor: Connection reset by peer
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 89, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 125, in tmpcb
callback(*args, **kwargs)
File "/usr/share/virt-manager/virtManager/domain.py", line 1388, in startup
self._backend.create()
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 999, in create
if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: Unable to read from monitor: Connection reset by peer
Close
------------------------------------------------------------------------------
I can start any of my VMs, if there is no other VM running. After
starting any VM, attempting to start any other VM as a second instance
triggers this failure.
I will paste the contents of /var/log/messages from the latest
attempt. In this case, qemu-Rawhide32 is the VM that was already
running, and had been running for some time, so I don't understand the
"New machine" message. I'm trying to start the qemu-CentOS7 machine.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): carrier is OFF
Jan 14 11:02:36 diannao kernel: [94268.464579] device vnet1 entered
promiscuous mode
Jan 14 11:02:36 diannao kernel: [94268.467198] virbr0: port 3(vnet1)
entered listening state
Jan 14 11:02:36 diannao kernel: [94268.467211] virbr0: port 3(vnet1)
entered listening state
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): new Tun
device (driver: 'unknown' ifindex: 7)
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): exported
as /org/freedesktop/NetworkManager/Devices/6
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (virbr0): bridge
port vnet1 was attached
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): enslaved to virbr0
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: unmanaged -> unavailable (reason 'connection-assumed')
[10 20 41]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): link connected
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: unavailable -> disconnected (reason
'connection-assumed') [20 30 41]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) starting connection vnet1'
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 1 of 5 (Device Prepare) scheduled...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 1 of 5 (Device Prepare) started...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: disconnected -> prepare (reason 'none') [30 40 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) scheduled...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 1 of 5 (Device Prepare) complete.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) starting...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: prepare -> config (reason 'none') [40 50 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) successful.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 2 of 5 (Device Configure) complete.
service name='org.freedesktop.machine1'
unit='dbus-org.freedesktop.machine1.service'
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 3 of 5 (IP Configure Start) scheduled.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 3 of 5 (IP Configure Start) started...
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: config -> ip-config (reason 'none') [50 70 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> Activation
(vnet1) Stage 3 of 5 (IP Configure Start) complete.
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: ip-config -> secondaries (reason 'none') [70 90 0]
Jan 14 11:02:36 diannao NetworkManager[946]: <info> (vnet1): device
state change: secondaries -> activated (reason 'none') [90 100 0]
Jan 14 11:02:36 diannao dbus[838]: [system] Successfully activated
service 'org.freedesktop.machine1'
Jan 14 11:02:36 diannao kernel: device vnet1 entered promiscuous mode
Jan 14 11:02:36 diannao kernel: virbr0: port 3(vnet1) entered listening state
Jan 14 11:02:36 diannao kernel: virbr0: port 3(vnet1) entered listening state
Jan 14 11:02:37 diannao systemd-machined: New machine qemu-Rawhide32.
Jan 14 11:02:37 diannao systemd-machined: New machine qemu-CentOS7.
Jan 14 11:02:37 diannao NetworkManager[946]: <info> Activation
(vnet1) successful, device activated.
service name='org.freedesktop.nm_dispatcher'
unit='dbus-org.freedesktop.nm-dispatcher.service'
Jan 14 11:02:37 diannao dbus[838]: [system] Successfully activated
service 'org.freedesktop.nm_dispatcher'
Jan 14 11:02:37 diannao nm-dispatcher: Dispatching action 'up' for vnet1
Jan 14 11:02:37 diannao systemd: Unit iscsi.service cannot be reloaded
because it is inactive.
Jan 14 11:02:37 diannao systemd: Unit named.service cannot be reloaded
because it is inactive.
sending ioctl 5326 to a partition!
Jan 14 11:02:37 diannao kernel: qemu-system-x86: sending ioctl 5326 to
a partition!
Jan 14 11:02:37 diannao kernel: qemu-system-x86: sending ioctl
80200204 to a partition!
sending ioctl 80200204 to a partition!
Jan 14 11:02:37 diannao kernel: [94269.448193] virbr0: port 3(vnet1)
entered disabled state
Jan 14 11:02:37 diannao kernel: virbr0: port 3(vnet1) entered disabled state
Jan 14 11:02:37 diannao kernel: device vnet1 left promiscuous mode
Jan 14 11:02:37 diannao kernel: [94269.448871] device vnet1 left
promiscuous mode
Jan 14 11:02:37 diannao kernel: [94269.448894] virbr0: port 3(vnet1)
entered disabled state
Jan 14 11:02:37 diannao kernel: virbr0: port 3(vnet1) entered disabled state
Jan 14 11:02:37 diannao NetworkManager[946]: <info> (vnet1): device
state change: activated -> unmanaged (reason 'removed') [100 10 36]
deactivating device (reason 'removed') [36]
Connection reset by peer
Jan 14 11:02:37 diannao NetworkManager[946]: <warn> (virbr0): failed
to detach bridge port vnet1
assertion 'self != NULL' failed
Jan 14 11:02:37 diannao NetworkManager[946]: <info> (vnet1): released
from master (null)
Jan 14 11:02:37 diannao systemd-machined: Machine qemu-CentOS7 terminated.
No machine 'qemu-CentOS7' known
Jan 14 11:02:38 diannao nm-dispatcher: Dispatching action 'down' for vnet1
Jan 14 11:02:38 diannao gnome-session: Gjs-Message: JS LOG: Removing a
network device that was not added
Jan 14 11:02:38 diannao systemd: Unit named.service cannot be reloaded
because it is inactive.
It looks like there is some kind of trouble setting up the network for
the second VM. Does anybody have any suggestions for things I could
try to resolve this? Are there known issues with upgrading from F20
to F21? Thank you,

Please file a bug against libvirt, and include:

1) sudo virsh dumpxml $vmname
2) /var/log/libvirt/qemu/$vmname.log

for the failing VM

- Cole

[Jerry James]

Post by Cole Robinson
1) sudo virsh dumpxml $vmname
2) /var/log/libvirt/qemu/$vmname.log
for the failing VM

Filed as https://bugzilla.redhat.com/show_bug.cgi?id=1186425. Thank you, Cole!