| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-16 | |||
| 16:32:19 | bauzas | #link https://bugs.launchpad.net/nova/+bug/1950310/comments/8 explaining the guest kernel panic related to stable/victoria branch | |
| 16:32:22 | sean-k-mooney | ya the fiew cases i looked at with you last week were all happing befoer detach | |
| 16:32:38 | sean-k-mooney | so its either the attach or live migration | |
| 16:32:44 | gibi | sean-k-mooney: I have more logs in the runs of https://review.opendev.org/c/openstack/nova/+/817564 if you are interested | |
| 16:32:52 | sean-k-mooney | i looked downstream at our qemu bugs but didnt see anythign relevent | |
| 16:33:09 | sean-k-mooney | gibi: sure ill try and take a look proably tomorrow | |
| 16:33:15 | sean-k-mooney | but ill open it in a tab | |
| 16:33:57 | gibi | sean-k-mooney: thanks, I will retrigger that patch for a couple times to see if the current sleep before the live migration helps | |
| 16:34:24 | bauzas | a good sleep always helps | |
| 16:34:39 | bauzas | :) | |
| 16:34:45 | elodilles | :] | |
| 16:34:46 | sean-k-mooney | when sleep does not work we can also try a trusty print statement | |
| 16:35:09 | gibi | sleep is not there as a solution but as a troubleshooting to see which step we are too fast :D | |
| 16:35:33 | gibi | and I do have a lot of print(server.console) like statements in the tempest :D | |
| 16:36:10 | sean-k-mooney | i think we can move on but its good you were able to confirm we were attaching before the kerenl finished booting | |
| 16:36:22 | sean-k-mooney | at least in some cases | |
| 16:36:57 | sean-k-mooney | that at least lend weight to the idea we are racing | |
| 16:37:05 | bauzas | ok, let's move on | |
| 16:37:12 | gibi | ack | |
| 16:37:12 | bauzas | again, large agenda todayu | |
| 16:37:17 | bauzas | #topic Sub/related team Highlights | |
| 16:37:23 | bauzas | damn | |
| 16:37:24 | bauzas | #topic Sub/related team Highlights | |
| 16:37:40 | bauzas | Libvirt : lyarwood ? | |
| 16:38:15 | bauzas | I guess nothing to tell | |
| 16:38:19 | bauzas | moving on to the last topic | |
| 16:38:31 | bauzas | #topic Open discussion | |
| 16:39:02 | bauzas | whoami-rajat: please queue | |
| 16:39:09 | whoami-rajat | thanks! | |
| 16:39:19 | bauzas | (kashyapc) Blueprint for review: "Switch to 'virtio' as the default display device" -- https://blueprints.launchpad.net/nova/+spec/virtio-as-default-display-device | |
| 16:39:27 | bauzas | this is a specless bp ask | |
| 16:39:37 | bauzas | kashyap said " The full rationale is in the blueprint; in short: "cirrus" display device has many limitations and is "considered harmful"[1] by QEMU graphics maintainers since 2014." | |
| 16:40:00 | bauzas | do we need a spec for this bp or are we OK for approving it by now ? | |
| 16:40:19 | whoami-rajat | so lyarwood had a concern with my reimage spec, we agreed to pass the connector info to reimage API (cinder) and cinder will do attachment update and return the connection info with events payload | |
| 16:40:22 | gibi | I think we don't need a spec this is pretty self contained in the libvirt driver | |
| 16:40:23 | bauzas | kashyap was unable to attend the meeting today | |
| 16:40:27 | whoami-rajat | (in PTG) | |
| 16:40:38 | sean-k-mooney | i think we are ok with approving it the main thing to call out is we will be chaing it for existing isntnace too | |
| 16:40:39 | bauzas | whoami-rajat: please hold, sorry | |
| 16:40:43 | whoami-rajat | oh ok | |
| 16:40:45 | gibi | the only open question we had with sean-k-mooney is how to change the default | |
| 16:41:05 | gibi | but kashyap tested it out that changing the default during hard reboot not cause any trouble to guests | |
| 16:41:16 | gibi | as the new video dev has a fallback vga mode | |
| 16:41:17 | bauzas | gibi: I'm thinking hard of any potential upgrade implication | |
| 16:41:32 | sean-k-mooney | right so when we dicussed this before we decied to change it only for new instances to avoid upgrade issue | |
| 16:41:50 | bauzas | correct | |
| 16:41:57 | sean-k-mooney | our downstream QE tested this with windows guests and linux guest and both seamd to be ok with the change | |
| 16:42:01 | bauzas | I'm in favor of not touching the running instances | |
| 16:42:11 | bauzas | or asking to rebuild them | |
| 16:42:18 | gibi | we are not toching the running instance, we only touch hard rebooting instances | |
| 16:42:22 | sean-k-mooney | so kasyap has impletne this for all instnaces | |
| 16:42:49 | bauzas | gibi: which happens when you stop/start, right? | |
| 16:42:54 | gibi | right | |
| 16:42:54 | sean-k-mooney | bauzas: yes as gibi says it will only take effect when the xml is next regenreted | |
| 16:42:59 | gibi | it happens while the guest is not running | |
| 16:43:21 | gibi | it is not an unplug/plug for a running guest | |
| 16:43:29 | bauzas | do we want admins to opt-in instances ? | |
| 16:43:41 | bauzas | or do we agree it would be done automatically? | |
| 16:43:42 | sean-k-mooney | it will happen on start/stop hard reboot or a non live move operations | |
| 16:44:03 | gibi | bauzas: I trust kashyap that it is safe to change this device | |
| 16:44:14 | bauzas | do we also want to have a nova-status upgrade check for yoga about this ? | |
| 16:44:23 | sean-k-mooney | no | |
| 16:44:24 | bauzas | gibi: me too | |
| 16:44:29 | sean-k-mooney | why would we need too | |
| 16:44:35 | sean-k-mooney | we are not removing support for cirrus | |
| 16:44:36 | gibi | we don't remove cirros | |
| 16:44:41 | sean-k-mooney | jsut not the default | |
| 16:44:46 | gibi | yepp | |
| 16:45:00 | sean-k-mooney | gibi: context is downstream it is being remvoed form rhel 9 | |
| 16:45:04 | bauzas | sean-k-mooney: sure, that just means that long-living instances could continue running cirros | |
| 16:45:14 | sean-k-mooney | so wwe need to care about it for our product | |
| 16:45:24 | sean-k-mooney | actully cirrus is not beeing remvoed in rhel 9 | |
| 16:45:35 | sean-k-mooney | but like in rhel 10 | |
| 16:46:06 | sean-k-mooney | bauzas: yep which i think is ok | |
| 16:46:27 | sean-k-mooney | we coudl have a nova status check but it woudl have to run on the compute nodes | |
| 16:46:35 | sean-k-mooney | which is kind of not nice | |
| 16:46:40 | sean-k-mooney | since it woudl have to check the xmls | |
| 16:46:51 | bauzas | I know | |
| 16:46:53 | sean-k-mooney | so i woudl not add it personally | |
| 16:47:19 | bauzas | I'm just saying that we enter a time that could last long | |
| 16:47:20 | gibi | I agree, we don't need upgrade check | |
| 16:48:16 | sean-k-mooney | shal we continue this in the patch review | |
| 16:48:17 | bauzas | but agreed on the fact this is not a problem until cirros support is removed and this is not an upstream question | |
| 16:48:31 | bauzas | sean-k-mooney: you're right, nothing needing a spec | |
| 16:49:06 | bauzas | #agreed https://blueprints.launchpad.net/nova/+spec/virtio-as-default-display-device is accepted as specless BP for the Yoga release timeframe | |
| 16:49:10 | bauzas | moving on | |
| 16:49:12 | gibi | \o/ | |
| 16:49:17 | bauzas | next item | |
| 16:49:30 | bauzas | (kashyapc) Blueprint for review: "Add ability to control the memory used by fully emulated QEMU guests -- https://blueprints.launchpad.net/nova/+spec/control-qemu-tb-cache | |
| 16:49:39 | bauzas | again, a specless bp ask | |
| 16:49:54 | bauzas | he said " This blueprint allows us to configure how much memory a plain-emulated (TCG) VM, which is what OpenStack CI uses. Recently, QEMU changed the default memory used by TCG VMs to be much higher, thus reducing the no. of VMs you TCG could run per host. Note: the libvirt patch required for this will be in libvirt-v7.10.0 (December 2021)." | |
| 16:49:59 | bauzas | " See this issue for more details: https://gitlab.com/qemu-project/qemu/-/issues/693 (Qemu increased memory usage with TCG)" | |
| 16:50:21 | sean-k-mooney | im a little torn on this | |
| 16:50:41 | sean-k-mooney | im not sure i like this being a per host config option | |
| 16:50:51 | sean-k-mooney | but its also breaking existing deployemnts | |
| 16:51:04 | sean-k-mooney | so we cant really adress that with flavor extra specs or iamge properties | |
| 16:51:17 | sean-k-mooney | sicne it would be a pain for operators to use | |
| 16:51:17 | gibi | but that requires rebuild of existing instances | |
| 16:51:21 | sean-k-mooney | yep | |
| 16:51:32 | sean-k-mooney | so with that in mind the config option proably is the way to go | |