| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-08-20 | |||
| 10:17:57 | artom | sean-k-mooney, yeah, you'll have to explain the multiple port bindings thing when I'm more awake | |
| 10:53:57 | stephenfin | luyao: Reviewed that series. Let me know if anything doesn't make sense | |
| 11:46:49 | admin0 | lyarwood, i have narrowed down my ceph issue, when i try to manually mount it, its asking me to disable object-map fast-diff deep-flatten | |
| 11:47:04 | admin0 | trying to do rbd map --pool vms --id cinder --conf /etc/ceph/ceph.conf $uuid_disk from the hypervsior directly | |
| 11:47:36 | lyarwood | admin0: you shouldn't need to mount it to export volumes | |
| 11:48:01 | lyarwood | admin0: rbd can export directly from the backend to a local file | |
| 11:48:02 | admin0 | my issue is when using raw, the instances do not boot up .. they are stuck in boot .. | |
| 11:48:13 | admin0 | even after a reformat, the behaviour is like that | |
| 11:48:37 | lyarwood | admin0: right, as we spoke about the other day the rbd import (not export sorry) command was stalling right? | |
| 11:55:12 | frickler | admin0: that sounds like you are trying to use rbd with an old kernel module, but I don't understand where this is needed, usually qemu should handle the rbd connection without kernel support | |
| 11:55:33 | frickler | but maybe you have a mismatch in ceph versions between your cluster and the hypervisor? | |
| 11:58:24 | admin0 | ceph is on 15.x on centos, clinet is 14.x on ubuntu | |
| 12:01:42 | lyarwood | admin0: did you stop using config drives? | |
| 12:01:48 | admin0 | yep | |
| 12:01:57 | lyarwood | admin0: and where is it failing now? | |
| 12:02:04 | admin0 | by stopping config drives, they can at least go active and show the console | |
| 12:03:25 | lyarwood | admin0: and by show in the console are you saying the instance boots as expected? | |
| 12:04:35 | admin0 | lyarwood, i have to run to a meeting now .. i will get back on this later | |
| 12:04:43 | admin0 | i think its the hypervisor -> osd connectivity that i think is the issue | |
| 12:27:52 | sean-k-mooney | admin0: did you figure out your nova-mange issue you are using openstack-helm they are configuring cells incorrectly | |
| 12:40:43 | admin0 | it was a network routing issue between hypervisor <-> osd | |
| 12:45:15 | sean-k-mooney | ok | |
| 12:46:07 | sean-k-mooney | there was someone else asking the same question later the same day that was cuased by helml not actully generig the correct config and not handeling the case wehre you go form a non tls deploymet to a tls deployment | |
| 12:46:12 | sean-k-mooney | as a result | |
| 12:46:28 | sean-k-mooney | admin0: i was jsut wondering if you were hitting the same issue. | |
| 12:48:46 | openstackgerrit | Merged openstack/nova master: zuul: use the new barbican simple-crypto job https://review.opendev.org/746583 | |
| 13:29:53 | gmann | stephenfin: np!, thank you too for re-reviewing policy patches. | |
| 13:30:00 | stephenfin | yw :) | |
| 13:38:54 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/stein: libvirt: Use 'writeback' QEMU cache mode when 'none' is not viable https://review.opendev.org/747163 | |
| 13:44:36 | openstackgerrit | Mohammed Naser proposed openstack/nova master: db: fix database migrations when name includes dash https://review.opendev.org/747164 | |
| 13:45:29 | openstackgerrit | Mohammed Naser proposed openstack/nova master: db: fix database migrations when name includes dash https://review.opendev.org/747164 | |
| 13:45:30 | mnaser | ^ very trivial, honestly, i don't know how we'd test it | |
| 14:00:19 | stephenfin | alex_xu: I think there are a whole class of bugs here to do with races between setting the migration status to foo and dropping a move claim | |
| 14:00:26 | stephenfin | Thanks for the rabbit hole \o/ :P | |
| 14:06:02 | kashyap | lyarwood: Or anyone who recalls stable rules (my mind is ill right now, can't recall), does the above 'writeback' backport (https://review.opendev.org/#/c/747163/) look okay? | |
| 14:13:10 | lyarwood | kashyap: we would retain the existing cachemodes during a LM right? | |
| 14:13:34 | kashyap | lyarwood: Correct, until the instance is cold-rebooted on the destination. | |
| 14:13:40 | lyarwood | kashyap: if it doesn't impact running instances during LM then I'd be okay with this assuming it's now doing the correct thing | |
| 14:14:28 | lyarwood | kashyap: I'll review it in detail later | |
| 14:14:33 | kashyap | Yeah, it does the right thing now. I've spent several days extensively documenting it | |
| 14:14:48 | kashyap | lyarwood: No rush at all for it; just noticed only when Vlad (above) posted it | |
| 14:32:41 | artom | sean-k-mooney, whenever you're around, wanna talk about the multiple vif bindings thing? | |
| 14:33:03 | sean-k-mooney | artom: am sure want to do it on video chat or here | |
| 14:33:08 | sean-k-mooney | i want to grab coffee first | |
| 14:33:11 | sean-k-mooney | in either case | |
| 14:33:24 | sean-k-mooney | but im more or less free so say in 10 mins | |
| 14:33:40 | sean-k-mooney | our quater to the hour | |
| 14:34:03 | artom | sean-k-mooney, so :45? OK | |
| 14:34:17 | sean-k-mooney | yes | |
| 14:34:23 | sean-k-mooney | that is what i ment by quater to the hour | |
| 14:34:40 | sean-k-mooney | i debated if :45 would translate well | |
| 14:42:04 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/ussuri: replace the "hide_hypervisor_id" to "hw:hide_hypervisor_id" https://review.opendev.org/747189 | |
| 14:42:52 | sean-k-mooney | i guess that is valid to backport | |
| 14:43:38 | sean-k-mooney | its techncially adding a new extra spec which we would allow downtream but its just an alias for an existing one that is broken/problematic | |
| 14:43:53 | sean-k-mooney | so its fixing a bug without really providing new functionality | |
| 14:45:33 | artom | sean-k-mooney, err, give me 15 minutes? Brb | |
| 14:50:49 | sean-k-mooney | ping me when it suits you | |
| 15:02:11 | sean-k-mooney | lyarwood: when through your libvirt version change series more less teh look correct but the cloud archive is not being used currently so you need your devstack change for them to pass | |
| 15:02:34 | sean-k-mooney | so stick a depends-on in https://review.opendev.org/#/c/746981/1 and rebase the reset | |
| 15:03:43 | lyarwood | sean-k-mooney: yup going to work through that again later | |
| 15:04:27 | sean-k-mooney | most of it looks sane and your deleteing lots of code whcih si always nice to see | |
| 15:05:12 | sean-k-mooney | we might want to send a note to thrid part ci maintainer on the mailing list too | |
| 15:05:25 | sean-k-mooney | since they will need to make sure they have the new min version avaiable too | |
| 15:07:09 | artom | sean-k-mooney, ok, back, ready when you are | |
| 15:08:51 | sean-k-mooney | ah finishing an email but more or less one sec | |
| 15:11:02 | sean-k-mooney | ok sent | |
| 15:11:41 | sean-k-mooney | what to chat on google meet | |
| 15:11:51 | sean-k-mooney | or here | |
| 15:12:42 | artom | sean-k-mooney, meet | |
| 15:13:19 | artom | meet.google.com/zzq-favq-pxj | |
| 15:50:58 | lyarwood | Meeting in #openstack-meeting-3 in ~10mins | |
| 15:51:04 | lyarwood | I'll keep it short again this week :) | |
| 16:00:02 | lyarwood | Starting the meeting shortly | |
| 18:53:35 | artom | sean-k-mooney, you'll probably also have to fiddle with the migrate_data.vifs stuff in nova/conductor/tasks/live_migrate.py for your eventual fix, btw | |
| 19:07:45 | artom | Also, writing this commit message is a major PITA | |
| #openstack-nova - 2020-08-21 | |||
| 00:30:34 | openstackgerrit | sean mooney proposed openstack/nova master: [WIP] Set migrate_data.vifs only when using multiple port bindings https://review.opendev.org/742180 | |
| 02:45:02 | openstackgerrit | Merged openstack/nova master: Pass the actual target in baremetal nodes policy https://review.opendev.org/741717 | |
| 02:45:13 | openstackgerrit | Merged openstack/nova master: Add scope and new default roles in extensions policies https://review.opendev.org/743046 | |
| 02:45:24 | openstackgerrit | Merged openstack/nova master: Add new default roles in networks policies https://review.opendev.org/742566 | |
| 02:45:31 | openstackgerrit | Merged openstack/nova master: Pass the actual target in networks policy https://review.opendev.org/742568 | |
| 02:45:38 | openstackgerrit | Merged openstack/nova master: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0 https://review.opendev.org/746210 | |
| 08:23:51 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/ussuri: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0 https://review.opendev.org/747357 | |
| 08:27:11 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0 https://review.opendev.org/747358 | |
| 08:49:59 | alex_xu | stephenfin: yea, guess, the db status and rt isn't update under same lock | |
| 08:50:07 | alex_xu | s/guess/guess so/ | |
| 08:50:47 | stephenfin | alex_xu: The ResourceTracker is so broken :'( | |
| 08:51:02 | stephenfin | Spent all day merely trying to figure out how broken it is, heh | |
| 08:51:14 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0 https://review.opendev.org/747360 | |
| 08:51:17 | alex_xu | hah | |
| 08:52:44 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/rocky: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0 https://review.opendev.org/747361 | |
| 08:55:17 | openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: libvirt: Do not reference VIR_ERR_DEVICE_MISSING when libvirt is < v4.1.0 https://review.opendev.org/747362 | |
| 08:56:22 | lyarwood | elod_off: https://review.opendev.org/#/q/topic:bug/1887946+status:open - When you're back this series will need some stable reviews :) | |
| 10:06:37 | openstackgerrit | Merged openstack/nova master: functional: Move single-use function to its caller https://review.opendev.org/745904 | |
| 10:06:43 | openstackgerrit | Merged openstack/nova master: functional: Drop '_api' suffix from placement fixture https://review.opendev.org/745905 | |
| 11:04:00 | lyarwood | https://github.com/openstack/nova/blob/835440e3f9dace4074511036473ebdae22f72c25/nova/context.py#L268-L279 - does anyone know if there's a way to get an admin context with a populated service catalog? | |
| 11:05:23 | lyarwood | ^ at the moment we are using this during self.driver.resume_state_on_host_boot and failing when attempting to hard reboot instances with encrypted volumes as the key_manager service isn't listed in the service catalog | |
| 11:06:07 | lyarwood | appreciate that even if it was the admin context wouldn't be able to fetch user secrets without changing policy but I'd still like to fix this if possible | |
| 11:49:32 | artom | sean-k-mooney, well my VPN just died, but https://github.com/openstack/nova/blob/master/nova/virt/libvirt/migration.py#L96 | |
| 11:50:07 | artom | And https://github.com/openstack/nova/blob/master/nova/virt/libvirt/migration.py#L144 | |
| 12:40:01 | sean-k-mooney | artom that only does it for numa migration | |