Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-11-23
11:26:53 opendevreview Merged openstack/nova master: db: Replace use of legacy select() calling style https://review.opendev.org/c/openstack/nova/+/805735
11:34:03 lyarwood bauzas: do we need a bp per spec? It appears to graph them out correctly in the bp at least
11:35:25 opendevreview Merged openstack/nova master: db: Replace 'insert.inline' parameter with 'Insert.inline()' method https://review.opendev.org/c/openstack/nova/+/805736
11:35:32 opendevreview Merged openstack/nova master: db: Don't pass strings to 'Connection.execute' https://review.opendev.org/c/openstack/nova/+/805737
13:09:46 jhartkopf Hi there, I'd like to add another topic to the open discussion section of today's Nova meeting, but it seems to be quite full already. Would you recommend to add the topic to next week's meeting instead? And how can I add a topic to the list? Should I just edit the Wiki page?
13:14:34 sean-k-mooney you can just add it to the adgenda
13:14:45 sean-k-mooney just edit the wiki
13:14:48 sean-k-mooney what is the topic
13:15:50 sean-k-mooney lyarwood: normally yes we have a blueprint per spec, you can model the depencies between the blueprints
13:16:25 sean-k-mooney lyarwood: i think that works across projects too but not tried that in a while so not 100% sure
13:16:50 sean-k-mooney lyarwood: im pretty sure i have linke a nova blueprint to a neutron one at some point
13:22:44 jhartkopf sean-k-mooney: Alright, thanks. The topic is the spec for updating user data. We've already discussed it on OpenDev but I'd like to bring up some specific use cases once again.
13:23:02 sean-k-mooney ack
13:28:40 opendevreview Lee Yarwood proposed openstack/nova-specs master: fup: Correct the libvirt ephemeral encryption blueprint link https://review.opendev.org/c/openstack/nova-specs/+/818917
13:29:07 lyarwood sean-k-mooney: right thanks I thought the single bp showed the spec relationship but it looks like you created another bp previously
13:29:32 lyarwood updated the spec to point to that second bp, iirc gibi was cool with one bp previously but I really don't mind either way
13:30:06 sean-k-mooney ya im fine with both where it was useful in the past was vhost-user
13:30:23 sean-k-mooney we had one blueprint for generic vhost-user support and a second for vhost-user with ovs-dpdk
13:30:57 sean-k-mooney we happend to enable both in the same release but if the ovs-dpdk supprot had slipped we could have closed the generic one and move the ovs one
13:31:17 sean-k-mooney we rarely have 2 feature that share a common part like that
13:31:37 sean-k-mooney at least rarely in the same cycle
13:34:05 sean-k-mooney lyarwood: actully one minor reason to have 2 bluepritns is the script we use to mark specs as implemented and move them uses the script file name to determin which blueprint to query
13:34:22 sean-k-mooney lyarwood: i thinkthat is why i originally created the second libvirt one
13:37:07 EugenMayer is it a known issue that ubuntu (20.04) does not accept the DNS server pushed by DHCP openstack (OVN)? Debian generic cloud image does work though
13:38:37 sean-k-mooney EugenMayer: thats odd i would have expected that to work
13:39:00 sean-k-mooney you set the nameservers on the subnet as normal
13:39:11 EugenMayer Same does me. Spinnging up a debian box right now, expecting it to work (since all the debian boxes worked)
13:39:14 sean-k-mooney and it just does not pic those up via dhcp?
13:39:16 EugenMayer yes i do
13:39:29 EugenMayer it ignores it on ubuntu, exactly
13:39:39 sean-k-mooney ya that sound like a dhcpclint bug in ubuntu 20.04
13:40:07 sean-k-mooney i assume they are both useing the came dhcp clinet implemenation by default
13:40:13 EugenMayer yes debian works. Same network same everything (using terraform here, so just replaced the image)
13:41:00 sean-k-mooney perhap its fixed in a later cloud image? im not sure if you are using the latest point release image of 20.04 but might be worth trying the nightly image
13:41:09 EugenMayer using the latest one
13:41:13 EugenMayer not older then 5 days
13:41:24 EugenMayer did you find a bug reeport?
13:41:37 sean-k-mooney no have not look
13:42:03 EugenMayer i look it up and try > 20.xx
13:42:10 sean-k-mooney it used to work i think but that the only thing i can think of is a bug in the dhcpclient or the cloud-init default behavior in ubuntu
13:42:26 EugenMayer maybe it is netplan related
13:43:37 EugenMayer i use what is referenced at Same does me. Spinnging up a debian box right now, expecting it to work about
13:43:40 EugenMayer sorry
13:43:45 EugenMayer here https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/configure-openstack.html#set-up-public-networking
13:49:46 EugenMayer interesting, resolvectl status shows up the right dns serevers... but the resolution of my internal domains does not work
13:49:57 EugenMayer i have to use 'dig @IP domain'
13:56:30 bauzas lyarwood: oh, sorry, I haven't seen https://blueprints.launchpad.net/nova/+spec/ephemeral-encryption-libvirt
13:56:41 bauzas lyarwood: then we need to update the spec to tell that the BP is ^
13:56:51 bauzas lyarwood: I mean, in https://specs.openstack.org/openstack/nova-specs/specs/yoga/approved/ephemeral-encryption-libvirt.html
13:57:32 lyarwood bauzas: yeah there's a fup above for that now
13:59:30 lyarwood https://review.opendev.org/c/openstack/nova-specs/+/818917 sorry
14:03:15 EugenMayer sean-k-mooney ok i found it. Damn i confused the isolated test. I had a typo in my CNAME thus the isolated test with a blank 20.04 box failed and i assumed a general issues. Now vanilla focal works. just focal + rke2 has the wrong DNS servers, which might be an RKE2 setup thing. Sorry for wasting time, got dragged away, missed that typo
14:04:01 sean-k-mooney EugenMayer: no worries
14:19:40 opendevreview Merged openstack/nova-specs master: fup: Correct the libvirt ephemeral encryption blueprint link https://review.opendev.org/c/openstack/nova-specs/+/818917
14:33:11 bauzas eeeeek https://github.com/libvirt/libvirt/commit/3bd8181bc5548a0ce81107cbfb480dfdcba5679d
14:33:17 bauzas sean-k-mooney ^
14:34:13 mnaser gibi: yeah i saw your comments and found that issue, it seems like even with retry=0, it still is kinda failing
14:34:20 mnaser since it gets stuck in a reconnecting loop
14:34:27 bauzas sean-k-mooney: context is https://bugs.launchpad.net/nova/+bug/1951656
14:34:51 gibi mnaser: yepp the retry config is only used to message sending but not connection setup
14:35:16 mnaser that's where i ended up and it felt pretty non-trivial at that point to go from there :(
14:35:34 mnaser at least, it's beyond my scope of comfortable knowledge anyways
14:35:38 gibi mnaser: it seems the current oslo.messaging driver interface does not have way to express that the consumer wants configurable retry for conenction setup
14:36:03 sean-k-mooney bauzas: we proably shoudl jsut not use libvirt for mdevs at all honestly but fun
14:36:09 gibi mnaser: I'm still digging oslo.messaging to understand more how to fit this in
14:36:32 bauzas sean-k-mooney: well, I'd prefer the other way, ie. asking libvirt to create mdevs
14:36:41 mnaser gibi: something i found interesting was - https://docs.celeryproject.org/projects/kombu/en/stable/_modules/kombu/connection.html#Connection.ensure_connection
14:36:42 bauzas sean-k-mooney: but they won't do it
14:36:58 mnaser it looks like kombu has it's own error handling inside ensure_connection that maybe didn't exist back when openstack started using it
14:37:11 sean-k-mooney bauzas: well the reason i say that is libvirt does not define that as a stable interface
14:37:29 sean-k-mooney and createing/remvoing mdevs via the filesystem is pretty trivial
14:37:35 gibi mnaser: yeah komubo has the flags in the interface
14:37:40 gibi kombu
14:37:54 bauzas anyway, when I'm seeing it, I wonder whether libvirt folks know there are some upper services that use them
14:37:59 gibi mnaser: so we could configure kombu to stop after x retries
14:38:12 gibi mnaser: we just don't have the scaffolding in oslo.messaging to use that
14:38:23 mnaser gibi: yeah but i think then that brings the other interesting issue of 'does that mean it gives up forever for all future notifications'
14:38:24 sean-k-mooney bauzas: they are partly aware i had a converstaion with them about this in relateino to vdpa
14:38:42 sean-k-mooney and mac adresses
14:38:58 sean-k-mooney they do not guarenttee the names are stabel and said we shoudl avoid realying on them
14:40:11 bauzas meh
14:40:13 bauzas OK
14:40:16 sean-k-mooney bauzas: to me this is incorrect behavior in mdevctl and the libvirt chagne should be reverted
14:40:22 bauzas gibi: heh https://bugs.launchpad.net/nova/+bug/1951623
14:40:36 bauzas gibi: I guess this will be fixed by your change, right ?
14:41:40 bauzas oh, nevermind, it was for reboot
14:43:52 mnaser i do see a point in 'if notifications are down, stop doing things that can be billed'
14:45:10 gibi bauzas: unfortunatly that is a different problem
14:45:16 sean-k-mooney mnaser: that is anyting other then delete including allowing the instance to continue running :)
14:45:26 bauzas gibi: looks like, yes
14:45:39 mnaser sean-k-mooney: but that means more $$$ am i rite? =ap
14:46:09 gibi mnaser: yepp, I agree, but I think this behavior needs to be configurable
14:46:40 gibi mnaser: to handle the case when notification only just good to have
14:46:43 gibi but not must have
14:47:06 opendevreview Artom Lifshitz proposed openstack/nova master: DNM: Explode libvirt builds to trigger tempest error code https://review.opendev.org/c/openstack/nova/+/818932
14:47:16 sean-k-mooney mnaser: :) i think the correct behavior if notification cant be sent is either to just log an error and drop them or continue to have the service work but set its healtch check to degraded
14:47:34 sean-k-mooney the only other option i see would be for the service to terminate
14:48:01 opendevreview Artom Lifshitz proposed openstack/nova master: DNM: Explode libvirt builds to trigger tempest error code https://review.opendev.org/c/openstack/nova/+/818932
14:48:20 sean-k-mooney mnaser: if we are going to stop some operation from workign i dont think we shoudl be reportign the srvice as up in our current status filed
14:48:37 sean-k-mooney but that is based on a heatbeat which might still be working

Earlier   Later