| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-08-24 | |||
| 16:04:28 | gibi | is it only the two of us or other still listening quietly? | |
| 16:05:41 | gibi | I won't do a full meeting just for the two of us | |
| 16:06:04 | gibi | lets see what is important from the agenda | |
| 16:06:35 | gibi | we have one critical bug https://bugs.launchpad.net/nova/+bug/1940555 around SQLAlchemy and the fix is being discussed in: https://review.opendev.org/c/openstack/nova/+/805663 | |
| 16:07:21 | gibi | I guess it is a blocker for tripleoo | |
| 16:08:07 | gibi | overall the nova gate is is an acceptable state. we are landing patches | |
| 16:08:55 | gibi | about the PTG: every info is in the PTG etherpad #link https://etherpad.opendev.org/p/nova-yoga-ptg | |
| 16:09:17 | gibi | we will have FF next week | |
| 16:09:31 | gibi | release todos are being tracked in #link https://etherpad.opendev.org/p/nova-xena-rc-potential | |
| 16:09:50 | gibi | if you see release critical bugs then please tag them with xena-rc-potential tag | |
| 16:10:08 | gibi | as per elodilles the stable gate also look healthy | |
| 16:10:16 | gibi | is there any question before I close this? | |
| 16:10:25 | elodilles | nope :X | |
| 16:12:08 | gibi | then lets close this | |
| 16:12:10 | gibi | #endmeeting | |
| 16:12:10 | opendevmeet | Meeting ended Tue Aug 24 16:12:10 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | |
| 16:12:10 | opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2021/nova.2021-08-24-16.00.html | |
| 16:12:10 | opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2021/nova.2021-08-24-16.00.txt | |
| 16:12:10 | opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2021/nova.2021-08-24-16.00.log.html | |
| 16:13:51 | elodilles | o/ | |
| 16:37:35 | rmart04 | Hello All, I know this is a deployment question but its pretty nova specific, hopefully you someone might be able to help. When you perform live migration of an instance with a block device, and the disk is pre-live migrated using scp, is there anyway to configure that connection string to be via IP address rather than by hostname? For cases where ssh is only bound to a specific interface? That might not be | |
| 16:37:35 | rmart04 | what DNS points at? | |
| 16:39:26 | sean-k-mooney | gibi: elodilles soory was working on something downstream so missed meeting | |
| 16:40:25 | sean-k-mooney | but ya the sql fix is on my todo list. we will see if we can get that fixed this week | |
| 16:43:00 | sean-k-mooney | elodilles: can you priortiese review of this backport https://review.opendev.org/c/openstack/nova/+/805818 | |
| 16:46:09 | elodilles | sean-k-mooney: sure, looking | |
| 16:47:07 | rmart04 | @sean-k-mooney if you had any thoughts on the above that would be awesome :D | |
| 16:57:47 | sean-k-mooney | rmart04: i think so one sec | |
| 16:58:10 | sean-k-mooney | rmart04: https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_inbound_addr | |
| 16:59:31 | rmart04 | thanks for having a think, yes we have that set to the host IP that ssh listens on, and as far as I can tell that is working well for live migrations (with rbd backends). I'm wondering if the pre_migration requires an additional variable to be set? | |
| 17:00:11 | sean-k-mooney | ill check if there is somthing for the scp specifcially | |
| 17:00:26 | sean-k-mooney | if not you could hardcode the hostname to ip mapping in /etc/hosts | |
| 17:02:47 | sean-k-mooney | so the two ways we have for copyting the block sotrage is scp or rsync https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/nova/virt/libvirt/volume/remotefs.py#L172-L222 | |
| 17:03:55 | sean-k-mooney | i think this is how libvirt is calling it indirectly https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/nova/virt/libvirt/utils.py#L214-L251 | |
| 17:04:43 | rmart04 | hmm, we did have hardcoded etc/hosts but we were trying to remove that dependency as we have a lot of hosts and updating this and restarting nova_compute on hundreds of boxes every time was getting tiresome (even in ansible) | |
| 17:05:14 | rmart04 | maybe we might have to look to re-implement that | |
| 17:05:57 | rmart04 | I guess I should mention we are still on stein too | |
| 17:06:15 | rmart04 | _sad face_ | |
| 17:08:49 | ganso | melwitt, lyarwood, bauzas: If you have a spare moment could you please take a look at this backport that already has a +2? Keep in mind that the merge conflict is also proposed and also has a +2: https://review.opendev.org/c/openstack/nova/+/805304 Thanks in advance! | |
| 17:13:31 | sean-k-mooney | im not direclty seeing it. to be honest i tought for live migration libvirt actully did the copy and we only used scp for cold migration | |
| 17:15:02 | sean-k-mooney | ah so e do this here https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/nova/virt/libvirt/driver.py#L9855-L9870 | |
| 17:15:33 | sean-k-mooney | rmart04: so we do the copy form the dest passing the source host name | |
| 17:15:57 | sean-k-mooney | although that is for the config drive | |
| 17:17:24 | rmart04 | the thing that flagged this up today was migrating amphora, unsure if octavia uses config drive by default... could be | |
| 17:17:29 | rmart04 | i'll check | |
| 17:18:33 | johnsom | Yes | |
| 17:18:54 | rmart04 | ahhhh.... yes, disk.config | |
| 17:20:14 | sean-k-mooney | you could work around this by using vfat config drives | |
| 17:20:39 | sean-k-mooney | also we dont need this anymore so we can remove it on master i think | |
| 17:20:53 | sean-k-mooney | the libvirt bug has been fixed | |
| 17:22:09 | sean-k-mooney | rmart04: so ya https://github.com/openstack/nova/blob/stable/stein/nova/conf/configdrive.py#L30 you can use vfat if that is supported by the guests instead if you need too | |
| 17:23:38 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Add migration to resolve shadow table discrepancies https://review.opendev.org/c/openstack/nova/+/805738 | |
| 17:27:35 | rmart04 | ok thanks for all the info, will see how i get on tomorrow | |
| 18:04:56 | opendevreview | Lee Yarwood proposed openstack/nova master: fup: Fix os-volume_attachments api-ref parameters https://review.opendev.org/c/openstack/nova/+/805876 | |
| 18:06:34 | lyarwood | gmann: ack fixed in a fup on top of the api change to avoid the respin, apologies again! | |
| 18:06:45 | lyarwood | ganso: ack I'll review this evening | |
| 18:11:49 | opendevreview | Lee Yarwood proposed openstack/python-novaclient master: Microversion 2.89 - os-volume_attachments https://review.opendev.org/c/openstack/python-novaclient/+/805022 | |
| 18:14:47 | NobodyCam | Good Morning Nova folks, was disk_available_least hypervisor stat removed in Ussuri? | |
| 18:14:58 | NobodyCam | for Ironic hypervisors? | |
| 18:17:08 | sean-k-mooney | i dont think so in ussuri but you should not be use the disk filter in ussuri for ironic or otherwise if that is why you are asking | |
| 18:19:29 | NobodyCam | hey hey Sean, in queens that was a value I used to validate the node was ready for deployment. | |
| 18:21:15 | sean-k-mooney | why not use the node status in ironic for that | |
| 18:21:25 | sean-k-mooney | its ready for deployment when its in the avaiable state | |
| 18:22:35 | NobodyCam | with that we hit no valid hosts, I should add that when we provision we use force_node to deploy to specific nodes | |
| 18:24:46 | NobodyCam | here is a sample if what I was seeing: | |
| 18:24:57 | NobodyCam | https://www.irccloud.com/pastebin/fuSfO22M/ | |
| 18:25:39 | sean-k-mooney | NobodyCam: ironic stopped reporting resouces for cpu ram and disk several cycles ago | |
| 18:26:04 | NobodyCam | ack | |
| 18:26:24 | sean-k-mooney | i dont rember off the top of my head when we moved to trackign ironic nodes with custom resouce classes but im sure tha tlikely had an effect of os-hyperviors output eventually | |
| 18:26:39 | NobodyCam | yep | |
| 18:27:07 | sean-k-mooney | in wally we have removed disk_avaiable_lset https://specs.openstack.org/openstack/nova-specs/specs/wallaby/implemented/modernize-os-hypervisors-api.html | |
| 18:27:39 | NobodyCam | ++ | |
| 18:28:02 | sean-k-mooney | NobodyCam: what you could do is see if the node exist in os-hyperviors rather then check for that value specifically | |
| 18:28:29 | sean-k-mooney | but still the expected way to check this would be via ironic | |
| 18:28:52 | NobodyCam | yea, let me try a few thing... yea we do check ironic | |
| 18:29:09 | sean-k-mooney | NobodyCam: i would proably check the state filed | |
| 18:29:16 | sean-k-mooney | and mayby status | |
| 18:29:28 | sean-k-mooney | so check for sate:up status:enabled | |
| 18:29:57 | sean-k-mooney | that should mean its schduleable | |
| 18:30:33 | NobodyCam | ++ Awesome, let try some deploys with that... Thank you very much sean-k-mooney !!!! | |
| 18:30:41 | NobodyCam | I'll let you know what I find | |
| 18:30:47 | sean-k-mooney | cool | |
| 18:31:23 | sean-k-mooney | the other thing you could check woudl be placment to see that the node is there but i think os-hyperviors will only have the node after its in placment anyway | |
| 18:31:29 | opendevreview | Merged openstack/nova master: scheduler: Merge driver into manager https://review.opendev.org/c/openstack/nova/+/773644 | |
| 18:31:41 | opendevreview | Merged openstack/nova master: tests: Merge 'test_utils', 'test_scheduler_utils' https://review.opendev.org/c/openstack/nova/+/773646 | |
| 18:31:51 | opendevreview | Merged openstack/nova master: conf: Remove deprecated aliases https://review.opendev.org/c/openstack/nova/+/773647 | |
| 18:55:46 | lyarwood | melwitt: https://review.opendev.org/c/openstack/placement/+/679441/ - if you're happy to look at the nits from me and Sean in a fup I'm happy to +W this now | |
| 18:57:23 | melwitt | lyarwood: sure thing, sorry I had only skimmed them so far | |
| 18:58:57 | lyarwood | melwitt: no issues they are nits after all, I'll look at the AttributeCache change now | |
| 19:00:55 | melwitt | lyarwood: thanks, appreciate it. and I will look at the respin of the nova-manage volume patch today | |
| 19:01:49 | lyarwood | melwitt: thanks, https://review.opendev.org/c/openstack/nova/+/805667 and https://review.opendev.org/c/openstack/nova/+/802317 could also use another core if you get time | |
| 19:02:13 | lyarwood | the first fixing a gate-failure I introduced | |
| 19:02:41 | melwitt | will do, added to the queue. thanks | |
| 19:30:58 | lyarwood | sean-k-mooney: are you about? | |
| 19:31:45 | sean-k-mooney | yes | |
| 19:32:07 | lyarwood | sean-k-mooney: https://zuul.opendev.org/t/openstack/build/ddb6cf53e65a404c89dac4dc4ca1cd51/log/controller/logs/screen-n-cpu.txt#13396 & https://e31f9be4742ab987838e-5d50e8ef24c23b6b3512cdd707b25e49.ssl.cf1.rackcdn.com/804286/4/gate/tempest-ipv6-only/ddb6cf5/testr_results.html - So I'm seeing a test failure dump out console logs suggesting that CirrOS is still trying and failing to load apic related stuff even with the feature disabled | |
| 19:32:08 | lyarwood | in the domain | |
| 19:35:13 | lyarwood | I wonder if we actually want <ioapic driver='qemu'/> | |
| 19:36:09 | sean-k-mooney | i dont think soe but we migtht want driver='none' | |
| 19:36:56 | sean-k-mooney | wait ioapic | |