Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-02-28
13:03:18 opendevreview Amit Uniyal proposed openstack/nova stable/wallaby: libvirt: Delegate OVS plug to os-vif https://review.opendev.org/c/openstack/nova/+/790447
13:07:21 opendevreview Amit Uniyal proposed openstack/nova stable/wallaby: libvirt: Delegate OVS plug to os-vif https://review.opendev.org/c/openstack/nova/+/790447
13:09:20 ratailor_ #openstack-nova can I get reviews for these patches https://review.opendev.org/q/Ia738a0972b050f549f446c85171d3f33e60ada4f and https://review.opendev.org/q/Id4c8c5f3b32985ac7d3d7c833b82e0876f7367c1 ?
13:20:53 opendevreview Jorge San Emeterio proposed openstack/nova master: WIP: Have schema for 'lock' action be applied to all microversions. https://review.opendev.org/c/openstack/nova/+/875653
13:21:23 opendevreview Jorge San Emeterio proposed openstack/nova master: WIP: Have schema for 'lock' action be applied to all microversions. https://review.opendev.org/c/openstack/nova/+/875653
13:43:42 lowercase stephenfin: I looked into our next enviornment, which also contains a build_requests table with no columns.
13:43:52 lowercase in an identical state to the dev
13:45:33 opendevreview Amit Uniyal proposed openstack/nova stable/wallaby: libvirt: Delegate OVS plug to os-vif https://review.opendev.org/c/openstack/nova/+/790447
13:46:01 lowercase and i just checked one of my production databases, also include a build_requests in the same state. select * from build_requests; Empty set (0.000 sec)
13:52:30 opendevreview Alexey Stupnikov proposed openstack/nova stable/victoria: Test aborting queued live migration https://review.opendev.org/c/openstack/nova/+/845748
13:52:41 opendevreview Alexey Stupnikov proposed openstack/nova stable/victoria: Add functional tests to reproduce bug #1960412 https://review.opendev.org/c/openstack/nova/+/845753
13:52:48 opendevreview Alexey Stupnikov proposed openstack/nova stable/victoria: Clean up when queued live migration aborted https://review.opendev.org/c/openstack/nova/+/845754
13:59:49 opendevreview Sylvain Bauza proposed openstack/nova master: Update to the PTL guide https://review.opendev.org/c/openstack/nova/+/875730
14:05:09 opendevreview Sylvain Bauza proposed openstack/nova master: Update to the PTL guide https://review.opendev.org/c/openstack/nova/+/875730
14:46:00 bauzas elodilles: I'm done with the meeting wikipage, feel free to amend it anytime you want
14:47:55 elodilles bauzas: ack, thanks
14:48:15 elodilles bauzas: well, actually, it is done o:)
14:48:35 elodilles no need to update :X
14:48:51 bauzas elodilles: I'll name you Barry Allen
14:50:53 elodilles just a sec, have to google this. :S
14:51:00 dansmith bauzas: why remove the grenade-slip-level job for bobcat? it's not required, but we might as well keep it.. is it because it hasn't yet been updated for zed->bobcat?
14:54:00 bauzas dansmith: good question I was about getting a coffee before pinging you to ask you what you would prefer
14:54:27 dansmith personally I'd rather leave it in place as much as we can, as I think it just adds extra coverage
14:54:39 dansmith however, I need to look at it again,
14:54:59 bauzas dansmith: okay, so we would leave non-voting in a non-SLURP branch and just make it voting after Bobcat RC1 ?
14:55:14 dansmith as it might be that we need to change it so that it tests the right releases, which we can't do until after rc of course
14:55:34 dansmith I would leave it voting personally
14:55:49 bauzas even for like B>D ?
14:56:10 dansmith yeah, I mean,
14:56:24 dansmith are you thinking we'll run into issues with deprecations?
14:56:50 dansmith unless it's something specifically in that test I think we'll be okay.. we've had it enabled for a while and never found any problems in nova (IIRC)
14:56:52 bauzas well,
14:57:26 bauzas I'm rather thinking about for example when we remove a RPC compat code
14:58:13 dansmith this isn't a live test, so I don't think that will matter
14:58:17 bauzas like for when removing https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L11306
15:00:42 bauzas anyway, if you prefer to continue the job, we could be just making it to voting, and in case after C** RC1, if we see it has some problem, then we could make it non-voting
15:01:34 bauzas I mean, make it voting after Bobcat RC1, and leave it voting even after C* RC1 unless we see a problem
15:02:11 dansmith sure, I mean, during bobcat it doesn't *have* to be voting, but I think it would be a good idea
15:02:28 dansmith at least we'll notice what we're breaking and we can make it n-v if that's what we need to proceed
15:04:29 sean-k-mooney as long as it moves to 22.04 form 20.04 so we can drop focal testing im ok with that
15:05:45 sean-k-mooney i think we need that change to bump our min libvirt version which we skipped doing now for a few releases
15:05:54 sean-k-mooney so we should do it in bobcat and do it early
15:06:29 stephenfin lowercase: I assume you mean no rows? There should be columns. The question is if they're the columns we're expecting, i.e. `vm_state`
15:06:32 dansmith ack, I think we should be able to switch it to zed->bobcat and 22.04 as soon as bobcat opens.. gmann you okay with that?
15:08:16 bauzas bobcat will open in 2 days :)
15:09:01 lowercase show tables; -> build_requests is present. select * from build_requests ; -> Empty set (0.001 sec) show full columns from build_requests ; -> there are about 23 columns there from a quick count.
15:09:25 dansmith bauzas: I was going to say it actually matters when the relevant projects have branches, but that's the grenade holdup.. since this is two back, we should be able to do it immediately, yeah
15:10:10 dansmith bauzas: the only concern would be people using the job to merge things on their rc branches that need coverage.. but I think we're going to need a grenade-skip-level-previous (or something) job anyway to keep stable happy, so maybe this is the time to practice that
15:10:36 bauzas hmmm
15:11:06 lowercase stephenfin: see previous response
15:12:27 stephenfin lowercase: can you dump the output of 'describe build_requests;' to a pastebin?
15:12:33 bauzas dansmith: have you seen my service version change ? We shouldn't longer need to use this https://github.com/openstack/grenade/blob/master/.zuul.yaml#L401
15:13:04 stephenfin lowercase: and 'select * from alembic_version;'
15:13:19 dansmith bauzas: something other than 875621?
15:13:27 stephenfin lowercase: This is for the next environment, of course
15:13:54 bauzas dansmith: https://review.opendev.org/c/openstack/nova/+/874932
15:13:58 lowercase stephenfin: this is my qa cluster that is on xena, and is not being upgraded to yoga yet. d67eeaabee36
15:15:06 dansmith bauzas: ah, yeah I had but I didn't connect the dots
15:15:44 bauzas dansmith: so are you agreeing on the change ? (like, just continuing to support Yoga)
15:15:54 stephenfin lowercase: Cool, so alembic_version looks correct. Now to ensure that the columns that migration b30f573d3377 removes are present there, as expected (the 'describe' command)
15:16:27 bauzas dansmith: and just after Antelope RC1, bumping the service version to Antelope
15:16:40 dansmith bauzas: yeah I mean, I think I'm okay with that.. the change doesn't really change anything (other than get ready for the future with the missing service versions) right?
15:17:10 lowercase stephenfin: https://paste.centos.org/view/529a12ed
15:17:16 dansmith bauzas: meaning I'm happy to loosen our requirements a cycle before we officially do
15:17:41 bauzas dansmith: yeah the problem that I have is that I need to understand again every cycle how we can support SLURP
15:17:59 bauzas so I want to make sure we do this correctlyu
15:19:34 stephenfin lowercase: Okay, they're all there as expected. I would expect the migrations to apply cleanly in that environment. So the question is how is that test environment created?
15:20:41 lowercase stephenfin: the test and qa environments are created using openstack-ansible and I do a really good job ensuring it stays very close to our remaining environments
15:20:59 stephenfin lowercase: In that case, I suspect either (a) someone has changed something behind alembic's back or (b) alembic ran but failed to record the updated version in the alembic_version table. I think you already ruled out (a)?
15:21:09 lowercase uhh.. so i just dropped that table in test and it now errors with 1146, \"Table 'nova_api.build_requests' doesn't exist\")", "[SQL: ALTER TABLE build_requests DROP COLUMN vm_state]", "(Background on this error at: https://sqlalche.me/e/14/f405)"]} bahaha
15:21:24 lowercase in test, this is the one that is going to yoga
15:22:22 stephenfin lowercase: that's expected. You've modified the schema behind alembic's back. It (intentionally) doesn't do introspection. It expects the schema to be in a given state and makes changes based on that
15:23:08 stephenfin The only thing that should be issuing CREATE, DROP or ALTER operations is alembic. Not the operator.
15:23:21 lowercase stephenfin: that's a fair statement. its just funny that im going to need to reverse engineer the creation of a table just so the script can remove it
15:23:39 stephenfin lowercase: It's not removing the table - it's modifying it
15:24:23 lowercase oh yep. okay i see that now. just dropping everything inside of it and emptying the schema
15:26:46 stephenfin lowercase: Nah, it doesn't drop everything inside of it either. As the name would suggest, the build requests table contains build requests. A record is created in there when a build is requested. It's deleted once that build is completed (either success or failure)
15:27:23 stephenfin That's why it looks empty. If you watched the table while creating instances though, you'd see stuff appearing and disappearing.
15:27:47 stephenfin https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L554-L563
15:29:33 stephenfin lowercase: Also, this is the schema from a Zed-based DevStack deployment https://paste.opendev.org/show/bm3psmcu51ltEvP1sT9D/
15:32:14 lowercase i just dumped the table out of qa cluster, and im using that to build the dev cluster to get it to pass
15:33:29 lowercase okay, that looks pretty good.. rerunning the code!
15:40:55 lowercase stephenfin: "Error: (pymysql.err.OperationalError) (1091, \"Can't DROP INDEX `shadow_instance_extra_idx`; check that it exists\")", "[SQL: ", "DROP INDEX shadow_instance_extra_idx ON shadow_instance_extra]",
15:41:40 lowercase regredably, im getting swamped over here. I don't have time to look into this further. I'll take some time to look at this and get back to you.
15:51:20 bauzas reminder : nova meeting in 9 mins here
15:51:49 bauzas I'll also need to bail out quickly after the meeting
16:02:00 bauzas #startmeeting nova
16:02:00 opendevmeet Meeting started Tue Feb 28 16:02:00 2023 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:02:00 opendevmeet Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:02:00 opendevmeet The meeting name has been set to 'nova'
16:02:06 bauzas sorry, bit late here
16:02:22 Uggla o/
16:02:30 elodilles o/
16:02:31 bauzas #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
16:02:40 bauzas let's start
16:02:44 bauzas #topic Bugs (stuck/critical)
16:02:44 dansmith o/
16:02:50 bauzas #info No Critical bug
16:02:54 bauzas #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 17 new untriaged bugs (+1 since the last meeting)
16:03:05 bauzas Uggla: any bug you wanna raise ?

Earlier   Later