Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-09-06
14:55:38 slaweq ?
14:55:44 slaweq thx in advance for help :)
15:28:58 gibi do we have USA out today due to Labor Day?
15:31:33 kashyap gibi: Yeah
15:31:40 kashyap Also Canada, IIRC
15:31:46 gibi OK, then I don't wait for them today :)
15:57:54 bauzas gibi: sorry was on a meeting
15:58:59 bauzas gibi: yup, saw your FFE request but as kashyap said and you guessed, a whole portion of the world located between Pacific and Atlantic oceans and above a certain latitude is currently shutdown for the day
15:59:20 gibi bauzas: no worries the release now queued
15:59:35 bauzas said a French guy working
15:59:59 gibi bauzas: I looked at melwitt's repro patch https://review.opendev.org/c/openstack/placement/+/807493 and I left some ideas but no real solutions yet
16:01:51 bauzas gibi: honestly we face the limits of the single-commit design we have with Placement
16:03:26 gibi bauzas: this is now more about writing a sane a repro test that excersise transaction isolation with mysql (as sqlite does not have it)
16:03:57 gibi I think the fix melwitt's proposed on top is sane but the repro test is racy somehow actually forks processes
16:04:02 gibi whihc is scarry
16:04:28 bauzas gibi: yup, I saw the tox modification and I understood the reasoning
16:12:04 melwitt I will try to find a different way to do it. it's inherently problematic to try and repro this because once one path starts a transaction, trying to do something in the middle of it (to cause the race state) poses a record locking problem. at one point I was trying to fake it by just returning bogus things to make it think it hit a generation conflict (without doing any database write), which worked to make it retry but that wasn't
16:12:04 melwitt showing the effect of the consistent read problem inside the transaction
16:21:40 gibi melwitt: yes, exactly, that was the dead end I went down this afternoon :/
16:22:16 gibi melwitt: I think at some point we can accept that we cannot reliably test this in func env and simply land the fix and monitor the tempest jobs to see if it is resolved the race or not
16:22:54 melwitt gibi: I went down the dead end three days in a row 😑
16:23:04 melwitt gibi: yeah. I was thinking that too. I kept thinking there has to be a way but if there is I'm not clever enough to find it haha
16:24:06 gibi melwitt: but enjoy your day off, we can continue this tomorrow
16:24:53 melwitt kk, thanks
#openstack-nova - 2021-09-07
01:49:39 opendevreview Jorhson Deng proposed openstack/nova master: recheck the attachment_id after the reschedule successful https://review.opendev.org/c/openstack/nova/+/796209
03:15:06 opendevreview Ade Lee proposed openstack/nova master: Add check job for FIPS https://review.opendev.org/c/openstack/nova/+/790519
07:57:50 opendevreview Wenping Song proposed openstack/placement master: Dropping lower constraints testing https://review.opendev.org/c/openstack/placement/+/787863
08:45:57 bbezak Hi - Can we make a release os-vif for Victoria? 2.2.1? this bug is quite critical - https://bugs.launchpad.net/os-vif/+bug/1892132. And it is not part of 2.2.0 release
09:02:02 sean-k-mooney its not actully a bug it was a kernel abi break
09:02:31 sean-k-mooney but yes we can proably do a release we just need too propose it to the release repo
09:03:00 sean-k-mooney bbezak: the release managmen it automated via a git repo
09:04:04 sean-k-mooney so we just need to update this with the corerct sha https://github.com/openstack/releases/blob/master/deliverables/victoria/os-vif.yaml
09:04:31 sean-k-mooney there is a script in the release repo for doing that or you can do it by hand
09:05:59 sean-k-mooney https://github.com/openstack/os-vif/compare/2.2.0...stable/victoria
09:06:16 kashyap sean-k-mooney: What's the kernel ABI break?
09:06:25 sean-k-mooney looking at the delta it does not look like there is much else included but release are pretty cheap
09:06:25 bbezak indeed - OFED 5.4 also introduced that change in renumeration at some level.
09:06:31 bbezak https://bugzilla.redhat.com/show_bug.cgi?id=1918703
09:06:40 kashyap Ah, a commit is linked - https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=123f0f53dd64b67e34142485fe866a8a581f12f1
09:06:42 bbezak kashyap: https://bugzilla.redhat.com/show_bug.cgi?id=1918703
09:06:51 kashyap Thanks
09:07:00 sean-k-mooney kashyap: melonox moved where the reprentor netdevs were advertised in /sys
09:07:26 sean-k-mooney we have fixed os-vif and libvirt to be able to handel both now
09:07:37 sean-k-mooney but we need to actully release os-vif with the fix
09:07:42 kashyap I see
09:08:02 kashyap This seems to be a "forced" ABI break - because of an external vendor
09:08:06 kashyap But yeah, a break nonetheless
09:08:33 sean-k-mooney well yes they chose to do this to enable a new feature in the future
09:09:02 sean-k-mooney we adapted on master a long time ago and the backports where held on review bandwith for some time
09:09:23 sean-k-mooney now that they have actully landed we can do a realase for them
09:09:51 bbezak yesterday ussuri was merged, but I think older ones are still waiting for +2s
09:10:12 bbezak and/or rechecks
09:10:16 sean-k-mooney i think train was reviewed. i should now have stable branch rights so ill take a look
09:10:50 sean-k-mooney ah you are right
09:10:54 sean-k-mooney they are still pending
09:12:31 sean-k-mooney its in the governace repo
09:12:43 bauzas ok, then it's not needed
09:13:02 sean-k-mooney https://github.com/openstack/governance/blob/master/reference/runtimes/xena.rst
09:13:41 sean-k-mooney technically i guess these are minimume but 3.6-3.8 is what is tested
09:14:04 sean-k-mooney technially i think we can mostly run on 3.5
09:14:57 sean-k-mooney bbezak: i have rechecked the train version ill take a look at them again later today
09:15:22 sean-k-mooney bbezak: are you going to porpos a release to the git repo or would you like me to do that
09:17:40 bbezak sean-k-mooney: thx, I will try to propose git release for victoria/ussuri
09:45:02 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: check nova job results with placement transaction fix https://review.opendev.org/c/openstack/nova/+/807558
10:07:12 bbezak ussuri, victoria os-vif release has been proposed, as per docs - https://releases.openstack.org/reference/using.html#using-new-release-command. PTL needs to approve those, please take a look gibi:
10:07:15 bbezak https://review.opendev.org/c/openstack/releases/+/807694 https://review.opendev.org/c/openstack/releases/+/807696
10:17:36 sean-k-mooney gibi: i have reviewed both for those ^ and they look good to me. we dont have any other pending backports that i see that we should wait for
10:18:26 sean-k-mooney i have some patch for master that i plan to backport but we can do another release for those in a few weeks they are not urgent and release are pretty cheap so i have no issue with defering them for now
10:18:28 bauzas gibi: permission to rewite your cycle highligts for the prelude ?
10:39:49 gibi bauzas: grandted :)
10:40:06 gibi sean-k-mooney: I will check them soon
11:10:29 gibi bauzas: when you need some distraction there is two doc / reno patch up for review https://review.opendev.org/c/openstack/nova/+/807564 and https://review.opendev.org/c/openstack/nova/+/705667
11:25:21 gibi stephenfin, sean-k-mooney: I added a topic about tox.ini basepython pinning to the PTG etherpad https://etherpad.opendev.org/p/nova-yoga-ptg L81
11:42:28 gibi lyarwood: if you have time I could use your oppinion on https://bugs.launchpad.net/nova/+bug/1942766
11:54:53 lyarwood gibi: yeah live and then persistent could be an option but ultimately a hard reboot would recover the situation so it's a `low` bug at best IMHO
11:55:37 gibi lyarwood: as the hard reboot will regenareate the xml from the db?
11:56:15 lyarwood gibi: correct it destroys the live domain and undefined the persistent domain before recreating everything based on what we have in the db
11:56:21 lyarwood undefines*
11:56:47 gibi cool, thanks
12:45:46 opendevreview Balazs Gibizer proposed openstack/nova master: Add more retries to TestMigrateFromDownHost tests https://review.opendev.org/c/openstack/nova/+/807714
12:52:41 gibi lyarwood: a further tuning on these tests ^^
12:52:59 gibi ohh I see you already checked
12:53:02 gibi thanks
12:53:08 lyarwood gibi: yup already reviewed, I did see a gate failure with this last week but didn't get time to look, thanks for sorting that again
12:53:35 gibi lyarwood: no problem, I like hunting down these :)
12:53:53 gibi and I think there is no way to avoid races sometime, nova is complex
12:53:54 lyarwood we all have our vices :D
12:54:03 gibi :D
13:35:55 bauzas man, I wish generated sphinx errors were simplier to debug with reno
13:40:49 kashyap bauzas: What's the error that's giving you grief?
13:41:10 bauzas /home/sbauza/git/openstack/nova/releasenotes/source/unreleased.rst:40: WARNING: Bullet list ends without a blank line; unexpected unindent.
13:41:45 kashyap Ah, I've seen this enough no. of times that it is now written into my brain's ROM
13:42:30 bauzas also, running the tox releasenotes target takes a while
13:42:38 bauzas it's repopulating all the releases
13:48:32 opendevreview Balazs Gibizer proposed openstack/nova master: Avoid unbound instance_uuid var during delete https://review.opendev.org/c/openstack/nova/+/805605
14:28:38 opendevreview Merged openstack/nova master: fup: Print message logging uncaught nova-manage exceptions https://review.opendev.org/c/openstack/nova/+/807358
14:29:10 opendevreview Merged openstack/nova master: console: Improve logging https://review.opendev.org/c/openstack/nova/+/778407
14:31:10 gibi cores: do we support providing the adminPassword to the guest via the metadata service?
14:31:39 gibi I see that the metadata service trying to fetch the password from instance.system_metadata https://github.com/openstack/nova/blob/402fe188b4e7ff76109e8a5ea1f24a5e915eaa09/nova/api/metadata/password.py#L37
14:31:52 gibi but I don't see we ever store the adminPassword there

Earlier   Later