Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-09-06
13:16:46 sean-k-mooney viks__: outside of openstack there are some host level sysctl tunings you can do to improve guest io perfromance like changing the io schduler or dirty data writeback tresholds
13:18:31 sean-k-mooney tuned can be used as a erfernce for some of the commnly tuned settings
13:18:33 sean-k-mooney https://github.com/redhat-performance/tuned/blob/master/profiles/throughput-performance/tuned.conf#L24-L59
13:19:52 viks__ sean-k-mooney: thanks for all the inputs.. will do some testing based on these .... thanks again...
13:22:42 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: check nova-next with osc-placement fix https://review.opendev.org/c/openstack/nova/+/807558
13:25:34 sean-k-mooney viks__: if you are testing virtio-scsi i the image options are hw_disk_bus=scsi hw_scsi_model=virtio-scsi by they way
13:25:51 sean-k-mooney i previously msityped it as hw_disk_bus=virtio-scsi
13:26:11 viks__ sean-k-mooney: ok...
14:06:16 bauzas huzzah, can see my review priorities : https://review.opendev.org/q/label:Review-Priority%253E%253D1%252Csbauza
14:20:19 opendevreview Merged openstack/osc-placement master: Repro allocation show bug with empty allocation https://review.opendev.org/c/openstack/osc-placement/+/807553
14:20:20 opendevreview Merged openstack/osc-placement master: Fix allocation show / unset with empty allocation https://review.opendev.org/c/openstack/osc-placement/+/807556
14:41:23 gibi lyarwood, bauzas: ^^ thanks for the quick approve. I will ask for a release freeze exception shortly
14:53:41 gibi RFE requested: http://lists.openstack.org/pipermail/openstack-discuss/2021-September/024686.html
14:55:35 slaweq hi stable nova cores, can somebody check https://review.opendev.org/c/openstack/nova/+/791421 /.
14:55:38 slaweq ?
14:55:44 slaweq thx in advance for help :)
15:28:58 gibi do we have USA out today due to Labor Day?
15:31:33 kashyap gibi: Yeah
15:31:40 kashyap Also Canada, IIRC
15:31:46 gibi OK, then I don't wait for them today :)
15:57:54 bauzas gibi: sorry was on a meeting
15:58:59 bauzas gibi: yup, saw your FFE request but as kashyap said and you guessed, a whole portion of the world located between Pacific and Atlantic oceans and above a certain latitude is currently shutdown for the day
15:59:20 gibi bauzas: no worries the release now queued
15:59:35 bauzas said a French guy working
15:59:59 gibi bauzas: I looked at melwitt's repro patch https://review.opendev.org/c/openstack/placement/+/807493 and I left some ideas but no real solutions yet
16:01:51 bauzas gibi: honestly we face the limits of the single-commit design we have with Placement
16:03:26 gibi bauzas: this is now more about writing a sane a repro test that excersise transaction isolation with mysql (as sqlite does not have it)
16:03:57 gibi I think the fix melwitt's proposed on top is sane but the repro test is racy somehow actually forks processes
16:04:02 gibi whihc is scarry
16:04:28 bauzas gibi: yup, I saw the tox modification and I understood the reasoning
16:12:04 melwitt showing the effect of the consistent read problem inside the transaction
16:12:04 melwitt I will try to find a different way to do it. it's inherently problematic to try and repro this because once one path starts a transaction, trying to do something in the middle of it (to cause the race state) poses a record locking problem. at one point I was trying to fake it by just returning bogus things to make it think it hit a generation conflict (without doing any database write), which worked to make it retry but that wasn't
16:21:40 gibi melwitt: yes, exactly, that was the dead end I went down this afternoon :/
16:22:16 gibi melwitt: I think at some point we can accept that we cannot reliably test this in func env and simply land the fix and monitor the tempest jobs to see if it is resolved the race or not
16:22:54 melwitt gibi: I went down the dead end three days in a row 😑
16:23:04 melwitt gibi: yeah. I was thinking that too. I kept thinking there has to be a way but if there is I'm not clever enough to find it haha
16:24:06 gibi melwitt: but enjoy your day off, we can continue this tomorrow
16:24:53 melwitt kk, thanks
#openstack-nova - 2021-09-07
01:49:39 opendevreview Jorhson Deng proposed openstack/nova master: recheck the attachment_id after the reschedule successful https://review.opendev.org/c/openstack/nova/+/796209
03:15:06 opendevreview Ade Lee proposed openstack/nova master: Add check job for FIPS https://review.opendev.org/c/openstack/nova/+/790519
07:57:50 opendevreview Wenping Song proposed openstack/placement master: Dropping lower constraints testing https://review.opendev.org/c/openstack/placement/+/787863
08:45:57 bbezak Hi - Can we make a release os-vif for Victoria? 2.2.1? this bug is quite critical - https://bugs.launchpad.net/os-vif/+bug/1892132. And it is not part of 2.2.0 release
09:02:02 sean-k-mooney its not actully a bug it was a kernel abi break
09:02:31 sean-k-mooney but yes we can proably do a release we just need too propose it to the release repo
09:03:00 sean-k-mooney bbezak: the release managmen it automated via a git repo
09:04:04 sean-k-mooney so we just need to update this with the corerct sha https://github.com/openstack/releases/blob/master/deliverables/victoria/os-vif.yaml
09:04:31 sean-k-mooney there is a script in the release repo for doing that or you can do it by hand
09:05:59 sean-k-mooney https://github.com/openstack/os-vif/compare/2.2.0...stable/victoria
09:06:16 kashyap sean-k-mooney: What's the kernel ABI break?
09:06:25 bbezak indeed - OFED 5.4 also introduced that change in renumeration at some level.
09:06:25 sean-k-mooney looking at the delta it does not look like there is much else included but release are pretty cheap
09:06:31 bbezak https://bugzilla.redhat.com/show_bug.cgi?id=1918703
09:06:40 kashyap Ah, a commit is linked - https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=123f0f53dd64b67e34142485fe866a8a581f12f1
09:06:42 bbezak kashyap: https://bugzilla.redhat.com/show_bug.cgi?id=1918703
09:06:51 kashyap Thanks
09:07:00 sean-k-mooney kashyap: melonox moved where the reprentor netdevs were advertised in /sys
09:07:26 sean-k-mooney we have fixed os-vif and libvirt to be able to handel both now
09:07:37 sean-k-mooney but we need to actully release os-vif with the fix
09:07:42 kashyap I see
09:08:02 kashyap This seems to be a "forced" ABI break - because of an external vendor
09:08:06 kashyap But yeah, a break nonetheless
09:08:33 sean-k-mooney well yes they chose to do this to enable a new feature in the future
09:09:02 sean-k-mooney we adapted on master a long time ago and the backports where held on review bandwith for some time
09:09:23 sean-k-mooney now that they have actully landed we can do a realase for them
09:09:51 bbezak yesterday ussuri was merged, but I think older ones are still waiting for +2s
09:10:12 bbezak and/or rechecks
09:10:16 sean-k-mooney i think train was reviewed. i should now have stable branch rights so ill take a look
09:10:50 sean-k-mooney ah you are right
09:10:54 sean-k-mooney they are still pending
09:12:31 sean-k-mooney its in the governace repo
09:12:43 bauzas ok, then it's not needed
09:13:02 sean-k-mooney https://github.com/openstack/governance/blob/master/reference/runtimes/xena.rst
09:13:41 sean-k-mooney technically i guess these are minimume but 3.6-3.8 is what is tested
09:14:04 sean-k-mooney technially i think we can mostly run on 3.5
09:14:57 sean-k-mooney bbezak: i have rechecked the train version ill take a look at them again later today
09:15:22 sean-k-mooney bbezak: are you going to porpos a release to the git repo or would you like me to do that
09:17:40 bbezak sean-k-mooney: thx, I will try to propose git release for victoria/ussuri
09:45:02 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: check nova job results with placement transaction fix https://review.opendev.org/c/openstack/nova/+/807558
10:07:12 bbezak ussuri, victoria os-vif release has been proposed, as per docs - https://releases.openstack.org/reference/using.html#using-new-release-command. PTL needs to approve those, please take a look gibi:
10:07:15 bbezak https://review.opendev.org/c/openstack/releases/+/807694 https://review.opendev.org/c/openstack/releases/+/807696
10:17:36 sean-k-mooney gibi: i have reviewed both for those ^ and they look good to me. we dont have any other pending backports that i see that we should wait for
10:18:26 sean-k-mooney i have some patch for master that i plan to backport but we can do another release for those in a few weeks they are not urgent and release are pretty cheap so i have no issue with defering them for now
10:18:28 bauzas gibi: permission to rewite your cycle highligts for the prelude ?
10:39:49 gibi bauzas: grandted :)
10:40:06 gibi sean-k-mooney: I will check them soon
11:10:29 gibi bauzas: when you need some distraction there is two doc / reno patch up for review https://review.opendev.org/c/openstack/nova/+/807564 and https://review.opendev.org/c/openstack/nova/+/705667
11:25:21 gibi stephenfin, sean-k-mooney: I added a topic about tox.ini basepython pinning to the PTG etherpad https://etherpad.opendev.org/p/nova-yoga-ptg L81
11:42:28 gibi lyarwood: if you have time I could use your oppinion on https://bugs.launchpad.net/nova/+bug/1942766
11:54:53 lyarwood gibi: yeah live and then persistent could be an option but ultimately a hard reboot would recover the situation so it's a `low` bug at best IMHO
11:55:37 gibi lyarwood: as the hard reboot will regenareate the xml from the db?
11:56:15 lyarwood gibi: correct it destroys the live domain and undefined the persistent domain before recreating everything based on what we have in the db
11:56:21 lyarwood undefines*
11:56:47 gibi cool, thanks
12:45:46 opendevreview Balazs Gibizer proposed openstack/nova master: Add more retries to TestMigrateFromDownHost tests https://review.opendev.org/c/openstack/nova/+/807714
12:52:41 gibi lyarwood: a further tuning on these tests ^^
12:52:59 gibi ohh I see you already checked
12:53:02 gibi thanks
12:53:08 lyarwood gibi: yup already reviewed, I did see a gate failure with this last week but didn't get time to look, thanks for sorting that again
12:53:35 gibi lyarwood: no problem, I like hunting down these :)
12:53:53 gibi and I think there is no way to avoid races sometime, nova is complex

Earlier   Later