Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-05-15
12:31:02 mdbooth mriedem: Let me clarify 'we didn't', btw: the eventlet band-aid thing didn't break it, it was already broken.
12:31:11 mdbooth I believe them that it's broken, though.
12:32:16 mdbooth mriedem: Yes. We discussed it at PTG and the feeling was that we'd address it later. However, I think that was when we believe we'd applied an effective band-aid already.
12:32:19 sean-k-mooney if we dont have a path forward with melwitt's serise we might want to consider reverting the scater gatehr stuff temporally and dropping eventlets
12:32:42 mdbooth sean-k-mooney: Also worthy of discussion.
12:36:41 sean-k-mooney so looks like https://review.opendev.org/#/c/657750/1 is the last patch in that series and it passing ci but i think we are currently stuck on the open question in https://review.opendev.org/#/c/650172/14
12:37:49 sean-k-mooney is the startx issue seperate for teh amqp issue reported on the mailing list
12:38:14 sean-k-mooney http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005822.html
12:42:38 mdbooth Thanks for the reminder, sean-k-mooney
12:54:46 mriedem tssurya: we need a novaclient release before osc can use the locked reason stuff https://review.opendev.org/#/c/659124/
13:01:12 mriedem i'll propose a release
13:06:30 openstackgerrit Matt Riedemann proposed openstack/python-novaclient master: Cap sphinx for py2 to match global requirements https://review.opendev.org/659202
13:14:30 openstackgerrit Matt Riedemann proposed openstack/python-novaclient master: Cap sphinx for py2 and drop keyring dependency https://review.opendev.org/659202
13:19:04 mriedem the requirements-check job is happy with ^ now if someone wants to approve so we can get that into the novaclient release i'm pushing
13:22:17 mdbooth mriedem: Thanks.
13:25:02 mdbooth Incidentally, MySQL's repeatable read isolation level (which we use), is seriously obtuse when trying to handle concurrency. If I hadn't had to handle it when we were working on this code previously I would never have believe anybody would have intentionally created this behaviour. However, MySQL goes to great lengths to be broken.
13:25:27 mdbooth They apparently see value in it.
13:28:48 mriedem job security
13:29:43 dansmith kashyap: I'm entirely against (a) nova automatic scheduling based on perceived threats from vulnerabilities and (b) putting traits in placement about which vulnerabilities the host has, has been patched for, or has enabled/disabled. Hopefully that clears it up.
13:30:53 kashyap Sure, I don't care about these things much at all; I wanted to put in a missing thing in traits and move on.
13:31:10 kashyap dansmith: Also: you write "perceived threats" -- they're not.
13:31:30 sean-k-mooney the add the pcid and 1ghugepage tratis and drop the other form the patch
13:31:30 kashyap You can _choose_ to take a policy stance, as "this is out of scope". That's perfectly fine :-)
13:32:07 dansmith kashyap: if I am running a single-tenant cloud full of workloads I control, then the threats are entirely perceived
13:33:16 kashyap dansmith: Fair, that's one valid example. That's why I only said: should we consider "warn".
13:33:44 kashyap I also noted a few more valid cases where people willingly want to run on vulnerable hosts in my second reply on the 'openstack-security' thread.
13:33:49 dansmith we most certainly should not put traits in placement for the purposes of warning, but I also do not agree that nova should be doing the warning
13:34:49 coreycb mriedem: i wonder if they could work around it by running with python3
13:35:19 kashyap dansmith: Yeah, I agre with the former (not put them in traits). The latter is a 'policy' decision -- so I'm fine with whatever the consensus of the community is.
13:39:37 amorin hey all, can someone take a look at that: https://review.opendev.org/#/c/658362/
13:45:28 mdbooth coreycb: Are you talking about the eventlet thing?
13:45:51 coreycb mdbooth: ye
13:45:52 coreycb yes
13:46:05 mdbooth coreycb: Unfortunately not. We're hitting this in OSP15, which is py3 only.
13:46:26 sean-k-mooney coreycb: if you dont use uswign and run the nova-api via the console script it shoudl not have this issue
13:46:39 mdbooth However, IIRC there's another project also hitting it?
13:46:39 sean-k-mooney *uwsgi
13:46:41 coreycb mdbooth: ok. i just saw py2.7 in the traceback and that was my thought. sean-k-mooney: ok
13:46:51 mdbooth sean-k-mooney: Do you happen to remember which one it was?
13:47:03 sean-k-mooney monasca i think
13:47:26 mdbooth It's a library thing, anyway. It's nothing specific to what Nova's doing.
13:47:34 mdbooth It's just that eventlet + wsgi is bad.
13:49:55 sean-k-mooney as i said the workaound currently is to not run the nova api under wsgi and fall back to using the python nova-api command and fronting that with a load blancer like ha proxy
13:51:39 mdbooth I mentioned in the review that I don't think the lack of ability to cancel a thread is a showstopper in practice, and certainly not as big an issue as eventlet + wsgi.
13:51:48 sean-k-mooney ah it was mistral
13:51:49 kashyap sean-k-mooney: What does "monasca" even do? Okay, let me go look up
13:52:02 mdbooth sean-k-mooney: That sounds better.
13:52:05 sean-k-mooney monsaca is a montioring service
13:52:13 kashyap Ah, thx.
13:52:28 mriedem amorin: you should have a test
13:52:28 sean-k-mooney but i just rememebre it began with m just fount th lin in the mailing list post
13:53:31 sean-k-mooney but ya long story short as mdbooth said eventlet+wsgi is just a bad idea in general
13:53:50 amorin mriedem: ok
14:12:46 efried alex_xu: looking...
14:21:44 tssurya mriedem: oh cool thansk :)
14:23:53 aarents mdbooth: thanks for reviewing https://review.opendev.org/#/c/659054/, you catch the test logic, I will add a comment in test as suggested
14:31:40 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Follow up for I9200c2b18369da8b9194a8ccbbf6f1baf6a9613c https://review.opendev.org/659300
14:31:57 stephenfin bauzas: Time to merge https://review.opendev.org/#/c/555081/ ?
14:32:24 stephenfin bauzas: I've posted a follow-up to address some of the outstanding comments and issues
14:32:35 bauzas cool then, I'll review those :)
14:32:48 bauzas but yeah I feel we agreed on saying cool cool at the PTG
14:41:44 efried alex_xu: How does that play with https://review.opendev.org/#/c/645520/ ?
15:10:39 openstackgerrit Alexandre arents proposed openstack/nova master: Fix live-migration when glance image deleted https://review.opendev.org/659054
15:22:02 openstackgerrit Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362
15:24:52 openstackgerrit Merged openstack/nova-specs master: Standardize CPU resource tracking https://review.opendev.org/555081
15:30:47 openstackgerrit Matthew Booth proposed openstack/nova stable/stein: Fix retry of instance_update_and_get_original https://review.opendev.org/659317
15:31:36 openstackgerrit Matthew Booth proposed openstack/nova stable/rocky: Fix retry of instance_update_and_get_original https://review.opendev.org/659318
15:33:02 openstackgerrit Matthew Booth proposed openstack/nova stable/queens: Fix retry of instance_update_and_get_original https://review.opendev.org/659320
15:33:20 openstackgerrit Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362
15:37:23 amorin hey mriedem and all
15:37:29 amorin I added some unit tests there:
15:37:31 amorin https://review.opendev.org/#/c/658362
15:50:38 mriedem ack it's in the tab queue
16:29:24 melwitt mriedem: yes, should be ready. I just looked through the latest comments again and replied. and it looks like it hasn't fallen into merge conflict yet
16:33:06 mriedem ok
16:33:21 mriedem into the runway slot vat of acid it will be thrown
16:34:41 melwitt haha, thanks
16:39:45 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Delete allocations even if _confirm_resize raises (part 2) https://review.opendev.org/659338
16:39:45 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Add functional confirm_migration_error test https://review.opendev.org/658834
16:39:46 mriedem ganso: ^
16:44:40 ganso mriedem: thanks! will rebase on top
16:45:09 ganso mriedem: oh you already rebased
16:45:11 ganso mriedem: thanks!
16:45:30 imacdonn mriedem: pls add https://review.opendev.org/#/c/653065/ to your tab queue for revisit? I have a feeling that the issue with the functional tests has something to do with sqlalchemy sessions, or something, but may be off on the wrong track
16:45:47 imacdonn (if there's a sqlalchemy guru available for a consult, that might work too)
16:45:56 weshay kashyap ping.. looking to add the qemu-agent rpm to nova, I think that would be on a container and not bm.. not sure also wanted to verify w/ you which container that rpm should be installed on
16:46:23 weshay kashyap context is: nova.exception.QemuGuestAgentNotEnabled: QEMU guest agent is not enabled
16:46:31 weshay https://bugs.launchpad.net/tripleo/+bug/1824315
16:46:32 openstack Launchpad bug 1824315 in tripleo "periodic fedora28 standalone job failing at test_volume_boot_pattern" [Critical,In progress] - Assigned to Quique Llorente (quiquell)
16:47:21 weshay melwitt maybe you would know? ^
16:50:07 melwitt weshay: the qemu guest agent is something that would have to be installed on the guest, which is currently not done as part of any CI that I'm aware of
16:50:40 weshay melwitt ok.. some folks were saying the container but the bm makes more sense to me
16:50:54 melwitt that is, if you create a nova instance, you'd then have to install the qemu guest agent on that guest after it comes up
16:52:06 melwitt but I'm wondering why it should be needed
16:52:42 dansmith if they're trying to do something that requires it, like quiesce disks (I think)
16:53:04 melwitt weshay: that error message is just saying it's going to skip quiescing the instance bc the guest agent isn't present, but it shouldn't fail the job because of that I don't think
16:53:34 weshay melwitt it does not fail consistently, maybe 1/10 times
16:53:43 weshay maybe this is a red herring
16:54:08 dansmith if it's failing on the guest agent really not being there 1/10 times, I'd suspect the guest isn't booting fully 1/10 times or something like that
16:54:24 melwitt weshay: ok. I don't think this is the root cause of the failure because AFAICT it's just skipping that. it should say that every time I think

Earlier   Later