Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-05-15
13:25:02 mdbooth Incidentally, MySQL's repeatable read isolation level (which we use), is seriously obtuse when trying to handle concurrency. If I hadn't had to handle it when we were working on this code previously I would never have believe anybody would have intentionally created this behaviour. However, MySQL goes to great lengths to be broken.
13:25:27 mdbooth They apparently see value in it.
13:28:48 mriedem job security
13:29:43 dansmith kashyap: I'm entirely against (a) nova automatic scheduling based on perceived threats from vulnerabilities and (b) putting traits in placement about which vulnerabilities the host has, has been patched for, or has enabled/disabled. Hopefully that clears it up.
13:30:53 kashyap Sure, I don't care about these things much at all; I wanted to put in a missing thing in traits and move on.
13:31:10 kashyap dansmith: Also: you write "perceived threats" -- they're not.
13:31:30 sean-k-mooney the add the pcid and 1ghugepage tratis and drop the other form the patch
13:31:30 kashyap You can _choose_ to take a policy stance, as "this is out of scope". That's perfectly fine :-)
13:32:07 dansmith kashyap: if I am running a single-tenant cloud full of workloads I control, then the threats are entirely perceived
13:33:16 kashyap dansmith: Fair, that's one valid example. That's why I only said: should we consider "warn".
13:33:44 kashyap I also noted a few more valid cases where people willingly want to run on vulnerable hosts in my second reply on the 'openstack-security' thread.
13:33:49 dansmith we most certainly should not put traits in placement for the purposes of warning, but I also do not agree that nova should be doing the warning
13:34:49 coreycb mriedem: i wonder if they could work around it by running with python3
13:35:19 kashyap dansmith: Yeah, I agre with the former (not put them in traits). The latter is a 'policy' decision -- so I'm fine with whatever the consensus of the community is.
13:39:37 amorin hey all, can someone take a look at that: https://review.opendev.org/#/c/658362/
13:45:28 mdbooth coreycb: Are you talking about the eventlet thing?
13:45:51 coreycb mdbooth: ye
13:45:52 coreycb yes
13:46:05 mdbooth coreycb: Unfortunately not. We're hitting this in OSP15, which is py3 only.
13:46:26 sean-k-mooney coreycb: if you dont use uswign and run the nova-api via the console script it shoudl not have this issue
13:46:39 mdbooth However, IIRC there's another project also hitting it?
13:46:39 sean-k-mooney *uwsgi
13:46:41 coreycb mdbooth: ok. i just saw py2.7 in the traceback and that was my thought. sean-k-mooney: ok
13:46:51 mdbooth sean-k-mooney: Do you happen to remember which one it was?
13:47:03 sean-k-mooney monasca i think
13:47:26 mdbooth It's a library thing, anyway. It's nothing specific to what Nova's doing.
13:47:34 mdbooth It's just that eventlet + wsgi is bad.
13:49:55 sean-k-mooney as i said the workaound currently is to not run the nova api under wsgi and fall back to using the python nova-api command and fronting that with a load blancer like ha proxy
13:51:39 mdbooth I mentioned in the review that I don't think the lack of ability to cancel a thread is a showstopper in practice, and certainly not as big an issue as eventlet + wsgi.
13:51:48 sean-k-mooney ah it was mistral
13:51:49 kashyap sean-k-mooney: What does "monasca" even do? Okay, let me go look up
13:52:02 mdbooth sean-k-mooney: That sounds better.
13:52:05 sean-k-mooney monsaca is a montioring service
13:52:13 kashyap Ah, thx.
13:52:28 mriedem amorin: you should have a test
13:52:28 sean-k-mooney but i just rememebre it began with m just fount th lin in the mailing list post
13:53:31 sean-k-mooney but ya long story short as mdbooth said eventlet+wsgi is just a bad idea in general
13:53:50 amorin mriedem: ok
14:12:46 efried alex_xu: looking...
14:21:44 tssurya mriedem: oh cool thansk :)
14:23:53 aarents mdbooth: thanks for reviewing https://review.opendev.org/#/c/659054/, you catch the test logic, I will add a comment in test as suggested
14:31:40 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Follow up for I9200c2b18369da8b9194a8ccbbf6f1baf6a9613c https://review.opendev.org/659300
14:31:57 stephenfin bauzas: Time to merge https://review.opendev.org/#/c/555081/ ?
14:32:24 stephenfin bauzas: I've posted a follow-up to address some of the outstanding comments and issues
14:32:35 bauzas cool then, I'll review those :)
14:32:48 bauzas but yeah I feel we agreed on saying cool cool at the PTG
14:41:44 efried alex_xu: How does that play with https://review.opendev.org/#/c/645520/ ?
15:10:39 openstackgerrit Alexandre arents proposed openstack/nova master: Fix live-migration when glance image deleted https://review.opendev.org/659054
15:22:02 openstackgerrit Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362
15:24:52 openstackgerrit Merged openstack/nova-specs master: Standardize CPU resource tracking https://review.opendev.org/555081
15:30:47 openstackgerrit Matthew Booth proposed openstack/nova stable/stein: Fix retry of instance_update_and_get_original https://review.opendev.org/659317
15:31:36 openstackgerrit Matthew Booth proposed openstack/nova stable/rocky: Fix retry of instance_update_and_get_original https://review.opendev.org/659318
15:33:02 openstackgerrit Matthew Booth proposed openstack/nova stable/queens: Fix retry of instance_update_and_get_original https://review.opendev.org/659320
15:33:20 openstackgerrit Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362
15:37:23 amorin hey mriedem and all
15:37:29 amorin I added some unit tests there:
15:37:31 amorin https://review.opendev.org/#/c/658362
15:50:38 mriedem ack it's in the tab queue
16:29:24 melwitt mriedem: yes, should be ready. I just looked through the latest comments again and replied. and it looks like it hasn't fallen into merge conflict yet
16:33:06 mriedem ok
16:33:21 mriedem into the runway slot vat of acid it will be thrown
16:34:41 melwitt haha, thanks
16:39:45 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Delete allocations even if _confirm_resize raises (part 2) https://review.opendev.org/659338
16:39:45 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Add functional confirm_migration_error test https://review.opendev.org/658834
16:39:46 mriedem ganso: ^
16:44:40 ganso mriedem: thanks! will rebase on top
16:45:09 ganso mriedem: oh you already rebased
16:45:11 ganso mriedem: thanks!
16:45:30 imacdonn mriedem: pls add https://review.opendev.org/#/c/653065/ to your tab queue for revisit? I have a feeling that the issue with the functional tests has something to do with sqlalchemy sessions, or something, but may be off on the wrong track
16:45:47 imacdonn (if there's a sqlalchemy guru available for a consult, that might work too)
16:45:56 weshay kashyap ping.. looking to add the qemu-agent rpm to nova, I think that would be on a container and not bm.. not sure also wanted to verify w/ you which container that rpm should be installed on
16:46:23 weshay kashyap context is: nova.exception.QemuGuestAgentNotEnabled: QEMU guest agent is not enabled
16:46:31 weshay https://bugs.launchpad.net/tripleo/+bug/1824315
16:46:32 openstack Launchpad bug 1824315 in tripleo "periodic fedora28 standalone job failing at test_volume_boot_pattern" [Critical,In progress] - Assigned to Quique Llorente (quiquell)
16:47:21 weshay melwitt maybe you would know? ^
16:50:07 melwitt weshay: the qemu guest agent is something that would have to be installed on the guest, which is currently not done as part of any CI that I'm aware of
16:50:40 weshay melwitt ok.. some folks were saying the container but the bm makes more sense to me
16:50:54 melwitt that is, if you create a nova instance, you'd then have to install the qemu guest agent on that guest after it comes up
16:52:06 melwitt but I'm wondering why it should be needed
16:52:42 dansmith if they're trying to do something that requires it, like quiesce disks (I think)
16:53:04 melwitt weshay: that error message is just saying it's going to skip quiescing the instance bc the guest agent isn't present, but it shouldn't fail the job because of that I don't think
16:53:34 weshay melwitt it does not fail consistently, maybe 1/10 times
16:53:43 weshay maybe this is a red herring
16:54:08 dansmith if it's failing on the guest agent really not being there 1/10 times, I'd suspect the guest isn't booting fully 1/10 times or something like that
16:54:24 melwitt weshay: ok. I don't think this is the root cause of the failure because AFAICT it's just skipping that. it should say that every time I think
16:55:09 melwitt dansmith: I don't think any CI installs the guest agent, at least I was not aware of any
16:55:26 openstackgerrit Merged openstack/python-novaclient master: Cap sphinx for py2 and drop keyring dependency https://review.opendev.org/659202
16:55:26 dansmith melwitt: if they are using a fedora guest...
16:55:44 melwitt dansmith: fedora guest has it by default?
16:56:01 weshay dansmith ya.. we this in fedora deployments.. but w/ cirros guest
16:56:07 dansmith melwitt: no idea, I'm just saying, different from a cirros guest
16:56:09 dansmith okay
16:56:26 dansmith the bug shows a big stack trace.. melwitt you think we're logging that trace just while checking the skip?
16:56:50 melwitt oh. yeah, anecdotally I've not had the qemu guest agent come with any cloud image, like centos for example I've used
16:56:59 melwitt dansmith: I think so, opening up the code to confirm
16:57:31 dansmith okay we should not be logging a trace for a check thing, amirite?
16:58:04 melwitt should not, no. I see a LOG.info for it but no re-raise, so not yet sure what's going on
16:58:26 dansmith that's why I'm wondering if we're actually leaking the exception out and not just skipping
16:58:58 melwitt yeah, let's see, the trace is happening in the driver, so that must be where
16:59:08 melwitt there's no re-raise in compute/api, I meant

Earlier   Later