Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-05-15
13:31:30 sean-k-mooney the add the pcid and 1ghugepage tratis and drop the other form the patch
13:31:30 kashyap You can _choose_ to take a policy stance, as "this is out of scope". That's perfectly fine :-)
13:32:07 dansmith kashyap: if I am running a single-tenant cloud full of workloads I control, then the threats are entirely perceived
13:33:16 kashyap dansmith: Fair, that's one valid example. That's why I only said: should we consider "warn".
13:33:44 kashyap I also noted a few more valid cases where people willingly want to run on vulnerable hosts in my second reply on the 'openstack-security' thread.
13:33:49 dansmith we most certainly should not put traits in placement for the purposes of warning, but I also do not agree that nova should be doing the warning
13:34:49 coreycb mriedem: i wonder if they could work around it by running with python3
13:35:19 kashyap dansmith: Yeah, I agre with the former (not put them in traits). The latter is a 'policy' decision -- so I'm fine with whatever the consensus of the community is.
13:39:37 amorin hey all, can someone take a look at that: https://review.opendev.org/#/c/658362/
13:45:28 mdbooth coreycb: Are you talking about the eventlet thing?
13:45:51 coreycb mdbooth: ye
13:45:52 coreycb yes
13:46:05 mdbooth coreycb: Unfortunately not. We're hitting this in OSP15, which is py3 only.
13:46:26 sean-k-mooney coreycb: if you dont use uswign and run the nova-api via the console script it shoudl not have this issue
13:46:39 mdbooth However, IIRC there's another project also hitting it?
13:46:39 sean-k-mooney *uwsgi
13:46:41 coreycb mdbooth: ok. i just saw py2.7 in the traceback and that was my thought. sean-k-mooney: ok
13:46:51 mdbooth sean-k-mooney: Do you happen to remember which one it was?
13:47:03 sean-k-mooney monasca i think
13:47:26 mdbooth It's a library thing, anyway. It's nothing specific to what Nova's doing.
13:47:34 mdbooth It's just that eventlet + wsgi is bad.
13:49:55 sean-k-mooney as i said the workaound currently is to not run the nova api under wsgi and fall back to using the python nova-api command and fronting that with a load blancer like ha proxy
13:51:39 mdbooth I mentioned in the review that I don't think the lack of ability to cancel a thread is a showstopper in practice, and certainly not as big an issue as eventlet + wsgi.
13:51:48 sean-k-mooney ah it was mistral
13:51:49 kashyap sean-k-mooney: What does "monasca" even do? Okay, let me go look up
13:52:02 mdbooth sean-k-mooney: That sounds better.
13:52:05 sean-k-mooney monsaca is a montioring service
13:52:13 kashyap Ah, thx.
13:52:28 mriedem amorin: you should have a test
13:52:28 sean-k-mooney but i just rememebre it began with m just fount th lin in the mailing list post
13:53:31 sean-k-mooney but ya long story short as mdbooth said eventlet+wsgi is just a bad idea in general
13:53:50 amorin mriedem: ok
14:12:46 efried alex_xu: looking...
14:21:44 tssurya mriedem: oh cool thansk :)
14:23:53 aarents mdbooth: thanks for reviewing https://review.opendev.org/#/c/659054/, you catch the test logic, I will add a comment in test as suggested
14:31:40 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Follow up for I9200c2b18369da8b9194a8ccbbf6f1baf6a9613c https://review.opendev.org/659300
14:31:57 stephenfin bauzas: Time to merge https://review.opendev.org/#/c/555081/ ?
14:32:24 stephenfin bauzas: I've posted a follow-up to address some of the outstanding comments and issues
14:32:35 bauzas cool then, I'll review those :)
14:32:48 bauzas but yeah I feel we agreed on saying cool cool at the PTG
14:41:44 efried alex_xu: How does that play with https://review.opendev.org/#/c/645520/ ?
15:10:39 openstackgerrit Alexandre arents proposed openstack/nova master: Fix live-migration when glance image deleted https://review.opendev.org/659054
15:22:02 openstackgerrit Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362
15:24:52 openstackgerrit Merged openstack/nova-specs master: Standardize CPU resource tracking https://review.opendev.org/555081
15:30:47 openstackgerrit Matthew Booth proposed openstack/nova stable/stein: Fix retry of instance_update_and_get_original https://review.opendev.org/659317
15:31:36 openstackgerrit Matthew Booth proposed openstack/nova stable/rocky: Fix retry of instance_update_and_get_original https://review.opendev.org/659318
15:33:02 openstackgerrit Matthew Booth proposed openstack/nova stable/queens: Fix retry of instance_update_and_get_original https://review.opendev.org/659320
15:33:20 openstackgerrit Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362
15:37:23 amorin hey mriedem and all
15:37:29 amorin I added some unit tests there:
15:37:31 amorin https://review.opendev.org/#/c/658362
15:50:38 mriedem ack it's in the tab queue
16:29:24 melwitt mriedem: yes, should be ready. I just looked through the latest comments again and replied. and it looks like it hasn't fallen into merge conflict yet
16:33:06 mriedem ok
16:33:21 mriedem into the runway slot vat of acid it will be thrown
16:34:41 melwitt haha, thanks
16:39:45 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Delete allocations even if _confirm_resize raises (part 2) https://review.opendev.org/659338
16:39:45 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Add functional confirm_migration_error test https://review.opendev.org/658834
16:39:46 mriedem ganso: ^
16:44:40 ganso mriedem: thanks! will rebase on top
16:45:09 ganso mriedem: oh you already rebased
16:45:11 ganso mriedem: thanks!
16:45:30 imacdonn mriedem: pls add https://review.opendev.org/#/c/653065/ to your tab queue for revisit? I have a feeling that the issue with the functional tests has something to do with sqlalchemy sessions, or something, but may be off on the wrong track
16:45:47 imacdonn (if there's a sqlalchemy guru available for a consult, that might work too)
16:45:56 weshay kashyap ping.. looking to add the qemu-agent rpm to nova, I think that would be on a container and not bm.. not sure also wanted to verify w/ you which container that rpm should be installed on
16:46:23 weshay kashyap context is: nova.exception.QemuGuestAgentNotEnabled: QEMU guest agent is not enabled
16:46:31 weshay https://bugs.launchpad.net/tripleo/+bug/1824315
16:46:32 openstack Launchpad bug 1824315 in tripleo "periodic fedora28 standalone job failing at test_volume_boot_pattern" [Critical,In progress] - Assigned to Quique Llorente (quiquell)
16:47:21 weshay melwitt maybe you would know? ^
16:50:07 melwitt weshay: the qemu guest agent is something that would have to be installed on the guest, which is currently not done as part of any CI that I'm aware of
16:50:40 weshay melwitt ok.. some folks were saying the container but the bm makes more sense to me
16:50:54 melwitt that is, if you create a nova instance, you'd then have to install the qemu guest agent on that guest after it comes up
16:52:06 melwitt but I'm wondering why it should be needed
16:52:42 dansmith if they're trying to do something that requires it, like quiesce disks (I think)
16:53:04 melwitt weshay: that error message is just saying it's going to skip quiescing the instance bc the guest agent isn't present, but it shouldn't fail the job because of that I don't think
16:53:34 weshay melwitt it does not fail consistently, maybe 1/10 times
16:53:43 weshay maybe this is a red herring
16:54:08 dansmith if it's failing on the guest agent really not being there 1/10 times, I'd suspect the guest isn't booting fully 1/10 times or something like that
16:54:24 melwitt weshay: ok. I don't think this is the root cause of the failure because AFAICT it's just skipping that. it should say that every time I think
16:55:09 melwitt dansmith: I don't think any CI installs the guest agent, at least I was not aware of any
16:55:26 openstackgerrit Merged openstack/python-novaclient master: Cap sphinx for py2 and drop keyring dependency https://review.opendev.org/659202
16:55:26 dansmith melwitt: if they are using a fedora guest...
16:55:44 melwitt dansmith: fedora guest has it by default?
16:56:01 weshay dansmith ya.. we this in fedora deployments.. but w/ cirros guest
16:56:07 dansmith melwitt: no idea, I'm just saying, different from a cirros guest
16:56:09 dansmith okay
16:56:26 dansmith the bug shows a big stack trace.. melwitt you think we're logging that trace just while checking the skip?
16:56:50 melwitt oh. yeah, anecdotally I've not had the qemu guest agent come with any cloud image, like centos for example I've used
16:56:59 melwitt dansmith: I think so, opening up the code to confirm
16:57:31 dansmith okay we should not be logging a trace for a check thing, amirite?
16:58:04 melwitt should not, no. I see a LOG.info for it but no re-raise, so not yet sure what's going on
16:58:26 dansmith that's why I'm wondering if we're actually leaking the exception out and not just skipping
16:58:58 melwitt yeah, let's see, the trace is happening in the driver, so that must be where
16:59:08 melwitt there's no re-raise in compute/api, I meant
17:06:41 cdent efried: is ocata nova still an active concern? there's a logging buglet in the reasource tracker + report client that can be pretty misleading but the issue goes away with pike (or maybe queens)
17:09:19 cdent (goes away with pike, just checked)
17:16:29 melwitt weshay: on your latest comment on LP, the log shows the code continuing on to do the volume snapshot http://logs.openstack.org/84/655684/2/check/tripleo-ci-fedora-28-standalone/eda3002/logs/undercloud/var/log/containers/nova/nova-api.log.txt.gz#_2019-05-13_16_03_50_120 so I think that trace is unrelated
17:17:03 melwitt weshay: I don't yet find how that trace is getting logged (it shouldn't be logged), so I'm still looking for how that's happening
17:17:34 weshay k k.. thanks.. so melwitt we're saying it's a red herring and the problem is elsewhere right?
17:17:43 weshay confirming that I guess

Earlier   Later