| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-05-15 | |||
| 12:17:17 | aarents | if not the qemu img disk have the size of backing file | |
| 12:17:51 | sean-k-mooney | so size in this case is the flavor root disk | |
| 12:17:59 | aarents | yes it is | |
| 12:18:55 | sean-k-mooney | well we are passing it like 10 lines above so it seams sane | |
| 12:21:31 | sean-k-mooney | but honestly i dont really know the full implication but at first glance it looks like an apporpiate fix for the bug you reported | |
| 12:24:08 | mdbooth | Could somebody hit https://review.opendev.org/#/c/658845/ for me, please? It has 1.5 +2s. It's the db retry bug. | |
| 12:24:23 | aarents | sean-k-mooney: ok thks, it fixs the issue in my environment | |
| 12:24:56 | mdbooth | mriedem: I merged the comment you asked for you at the request of stephenfin, btw. | |
| 12:25:17 | sean-k-mooney | aarents: it also passed ci so it dose not break the happy path although i dont think we have a function or tempest test that validates teh behavior | |
| 12:25:54 | mriedem | mdbooth: yeah looking | |
| 12:26:00 | mdbooth | mriedem: Thanks | |
| 12:26:04 | sean-k-mooney | mdbooth: oh this is the modifyicion of input parmaters thing you sent to the mailing list | |
| 12:26:11 | mdbooth | sean-k-mooney: Yeah. | |
| 12:27:17 | mriedem | sean-k-mooney: the sriov live migration blueprint isn't closed yet b/c surely there are docs updates to be made, right? | |
| 12:27:25 | mriedem | at least this https://docs.openstack.org/neutron/latest/admin/config-sriov.html#known-limitations | |
| 12:27:46 | mriedem | melwitt: are you ready for https://blueprints.launchpad.net/nova/+spec/count-quota-usage-from-placement to go back into a runway slot? | |
| 12:28:07 | sean-k-mooney | that in neutreon but yes i should go update that. | |
| 12:28:18 | mriedem | mdbooth: coreycb: btw, starlingx is saying since we merged the eventlet thing on stable/stein that they are broken ow | |
| 12:28:19 | mriedem | *now | |
| 12:28:21 | mriedem | so that's fun | |
| 12:28:38 | mriedem | https://bugs.launchpad.net/nova/+bug/1829062 | |
| 12:28:39 | openstack | Launchpad bug 1829062 in StarlingX "nova placement api non-responsive due to eventlet error" [Critical,New] - Assigned to Bruce Jones (brucej) | |
| 12:29:09 | sean-k-mooney | mriedem: dpending on how its run there were issues with some rabbitmq timeouts or somthing right | |
| 12:29:13 | mriedem | granted if starlingx moves to extracted placement in stein they won't have to deal with eventlet | |
| 12:29:16 | mdbooth | mriedem: Yeah, we're seeing similar downstream. However, I'm pretty sure we didn't. | |
| 12:29:58 | mdbooth | mriedem: Honestly what I really want to do is prioritise un-eventleting the multi-cell instance list code, and let them switch eventlet off. | |
| 12:30:37 | sean-k-mooney | ya althoeugh we have an open question of how to we kill long running task that will timeout due to a down cell | |
| 12:30:37 | mriedem | which is mel's series isn't it? | |
| 12:30:46 | sean-k-mooney | mriedem: yes | |
| 12:31:02 | mdbooth | mriedem: Let me clarify 'we didn't', btw: the eventlet band-aid thing didn't break it, it was already broken. | |
| 12:31:11 | mdbooth | I believe them that it's broken, though. | |
| 12:32:16 | mdbooth | mriedem: Yes. We discussed it at PTG and the feeling was that we'd address it later. However, I think that was when we believe we'd applied an effective band-aid already. | |
| 12:32:19 | sean-k-mooney | if we dont have a path forward with melwitt's serise we might want to consider reverting the scater gatehr stuff temporally and dropping eventlets | |
| 12:32:42 | mdbooth | sean-k-mooney: Also worthy of discussion. | |
| 12:36:41 | sean-k-mooney | so looks like https://review.opendev.org/#/c/657750/1 is the last patch in that series and it passing ci but i think we are currently stuck on the open question in https://review.opendev.org/#/c/650172/14 | |
| 12:37:49 | sean-k-mooney | is the startx issue seperate for teh amqp issue reported on the mailing list | |
| 12:38:14 | sean-k-mooney | http://lists.openstack.org/pipermail/openstack-discuss/2019-May/005822.html | |
| 12:42:38 | mdbooth | Thanks for the reminder, sean-k-mooney | |
| 12:54:46 | mriedem | tssurya: we need a novaclient release before osc can use the locked reason stuff https://review.opendev.org/#/c/659124/ | |
| 13:01:12 | mriedem | i'll propose a release | |
| 13:06:30 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Cap sphinx for py2 to match global requirements https://review.opendev.org/659202 | |
| 13:14:30 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Cap sphinx for py2 and drop keyring dependency https://review.opendev.org/659202 | |
| 13:19:04 | mriedem | the requirements-check job is happy with ^ now if someone wants to approve so we can get that into the novaclient release i'm pushing | |
| 13:22:17 | mdbooth | mriedem: Thanks. | |
| 13:25:02 | mdbooth | Incidentally, MySQL's repeatable read isolation level (which we use), is seriously obtuse when trying to handle concurrency. If I hadn't had to handle it when we were working on this code previously I would never have believe anybody would have intentionally created this behaviour. However, MySQL goes to great lengths to be broken. | |
| 13:25:27 | mdbooth | They apparently see value in it. | |
| 13:28:48 | mriedem | job security | |
| 13:29:43 | dansmith | kashyap: I'm entirely against (a) nova automatic scheduling based on perceived threats from vulnerabilities and (b) putting traits in placement about which vulnerabilities the host has, has been patched for, or has enabled/disabled. Hopefully that clears it up. | |
| 13:30:53 | kashyap | Sure, I don't care about these things much at all; I wanted to put in a missing thing in traits and move on. | |
| 13:31:10 | kashyap | dansmith: Also: you write "perceived threats" -- they're not. | |
| 13:31:30 | sean-k-mooney | the add the pcid and 1ghugepage tratis and drop the other form the patch | |
| 13:31:30 | kashyap | You can _choose_ to take a policy stance, as "this is out of scope". That's perfectly fine :-) | |
| 13:32:07 | dansmith | kashyap: if I am running a single-tenant cloud full of workloads I control, then the threats are entirely perceived | |
| 13:33:16 | kashyap | dansmith: Fair, that's one valid example. That's why I only said: should we consider "warn". | |
| 13:33:44 | kashyap | I also noted a few more valid cases where people willingly want to run on vulnerable hosts in my second reply on the 'openstack-security' thread. | |
| 13:33:49 | dansmith | we most certainly should not put traits in placement for the purposes of warning, but I also do not agree that nova should be doing the warning | |
| 13:34:49 | coreycb | mriedem: i wonder if they could work around it by running with python3 | |
| 13:35:19 | kashyap | dansmith: Yeah, I agre with the former (not put them in traits). The latter is a 'policy' decision -- so I'm fine with whatever the consensus of the community is. | |
| 13:39:37 | amorin | hey all, can someone take a look at that: https://review.opendev.org/#/c/658362/ | |
| 13:45:28 | mdbooth | coreycb: Are you talking about the eventlet thing? | |
| 13:45:51 | coreycb | mdbooth: ye | |
| 13:45:52 | coreycb | yes | |
| 13:46:05 | mdbooth | coreycb: Unfortunately not. We're hitting this in OSP15, which is py3 only. | |
| 13:46:26 | sean-k-mooney | coreycb: if you dont use uswign and run the nova-api via the console script it shoudl not have this issue | |
| 13:46:39 | mdbooth | However, IIRC there's another project also hitting it? | |
| 13:46:39 | sean-k-mooney | *uwsgi | |
| 13:46:41 | coreycb | mdbooth: ok. i just saw py2.7 in the traceback and that was my thought. sean-k-mooney: ok | |
| 13:46:51 | mdbooth | sean-k-mooney: Do you happen to remember which one it was? | |
| 13:47:03 | sean-k-mooney | monasca i think | |
| 13:47:26 | mdbooth | It's a library thing, anyway. It's nothing specific to what Nova's doing. | |
| 13:47:34 | mdbooth | It's just that eventlet + wsgi is bad. | |
| 13:49:55 | sean-k-mooney | as i said the workaound currently is to not run the nova api under wsgi and fall back to using the python nova-api command and fronting that with a load blancer like ha proxy | |
| 13:51:39 | mdbooth | I mentioned in the review that I don't think the lack of ability to cancel a thread is a showstopper in practice, and certainly not as big an issue as eventlet + wsgi. | |
| 13:51:48 | sean-k-mooney | ah it was mistral | |
| 13:51:49 | kashyap | sean-k-mooney: What does "monasca" even do? Okay, let me go look up | |
| 13:52:02 | mdbooth | sean-k-mooney: That sounds better. | |
| 13:52:05 | sean-k-mooney | monsaca is a montioring service | |
| 13:52:13 | kashyap | Ah, thx. | |
| 13:52:28 | mriedem | amorin: you should have a test | |
| 13:52:28 | sean-k-mooney | but i just rememebre it began with m just fount th lin in the mailing list post | |
| 13:53:31 | sean-k-mooney | but ya long story short as mdbooth said eventlet+wsgi is just a bad idea in general | |
| 13:53:50 | amorin | mriedem: ok | |
| 14:12:46 | efried | alex_xu: looking... | |
| 14:21:44 | tssurya | mriedem: oh cool thansk :) | |
| 14:23:53 | aarents | mdbooth: thanks for reviewing https://review.opendev.org/#/c/659054/, you catch the test logic, I will add a comment in test as suggested | |
| 14:31:40 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Follow up for I9200c2b18369da8b9194a8ccbbf6f1baf6a9613c https://review.opendev.org/659300 | |
| 14:31:57 | stephenfin | bauzas: Time to merge https://review.opendev.org/#/c/555081/ ? | |
| 14:32:24 | stephenfin | bauzas: I've posted a follow-up to address some of the outstanding comments and issues | |
| 14:32:35 | bauzas | cool then, I'll review those :) | |
| 14:32:48 | bauzas | but yeah I feel we agreed on saying cool cool at the PTG | |
| 14:41:44 | efried | alex_xu: How does that play with https://review.opendev.org/#/c/645520/ ? | |
| 15:10:39 | openstackgerrit | Alexandre arents proposed openstack/nova master: Fix live-migration when glance image deleted https://review.opendev.org/659054 | |
| 15:22:02 | openstackgerrit | Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362 | |
| 15:24:52 | openstackgerrit | Merged openstack/nova-specs master: Standardize CPU resource tracking https://review.opendev.org/555081 | |
| 15:30:47 | openstackgerrit | Matthew Booth proposed openstack/nova stable/stein: Fix retry of instance_update_and_get_original https://review.opendev.org/659317 | |
| 15:31:36 | openstackgerrit | Matthew Booth proposed openstack/nova stable/rocky: Fix retry of instance_update_and_get_original https://review.opendev.org/659318 | |
| 15:33:02 | openstackgerrit | Matthew Booth proposed openstack/nova stable/queens: Fix retry of instance_update_and_get_original https://review.opendev.org/659320 | |
| 15:33:20 | openstackgerrit | Arnaud Morin proposed openstack/nova master: Always Set dhcp_server in network_info https://review.opendev.org/658362 | |
| 15:37:23 | amorin | hey mriedem and all | |
| 15:37:29 | amorin | I added some unit tests there: | |