| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-04 | |||
| 18:53:34 | cdent | aspiers: https://bugs.launchpad.net/nova/+bug/1818560 | |
| 18:53:35 | openstack | Launchpad bug 1818560 in OpenStack Compute (nova) "Nova's use of the placement database fixture from test_report_client doesn't register opts" [Critical,Confirmed] | |
| 18:53:50 | cdent | in case I wasn't clear there, the fix is on the placement side | |
| 19:15:06 | cdent | aspiers: there are two levels of problem here | |
| 19:16:43 | cdent | requires fixes in both placement and nova, mabye | |
| 19:16:51 | cdent | still looking to try to figure out the cleanest way | |
| 19:19:24 | aspiers | cdent: awesome, thanks! | |
| 19:19:45 | mriedem | dansmith: am i missing what you're asking for here? https://review.openstack.org/#/c/636412/1/nova/scheduler/client/report.py@1758 | |
| 19:20:56 | dansmith | mriedem: ah, sorry I missed that the original change was already including one direction | |
| 19:21:39 | mriedem | i love one direction | |
| 19:21:46 | aspiers | X-D | |
| 19:22:00 | cfriesen | mriedem: in the case of resize, I think the fact that sometimes it *can* go through (if the image is already cached, or we're using "raw") means that we can't fail the operation due to image inactivity. | |
| 19:22:17 | mriedem | cfriesen: sure | |
| 19:22:22 | mriedem | not a huge sticking point for me | |
| 19:25:08 | openstackgerrit | Chris Friesen proposed openstack/nova master: Flavor extra spec and image properties validation https://review.openstack.org/620706 | |
| 19:25:09 | openstackgerrit | Chris Friesen proposed openstack/nova master: Improve existing flavor and image metadata validation https://review.openstack.org/640841 | |
| 19:26:44 | openstackgerrit | Chris Friesen proposed openstack/nova master: Flavor extra spec and image properties validation from API https://review.openstack.org/620706 | |
| 19:27:06 | cfriesen | figured I'd differentiate the two titles a bit better | |
| 19:27:16 | eandersson | nova-hostname (from metadata) is set by dhcp_domain right? | |
| 19:27:25 | eandersson | That is deprecated | |
| 19:27:30 | eandersson | What is supposed to populate that field? | |
| 19:28:30 | mriedem | eandersson: you just stepped into something | |
| 19:29:40 | mriedem | eandersson: https://bugs.launchpad.net/nova/+bug/1698010 | |
| 19:29:42 | openstack | Launchpad bug 1698010 in OpenStack Compute (nova) "neutron-based instances should not use the nova-network 'dhcp_domain' option" [High,In progress] - Assigned to Stephen Finucane (stephenfinucane) | |
| 19:30:30 | sean-k-mooney | eandersson: if you dont have designate deployed i dont think neutron will populate that value. | |
| 19:30:43 | sean-k-mooney | ignoring the nova bug | |
| 19:32:11 | melwitt | dansmith, mriedem: I updated the user_id patch to make the field non-nullable. unit tests are all broken and some/all of the next patch need to be squashed in, but the next patch passes grenade/tempest. I hoped for a sanity check from you before I go ahead and do any commit squashing and unit test fixing https://review.openstack.org/633350 | |
| 19:32:44 | cdent | aspiers: you should be proud, you've found a hole in the space time continuium, I'm looping back on myself | |
| 19:33:07 | aspiers | cdent: just don't destroy the universe in the process | |
| 19:35:05 | cdent | efried: are you back yet? I think I might need to consult with you on this | |
| 19:36:15 | cdent | the issue I'm trying to contend with this that at https://github.com/openstack/nova/blob/master/nova/tests/functional/test_report_client.py#L124 that CONF is a nova conf, not a placement CONF | |
| 19:36:33 | cdent | so now that we've added a new conf item to placement things are awry | |
| 19:40:39 | openstackgerrit | Adam Spiers proposed openstack/nova master: Parse |
|
| 19:42:43 | mriedem | melwitt: it looks like what i'd expect | |
| 19:43:09 | melwitt | mriedem: thank ya | |
| 19:44:20 | mriedem | jackding: are you able to rebase this? https://review.openstack.org/#/c/603844/ | |
| 19:44:28 | mriedem | jaypipes: ^ i think i answered your question in there | |
| 19:45:30 | dansmith | melwitt: I gotta run to an appt but will try to look when I get back | |
| 19:45:56 | melwitt | dansmith: ok, thank you | |
| 19:46:43 | jackding | mriedem: will do | |
| 19:49:47 | eandersson | thanks mriedem sean-k-mooney | |
| 19:50:05 | cfriesen | jaypipes: FYI, this was based on an instance seen in the wild. | |
| 19:51:26 | cfriesen | jaypipes: (re: your question in jackding's review) | |
| 19:53:41 | openstackgerrit | Jack Ding proposed openstack/nova master: Correct instance port binding for rebuilds https://review.openstack.org/603844 | |
| 19:54:13 | jackding | mriedem: rebase done. Thanks | |
| 19:56:11 | mriedem | cfriesen: real soft -1 on https://review.openstack.org/#/c/640841/ | |
| 19:56:16 | mriedem | just update an existing api unit test and i'm +2 | |
| 19:56:50 | cfriesen | cool, will respin | |
| 20:00:05 | jaypipes | cfriesen: that's why I didn't -1 it... | |
| 20:00:16 | jaypipes | cfriesen: I wasn't 100% sure about it. | |
| 20:04:05 | jaypipes | jackding: I've +2'd 603844 since mriedem was +2 on it previously. | |
| 20:08:08 | eandersson | Something got a lot more expensive for us after moving to Rocky | |
| 20:08:19 | eandersson | It looks like Neutron is getting a lot more calls | |
| 20:08:38 | eandersson | but difficult to say what is causing cpu load to increase so drastically | |
| 20:09:01 | eandersson | We upgraded Nova, Neutron and Cinder | |
| 20:09:13 | eandersson | and CPU load has gone up 5x | |
| 20:09:35 | eandersson | Anyone seen this? | |
| 20:09:46 | sean-k-mooney | eandersson: are you using OSP? | |
| 20:09:54 | eandersson | What is OSP? | |
| 20:10:05 | sean-k-mooney | redhat openstack plathform | |
| 20:10:10 | eandersson | nah | |
| 20:10:33 | eandersson | Running something closer to Kolla | |
| 20:10:50 | sean-k-mooney | ok we had a perfromce issue awhile ago that was related to not enableing memcahce to cache keystone auth tokens the slowed everything down | |
| 20:14:01 | eandersson | We do have that enabled | |
| 20:14:04 | openstackgerrit | Chris Dent proposed openstack/nova master: Use a placement conf when testing report client https://review.openstack.org/640853 | |
| 20:14:08 | eandersson | Assuming you don't need that con computes right? | |
| 20:14:40 | sean-k-mooney | the compute nodes do hit the neutron and cinder apis | |
| 20:15:17 | sean-k-mooney | i dont know if you need/should configure caching on those too | |
| 20:15:27 | cdent | aspiers: https://review.openstack.org/#/q/topic:bug/1818560 should fix it. mriedem: that's a gate blocker fix for functional/test_report_client | |
| 20:16:52 | mriedem | is that just preventing https://review.openstack.org/#/c/538498/ from merging or are all nova changes now broken? | |
| 20:16:59 | cdent | mriedem: I think all | |
| 20:17:08 | mriedem | what changed/regressed? | |
| 20:17:26 | cdent | the fact that nova's conf was being used to run a placement wsgi app was revealed | |
| 20:17:35 | cdent | by the fact that we added a config on the placement side | |
| 20:18:14 | mriedem | link to the placement change? | |
| 20:18:15 | cdent | so it was going to happen as soon as that divergence happened and it just so happens it was today | |
| 20:18:37 | cdent | mriedem: it's linked from both commit messages at the topic above | |
| 20:18:40 | mriedem | https://review.openstack.org/#/c/619050/ | |
| 20:18:45 | mriedem | ok i was looking at the bug | |
| 20:19:13 | cdent | the bug description wasn't quite right, because I was trying to be hasty | |
| 20:23:24 | cdent | I've udpated the title and description | |
| 20:23:59 | efried | cdent: I'm back now. How can I help? | |
| 20:24:26 | cdent | efried: I figured it out, but if you could review https://review.openstack.org/#/q/topic:bug/1818560 to confirm I figured it out the right way, that would be good | |
| 20:25:55 | efried | ... | |
| 20:28:39 | mriedem | cdent: comments inline but if it passes it passes | |
| 20:29:05 | mriedem | we likely need to talk about test coverage for the externally-used placement fixture | |
| 20:29:12 | mriedem | *at some point | |
| 20:30:00 | cdent | yes | |
| 20:30:10 | cdent | but I figured at this stage of the game it was better to just fix it | |
| 20:30:33 | mriedem | yeah i agree | |
| 20:30:44 | openstackgerrit | Chris Friesen proposed openstack/nova master: Improve existing flavor and image metadata validation https://review.openstack.org/640841 | |
| 20:34:24 | sean-k-mooney | mriedem: cdent i dont know if we could somehow run the nova functional tests on change to the placement repo or perhaps just that fixture in the placmeent repo? | |
| 20:34:36 | efried | cdent: ugh, I had wanted to get rid of either PlacementFixture or PlacementDirect, and now we've made them inextricably interdependent. | |
| 20:35:06 | cdent | efried: we already had both and we're trying to make a quick fix | |
| 20:35:16 | cdent | that doesn't remove the possibility of fixing it better late | |
| 20:35:25 | efried | yeah, I get it. | |
| 20:35:37 | cdent | I personally think we should get rid of PlacementDirect, probably by exposing the intercept functionality in PlacementFixture | |
| 20:35:40 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Flavor extra spec and image properties validation from API https://review.openstack.org/620706 | |
| 20:36:01 | cdent | PlacementDirect is for a _very_ specific thing, in which a placement conf (a real one) is expected to exist | |
| 20:36:22 | cdent | however I have seen zero evidence that PlacementDirect ever should have existed, so I'm not inclined to put a lot of energy into it | |