| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-04 | |||
| 19:32:44 | cdent | aspiers: you should be proud, you've found a hole in the space time continuium, I'm looping back on myself | |
| 19:33:07 | aspiers | cdent: just don't destroy the universe in the process | |
| 19:35:05 | cdent | efried: are you back yet? I think I might need to consult with you on this | |
| 19:36:15 | cdent | the issue I'm trying to contend with this that at https://github.com/openstack/nova/blob/master/nova/tests/functional/test_report_client.py#L124 that CONF is a nova conf, not a placement CONF | |
| 19:36:33 | cdent | so now that we've added a new conf item to placement things are awry | |
| 19:40:39 | openstackgerrit | Adam Spiers proposed openstack/nova master: Parse |
|
| 19:42:43 | mriedem | melwitt: it looks like what i'd expect | |
| 19:43:09 | melwitt | mriedem: thank ya | |
| 19:44:20 | mriedem | jackding: are you able to rebase this? https://review.openstack.org/#/c/603844/ | |
| 19:44:28 | mriedem | jaypipes: ^ i think i answered your question in there | |
| 19:45:30 | dansmith | melwitt: I gotta run to an appt but will try to look when I get back | |
| 19:45:56 | melwitt | dansmith: ok, thank you | |
| 19:46:43 | jackding | mriedem: will do | |
| 19:49:47 | eandersson | thanks mriedem sean-k-mooney | |
| 19:50:05 | cfriesen | jaypipes: FYI, this was based on an instance seen in the wild. | |
| 19:51:26 | cfriesen | jaypipes: (re: your question in jackding's review) | |
| 19:53:41 | openstackgerrit | Jack Ding proposed openstack/nova master: Correct instance port binding for rebuilds https://review.openstack.org/603844 | |
| 19:54:13 | jackding | mriedem: rebase done. Thanks | |
| 19:56:11 | mriedem | cfriesen: real soft -1 on https://review.openstack.org/#/c/640841/ | |
| 19:56:16 | mriedem | just update an existing api unit test and i'm +2 | |
| 19:56:50 | cfriesen | cool, will respin | |
| 20:00:05 | jaypipes | cfriesen: that's why I didn't -1 it... | |
| 20:00:16 | jaypipes | cfriesen: I wasn't 100% sure about it. | |
| 20:04:05 | jaypipes | jackding: I've +2'd 603844 since mriedem was +2 on it previously. | |
| 20:08:08 | eandersson | Something got a lot more expensive for us after moving to Rocky | |
| 20:08:19 | eandersson | It looks like Neutron is getting a lot more calls | |
| 20:08:38 | eandersson | but difficult to say what is causing cpu load to increase so drastically | |
| 20:09:01 | eandersson | We upgraded Nova, Neutron and Cinder | |
| 20:09:13 | eandersson | and CPU load has gone up 5x | |
| 20:09:35 | eandersson | Anyone seen this? | |
| 20:09:46 | sean-k-mooney | eandersson: are you using OSP? | |
| 20:09:54 | eandersson | What is OSP? | |
| 20:10:05 | sean-k-mooney | redhat openstack plathform | |
| 20:10:10 | eandersson | nah | |
| 20:10:33 | eandersson | Running something closer to Kolla | |
| 20:10:50 | sean-k-mooney | ok we had a perfromce issue awhile ago that was related to not enableing memcahce to cache keystone auth tokens the slowed everything down | |
| 20:14:01 | eandersson | We do have that enabled | |
| 20:14:04 | openstackgerrit | Chris Dent proposed openstack/nova master: Use a placement conf when testing report client https://review.openstack.org/640853 | |
| 20:14:08 | eandersson | Assuming you don't need that con computes right? | |
| 20:14:40 | sean-k-mooney | the compute nodes do hit the neutron and cinder apis | |
| 20:15:17 | sean-k-mooney | i dont know if you need/should configure caching on those too | |
| 20:15:27 | cdent | aspiers: https://review.openstack.org/#/q/topic:bug/1818560 should fix it. mriedem: that's a gate blocker fix for functional/test_report_client | |
| 20:16:52 | mriedem | is that just preventing https://review.openstack.org/#/c/538498/ from merging or are all nova changes now broken? | |
| 20:16:59 | cdent | mriedem: I think all | |
| 20:17:08 | mriedem | what changed/regressed? | |
| 20:17:26 | cdent | the fact that nova's conf was being used to run a placement wsgi app was revealed | |
| 20:17:35 | cdent | by the fact that we added a config on the placement side | |
| 20:18:14 | mriedem | link to the placement change? | |
| 20:18:15 | cdent | so it was going to happen as soon as that divergence happened and it just so happens it was today | |
| 20:18:37 | cdent | mriedem: it's linked from both commit messages at the topic above | |
| 20:18:40 | mriedem | https://review.openstack.org/#/c/619050/ | |
| 20:18:45 | mriedem | ok i was looking at the bug | |
| 20:19:13 | cdent | the bug description wasn't quite right, because I was trying to be hasty | |
| 20:23:24 | cdent | I've udpated the title and description | |
| 20:23:59 | efried | cdent: I'm back now. How can I help? | |
| 20:24:26 | cdent | efried: I figured it out, but if you could review https://review.openstack.org/#/q/topic:bug/1818560 to confirm I figured it out the right way, that would be good | |
| 20:25:55 | efried | ... | |
| 20:28:39 | mriedem | cdent: comments inline but if it passes it passes | |
| 20:29:05 | mriedem | we likely need to talk about test coverage for the externally-used placement fixture | |
| 20:29:12 | mriedem | *at some point | |
| 20:30:00 | cdent | yes | |
| 20:30:10 | cdent | but I figured at this stage of the game it was better to just fix it | |
| 20:30:33 | mriedem | yeah i agree | |
| 20:30:44 | openstackgerrit | Chris Friesen proposed openstack/nova master: Improve existing flavor and image metadata validation https://review.openstack.org/640841 | |
| 20:34:24 | sean-k-mooney | mriedem: cdent i dont know if we could somehow run the nova functional tests on change to the placement repo or perhaps just that fixture in the placmeent repo? | |
| 20:34:36 | efried | cdent: ugh, I had wanted to get rid of either PlacementFixture or PlacementDirect, and now we've made them inextricably interdependent. | |
| 20:35:06 | cdent | efried: we already had both and we're trying to make a quick fix | |
| 20:35:16 | cdent | that doesn't remove the possibility of fixing it better late | |
| 20:35:25 | efried | yeah, I get it. | |
| 20:35:37 | cdent | I personally think we should get rid of PlacementDirect, probably by exposing the intercept functionality in PlacementFixture | |
| 20:35:40 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Flavor extra spec and image properties validation from API https://review.openstack.org/620706 | |
| 20:36:01 | cdent | PlacementDirect is for a _very_ specific thing, in which a placement conf (a real one) is expected to exist | |
| 20:36:22 | cdent | however I have seen zero evidence that PlacementDirect ever should have existed, so I'm not inclined to put a lot of energy into it | |
| 20:36:24 | cdent | (for now) | |
| 20:37:01 | cdent | s/get rid of/get rid of (on the nova side)/ | |
| 20:37:38 | sean-k-mooney | by the way i assume the current lag in jobs starting in the gate is jsut the usual 3 days to feature freeze lag? | |
| 20:38:09 | cfriesen | no, zuul has a personal grudge | |
| 20:38:27 | melwitt | speaking of test coverage, mnaser: how goes it with the OSA job patch? https://review.openstack.org/635852 | |
| 20:38:52 | mriedem | btw http://status.openstack.org/elastic-recheck/data/integrated_gate.html "Generated at: 2019-02-08" so we don't know the true status of the gate right now | |
| 20:39:00 | mriedem | clark is investigating | |
| 20:39:05 | sean-k-mooney | cfriesen: :) first jenkins and now zuul well at least the ci is consitnt | |
| 20:39:20 | mnaser | melwitt: that test should be working and pulling down nova | |
| 20:39:45 | mnaser | we could probably add it as non voting | |
| 20:39:45 | mnaser | i need to find time to integrate os_placement into the main tests/roles but | |
| 20:39:55 | mnaser | a lot of our cores were laid off so we're struggling in osa land </3 | |
| 20:40:02 | melwitt | oh :( | |
| 20:40:07 | melwitt | I'm sorry to hear that | |
| 20:40:24 | cfriesen | This is a standalone disk performance optimization patch...has +1 from mdbooth. https://review.openstack.org/#/c/616692/ | |
| 20:40:37 | mnaser | yeah, so just trying to keep things afloat, the placement role is ready but ill make sure we get a job that continues to deploy using in-repo placement, and another one that deploys using out-of-repo | |
| 20:41:09 | melwitt | mnaser: ok, I wasn't sure what the job in that patch is missing/why WIP | |
| 20:41:56 | mnaser | melwitt: i wanted to leave it that way in case someone was to make an OSA change that starts deploying with out-of-repo by default and then the job becomes redundant | |
| 20:42:37 | mnaser | i need to put sometime to add an extra tox target to deploy with in-repo placement, which will ensure we don't accidentally start testing from out-of-tree when we switch the default | |
| 20:43:04 | melwitt | oh, I see | |
| 20:44:36 | mnaser | i'll get around that eventually | |
| 20:44:55 | mnaser | (and i can help someone if they want to help do it as well) | |
| 20:45:19 | melwitt | ok, and that's different than the TODO here? https://review.openstack.org/#/c/635852/1/.zuul.yaml@211 | |
| 20:46:15 | mnaser | melwitt: oh no that's exactly that todo :P | |
| 20:47:05 | melwitt | "without placement" confused me bc whether it's in-repo or out, I thought that's still "with placement" :P | |
| 20:47:21 | mnaser | pretty much involves creating a new tox.ini target in openstack/openstack-ansible-os_nova and creating an extra var like "nova_use_in_repo_placement=True" | |
| 20:47:36 | mnaser | yeah that should say "without os_placement" (which is our extra role) | |