| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-26 | |||
| 13:34:51 | openstack | Launchpad bug 1784020 in OpenStack Compute (nova) "Shared storage providers are not supported and will break things if used" [Medium,Fix released] | |
| 13:34:53 | cdent | I kinda feel like some sort of forcing function is required, but that may be either unrealistic or rude | |
| 13:35:59 | dansmith | yeah could be, but before the previous release is even out might be a teensy bit aggresive | |
| 13:36:04 | dansmith | (but spelled properly) | |
| 13:36:58 | dansmith | the deployment projects also receive the forcing-function love from all projects simultaneously, which isn't super fun :) | |
| 13:37:15 | cdent | if we choose to wait, I will keep the patch up to date | |
| 13:37:33 | cdent | somebody just keep me informed please | |
| 13:37:57 | mriedem | oh i didn't realize there were +2s stacked up on https://review.openstack.org/#/c/618215/ | |
| 13:38:02 | mriedem | yeah that's way too aggressive IMO | |
| 13:38:15 | dansmith | easy for me to say, but it doesn't look like there are too many places that it will be conflict-heavy | |
| 13:38:36 | dansmith | mriedem: yeah, that was my concern | |
| 13:39:24 | cdent | I think only conf and cmd stuff is likely to conflict, if at all | |
| 13:39:54 | mriedem | honestly i'd probably proceduraly -2 that for now | |
| 13:39:57 | mriedem | efried: stephenfin: ^ | |
| 13:40:20 | cdent | it was until yesterday | |
| 13:40:37 | stephenfin | mriedem: Yup, I wasn't sure so I held off on the +W. Happy for someone -2 that for now, yup | |
| 13:41:06 | cdent | If this turns into nova-net, I'm going to burn a hole through the earth | |
| 13:41:34 | dansmith | could we maybe tell the deployment projects that we're going to merge this the week of the PTG, so they should either get it done before then or show up with lots of reasons and gifts if not? | |
| 13:41:50 | cdent | seems a reasonable plan | |
| 13:43:08 | mriedem | i think dropping at the ptg is a good idea b/c there is a forum session on extraction right before that as well | |
| 13:43:21 | mriedem | as a final 'in 2 days we're going to drop this code from nova, any questions?' | |
| 13:43:23 | dansmith | yeah | |
| 13:56:06 | mnaser | dansmith, mriedem, cdent: OSA hasn’t progressed unfortunately. However, having an official timeline would be nice | |
| 13:56:29 | mnaser | So I’m supportive of at least knowing when it will be done. | |
| 13:56:53 | mnaser | We can probably get deploy from extracted easily but orchestrating the upgrade is the hard part | |
| 13:57:58 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: add more details to confirmResize troubleshooting https://review.openstack.org/647741 | |
| 13:58:16 | mriedem | efried: there is some wording for the API reference ^ | |
| 13:59:23 | dansmith | mnaser: oh I thought deploy from extracted was already working, but I'm sure I was just confusing that with tripleo or something | |
| 14:01:48 | mnaser | Yeah, well, we have an ansible role that deploys placement only. But we have not integrated the role with our playbooks | |
| 14:01:55 | dansmith | ah, okay | |
| 14:12:31 | mriedem | mdbooth: if you're busy do you care if i add the unit test for https://review.openstack.org/#/c/645546/ ? | |
| 14:14:19 | trident | Hmm. Is trait:CUSTOM_FOO=forbidden set as flavor property expected to work in stable/rocky? I get instances spun up with flavors with that set ending up on resource providers (compute nodes) with that trait set. The other way around seem to work - a flavor with trait:CUSTOM_FOO=required makes instances with that flavor end up on the resource providers with that trait set. | |
| 14:15:51 | mriedem | trident: says it should https://docs.openstack.org/nova/rocky/user/flavors.html | |
| 14:15:58 | mriedem | "Added in the 18.0.0 Rocky release." | |
| 14:16:49 | mdbooth | mriedem: I'd be delighted :) I'm currently working on https://bugs.launchpad.net/nova/+bug/1821373 | |
| 14:16:50 | openstack | Launchpad bug 1821373 in OpenStack Compute (nova) "Most instance actions can be called concurrently" [Undecided,New] | |
| 14:17:52 | trident | mriedem: Yeah, that's what I thought as well, so I am a bit surprised. Didn't really find any bugs that seem relevant either. | |
| 14:17:58 | mriedem | mdbooth: ok will do. another thing was https://review.openstack.org/#/c/551349/ just needs some tweaks i think, but it looks like maybe lyarwood is working that now | |
| 14:18:08 | mriedem | unless lyarwood is stuck in tripleo limbo for the next month | |
| 14:18:27 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add doc on VGPU allocs and inventories for nrp https://review.openstack.org/647519 | |
| 14:18:46 | mdbooth | mriedem: He's away this week, but he has taken that patch. | |
| 14:19:29 | mriedem | trident: if it's helpful, here is a functional test that does traits-based scheduling https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/tests/functional/test_servers.py#L4451 | |
| 14:20:09 | trident | mriedem: Butt still the API call to placement /allocation_candidates doesn't seem to pass the forbidden trait (which if I understand it correctly should be "required=!CUSTOM_FOO" while it does pass it for the flavor with trait required: "required=CUSTOM_FOO" | |
| 14:21:08 | trident | mriedem: Thanks, I'll keep looking then, just wanted to check that I am not missing anything obvious. | |
| 14:21:33 | mriedem | hmm, looking at the test in there that uses a forbidden trait on a flavor https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/tests/functional/test_servers.py#L4824 it doesn't seem like the forbidden trait is actually testing anything useful | |
| 14:22:26 | mriedem | meaning, a better test would be to have something like 2 computes, one with trait CUSTOM_FOO and one without, and disable the one without so we can't schedule there, then schedule with forbidden CUSTOM_FOO and it should have a NoValidHost result, | |
| 14:22:33 | mriedem | but you're saying it sounds like it might just pass | |
| 14:23:42 | trident | mriedem: I'll see what I find continuing the troubleshooting and get back... | |
| 14:24:13 | mriedem | ack, this is the code that builds the GET /allocation_candidates query string fwiw https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/scheduler/utils.py#L257 | |
| 14:28:03 | trident | mriedem: Thanks. that's exactly where I am currently looking :) | |
| 15:01:36 | brandor5 | hello everyone: I'm trying to use hypervisor_hostname as a scheduler hint and it's worked in the past but I'm running into issues now... I get a 400 bad request back... anyone have any ideas why that could be? | |
| 15:02:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix incomplete instance data returned after build failure https://review.openstack.org/645546 | |
| 15:02:14 | mriedem | mdbooth: done ^ | |
| 15:03:47 | mdbooth | mriedem: Thanks! You ought to add a co-authored line, esp considering the tests are longer than the fix :) | |
| 15:04:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix incomplete instance data returned after build failure https://review.openstack.org/645546 | |
| 15:06:13 | mriedem | melwitt: want to hit ^ when you get a chance? | |
| 15:06:15 | mriedem | goes back to pike | |
| 15:06:48 | mriedem | brandor5: would be helpful if you shared the actual 400 error message | |
| 15:07:15 | brandor5 | mriedem: I'm trying to dig it out, 1 second :) | |
| 15:07:59 | mriedem | scheduler hints are open-ended for the most part, and hypervisor_hostname is not a known hint https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/api/openstack/compute/schemas/servers.py#L93 | |
| 15:08:07 | mriedem | so i'm assuming you're using some out of tree scheduler filter for this hint? | |
| 15:08:27 | brandor5 | http://sprunge.us/9pGzX9 | |
| 15:08:37 | mriedem | like forcing to build on a specific baremetal node? | |
| 15:08:44 | brandor5 | exactly | |
| 15:08:51 | mriedem | that doesn't give the error message | |
| 15:08:59 | brandor5 | query: '["=","$hypervisor_hostname","b23bbc4a-9a43-4ed8-b028-43d2aca3a6d7"]' | |
| 15:09:05 | brandor5 | that's the only 400 I see anywhere | |
| 15:09:17 | brandor5 | when i'm running the command | |
| 15:09:26 | mriedem | you don't get an error message on the client side? | |
| 15:09:29 | brandor5 | ansible returns: msg: 'BadRequestException: 400' | |
| 15:09:47 | brandor5 | if I remove the filter from the call it works fine | |
| 15:09:48 | mriedem | there is nothing else in the logs for req-f5f66dc0-8035-4c07-a631-a438fb79ba42 ? | |
| 15:10:07 | mriedem | so you're hitting something with this https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/api/openstack/compute/schemas/servers.py#L125 | |
| 15:10:24 | mriedem | i want to say i've heard this reported bfore | |
| 15:10:26 | mriedem | *before | |
| 15:11:13 | brandor5 | nothing else found in the logs with that request | |
| 15:11:32 | brandor5 | the 400 would be coming from nova, correct? | |
| 15:11:37 | brandor5 | I shouldn't be looking down in ironic? | |
| 15:11:46 | mriedem | it's coming from nova | |
| 15:11:57 | brandor5 | that's what I thought | |
| 15:12:34 | mdbooth | Is it worth writing new code whose only consumer would be cellsv1? | |
| 15:12:52 | mriedem | mdbooth: nope | |
| 15:12:58 | mdbooth | Specifically, it looks like the only user of expected_vm_state is cellsv1 | |
| 15:13:13 | mdbooth | i.e. instance.save(expected_vm_state=FOO) | |
| 15:13:15 | mriedem | probably because cells v1 was racy | |
| 15:13:26 | mriedem | syncing state between child and parent cell | |
| 15:13:55 | mdbooth | Ok. It'll look a bit odd to add new expected_task_state code without expected_vm_state code, though, so I'll add a comment | |
| 15:15:53 | mriedem | brandor5: i don't know what would be returning that 400 except the json schema validation code, but that query hint is just marked as a string or object, and you're passing a json string | |
| 15:16:22 | mriedem | the actual query hint isn't used until we hit the JsonFilter scheduler filter, which is long after you'd get that 400 | |
| 15:16:43 | mriedem | so my guess is the request schema validation is somehow puking | |
| 15:16:54 | brandor5 | ok, i'll keep digging and will report back if I figure it out | |
| 15:17:08 | mriedem | i very much doubt we have any functional testing for JsonFilter | |
| 15:17:35 | mriedem | brandor5: i think the last person that had this problem just used availability_zone=nova::b23bbc4a-9a43-4ed8-b028-43d2aca3a6d7 | |
| 15:17:45 | mriedem | the only problem with that is it bypasses the filters and forces to that node | |
| 15:17:55 | brandor5 | that would be okay in this case | |
| 15:18:04 | mriedem | https://docs.openstack.org/nova/latest/admin/availability-zones.html | |
| 15:18:05 | brandor5 | so I'll try that if the thing im about to test doesn't work | |
| 15:18:07 | mriedem | then force away | |
| 15:23:38 | brandor5 | mriedem: it turns out it was a bug in ansible, updating to latest and I don't see that issue now | |
| 15:26:18 | mriedem | huh | |