Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-03-26
13:43:08 mriedem i think dropping at the ptg is a good idea b/c there is a forum session on extraction right before that as well
13:43:21 mriedem as a final 'in 2 days we're going to drop this code from nova, any questions?'
13:43:23 dansmith yeah
13:56:06 mnaser dansmith, mriedem, cdent: OSA hasn’t progressed unfortunately. However, having an official timeline would be nice
13:56:29 mnaser So I’m supportive of at least knowing when it will be done.
13:56:53 mnaser We can probably get deploy from extracted easily but orchestrating the upgrade is the hard part
13:57:58 openstackgerrit Matt Riedemann proposed openstack/nova master: api-ref: add more details to confirmResize troubleshooting https://review.openstack.org/647741
13:58:16 mriedem efried: there is some wording for the API reference ^
13:59:23 dansmith mnaser: oh I thought deploy from extracted was already working, but I'm sure I was just confusing that with tripleo or something
14:01:48 mnaser Yeah, well, we have an ansible role that deploys placement only. But we have not integrated the role with our playbooks
14:01:55 dansmith ah, okay
14:12:31 mriedem mdbooth: if you're busy do you care if i add the unit test for https://review.openstack.org/#/c/645546/ ?
14:14:19 trident Hmm. Is trait:CUSTOM_FOO=forbidden set as flavor property expected to work in stable/rocky? I get instances spun up with flavors with that set ending up on resource providers (compute nodes) with that trait set. The other way around seem to work - a flavor with trait:CUSTOM_FOO=required makes instances with that flavor end up on the resource providers with that trait set.
14:15:51 mriedem trident: says it should https://docs.openstack.org/nova/rocky/user/flavors.html
14:15:58 mriedem "Added in the 18.0.0 Rocky release."
14:16:49 mdbooth mriedem: I'd be delighted :) I'm currently working on https://bugs.launchpad.net/nova/+bug/1821373
14:16:50 openstack Launchpad bug 1821373 in OpenStack Compute (nova) "Most instance actions can be called concurrently" [Undecided,New]
14:17:52 trident mriedem: Yeah, that's what I thought as well, so I am a bit surprised. Didn't really find any bugs that seem relevant either.
14:17:58 mriedem mdbooth: ok will do. another thing was https://review.openstack.org/#/c/551349/ just needs some tweaks i think, but it looks like maybe lyarwood is working that now
14:18:08 mriedem unless lyarwood is stuck in tripleo limbo for the next month
14:18:27 openstackgerrit Sylvain Bauza proposed openstack/nova master: Add doc on VGPU allocs and inventories for nrp https://review.openstack.org/647519
14:18:46 mdbooth mriedem: He's away this week, but he has taken that patch.
14:19:29 mriedem trident: if it's helpful, here is a functional test that does traits-based scheduling https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/tests/functional/test_servers.py#L4451
14:20:09 trident mriedem: Butt still the API call to placement /allocation_candidates doesn't seem to pass the forbidden trait (which if I understand it correctly should be "required=!CUSTOM_FOO" while it does pass it for the flavor with trait required: "required=CUSTOM_FOO"
14:21:08 trident mriedem: Thanks, I'll keep looking then, just wanted to check that I am not missing anything obvious.
14:21:33 mriedem hmm, looking at the test in there that uses a forbidden trait on a flavor https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/tests/functional/test_servers.py#L4824 it doesn't seem like the forbidden trait is actually testing anything useful
14:22:26 mriedem meaning, a better test would be to have something like 2 computes, one with trait CUSTOM_FOO and one without, and disable the one without so we can't schedule there, then schedule with forbidden CUSTOM_FOO and it should have a NoValidHost result,
14:22:33 mriedem but you're saying it sounds like it might just pass
14:23:42 trident mriedem: I'll see what I find continuing the troubleshooting and get back...
14:24:13 mriedem ack, this is the code that builds the GET /allocation_candidates query string fwiw https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/scheduler/utils.py#L257
14:28:03 trident mriedem: Thanks. that's exactly where I am currently looking :)
15:01:36 brandor5 hello everyone: I'm trying to use hypervisor_hostname as a scheduler hint and it's worked in the past but I'm running into issues now... I get a 400 bad request back... anyone have any ideas why that could be?
15:02:12 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix incomplete instance data returned after build failure https://review.openstack.org/645546
15:02:14 mriedem mdbooth: done ^
15:03:47 mdbooth mriedem: Thanks! You ought to add a co-authored line, esp considering the tests are longer than the fix :)
15:04:42 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix incomplete instance data returned after build failure https://review.openstack.org/645546
15:06:13 mriedem melwitt: want to hit ^ when you get a chance?
15:06:15 mriedem goes back to pike
15:06:48 mriedem brandor5: would be helpful if you shared the actual 400 error message
15:07:15 brandor5 mriedem: I'm trying to dig it out, 1 second :)
15:07:59 mriedem scheduler hints are open-ended for the most part, and hypervisor_hostname is not a known hint https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/api/openstack/compute/schemas/servers.py#L93
15:08:07 mriedem so i'm assuming you're using some out of tree scheduler filter for this hint?
15:08:27 brandor5 http://sprunge.us/9pGzX9
15:08:37 mriedem like forcing to build on a specific baremetal node?
15:08:44 brandor5 exactly
15:08:51 mriedem that doesn't give the error message
15:08:59 brandor5 query: '["=","$hypervisor_hostname","b23bbc4a-9a43-4ed8-b028-43d2aca3a6d7"]'
15:09:05 brandor5 that's the only 400 I see anywhere
15:09:17 brandor5 when i'm running the command
15:09:26 mriedem you don't get an error message on the client side?
15:09:29 brandor5 ansible returns: msg: 'BadRequestException: 400'
15:09:47 brandor5 if I remove the filter from the call it works fine
15:09:48 mriedem there is nothing else in the logs for req-f5f66dc0-8035-4c07-a631-a438fb79ba42 ?
15:10:07 mriedem so you're hitting something with this https://github.com/openstack/nova/blob/358776a303b0c5dba3c279f242c82b7a12ff5ce8/nova/api/openstack/compute/schemas/servers.py#L125
15:10:24 mriedem i want to say i've heard this reported bfore
15:10:26 mriedem *before
15:11:13 brandor5 nothing else found in the logs with that request
15:11:32 brandor5 the 400 would be coming from nova, correct?
15:11:37 brandor5 I shouldn't be looking down in ironic?
15:11:46 mriedem it's coming from nova
15:11:57 brandor5 that's what I thought
15:12:34 mdbooth Is it worth writing new code whose only consumer would be cellsv1?
15:12:52 mriedem mdbooth: nope
15:12:58 mdbooth Specifically, it looks like the only user of expected_vm_state is cellsv1
15:13:13 mdbooth i.e. instance.save(expected_vm_state=FOO)
15:13:15 mriedem probably because cells v1 was racy
15:13:26 mriedem syncing state between child and parent cell
15:13:55 mdbooth Ok. It'll look a bit odd to add new expected_task_state code without expected_vm_state code, though, so I'll add a comment
15:15:53 mriedem brandor5: i don't know what would be returning that 400 except the json schema validation code, but that query hint is just marked as a string or object, and you're passing a json string
15:16:22 mriedem the actual query hint isn't used until we hit the JsonFilter scheduler filter, which is long after you'd get that 400
15:16:43 mriedem so my guess is the request schema validation is somehow puking
15:16:54 brandor5 ok, i'll keep digging and will report back if I figure it out
15:17:08 mriedem i very much doubt we have any functional testing for JsonFilter
15:17:35 mriedem brandor5: i think the last person that had this problem just used availability_zone=nova::b23bbc4a-9a43-4ed8-b028-43d2aca3a6d7
15:17:45 mriedem the only problem with that is it bypasses the filters and forces to that node
15:17:55 brandor5 that would be okay in this case
15:18:04 mriedem https://docs.openstack.org/nova/latest/admin/availability-zones.html
15:18:05 brandor5 so I'll try that if the thing im about to test doesn't work
15:18:07 mriedem then force away
15:23:38 brandor5 mriedem: it turns out it was a bug in ansible, updating to latest and I don't see that issue now
15:26:18 mriedem huh
15:26:29 mriedem well you made me find our docs are busted at least :) https://bugs.launchpad.net/nova/+bug/1821764
15:26:30 openstack Launchpad bug 1821764 in OpenStack Compute (nova) "docs: JsonFilter query hint examples do not use valid json strings" [Medium,Triaged]
15:26:30 brandor5 yeah, weird
15:26:46 brandor5 hehe glad to be of assistance :D
15:28:55 sean-k-mooney mriedem: [">=", "$free_ram_mb", 1024] is a vlaid json array
15:29:21 mriedem sean-k-mooney: did you read the bug?
15:30:08 sean-k-mooney im just looking at it but '[">=", "$free_ram_mb", 1024]' is not the same thing as json.loads("[>=,$free_ram_mb,1024]")
15:30:57 mriedem serenity now
15:32:06 openstackgerrit Merged openstack/nova stable/pike: Fix WeighedHost logging regression https://review.openstack.org/641398
15:33:48 sean-k-mooney oh you are saying its correct in the schduler doc but not in the create server example
15:34:40 mriedem i don't see how what i wrote in the bug report is unclear
15:36:48 sean-k-mooney i was confused by the fact that it was correct in the osc command but not in the api example in the json filter doc
15:37:46 sean-k-mooney i intially looked at it and didnt see teh api example was wrong as my brain just filled in teh missing quots ared the args
15:38:53 sean-k-mooney its why im terible are prof readign my own stuff if i read two things that are almost identical and expect them to be the same i have troble seeign the difference
15:38:55 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix JsonFilter query hint examples in docs https://review.openstack.org/647778
15:42:44 mriedem ooo dansmith i could use the JsonFilter to schedule directly to a cell, that's fun
15:43:22 sean-k-mooney mriedem: you can you the json filter to do a lot of horribel things
15:43:57 dansmith mriedem: you can probably use it to do all kinds of things
15:45:36 sean-k-mooney as far as i rememebr it give you acess to any value contined in the hoststate object

Earlier   Later