| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-03 | |||
| 13:34:20 | johnthetubaguy | could we use that NodeNotFound error to trigger a check of that resource provider, thinking about it | |
| 13:34:36 | openstackgerrit | Sergey Nikitin proposed openstack/nova-specs master: PCI NUMA Policies https://review.openstack.org/361140 | |
| 13:34:40 | vdrok | if the compute node was deleted but resource provider was not, the bit of deleting resource provider will not be retried by the periodic task? | |
| 13:36:38 | jroll | vdrok: yeah, that could be a problem, maybe we do it the other way (RP first, CN second), and if RP blows up don't delete the compute node either | |
| 13:36:44 | jroll | hrm | |
| 13:37:17 | cdent | mriedem, johnthetubaguy, vdrok, jroll: I'll hold off on pushing up anything until this discussion resolves in some fashion, otherwise it feels like we're going to churn | |
| 13:37:17 | jroll | johnthetubaguy: oh, we totally could | |
| 13:37:46 | jroll | cdent: maybe etherpad the reno draft for now, then, so we can get that handled? | |
| 13:37:57 | johnthetubaguy | mriedem: I am tempted to merge the current fix with a note, that stops *more* badness when stuff gets deleted, feels like we discuss the extra bits on top? | |
| 13:38:20 | cdent | jroll: except that it sounds like we potentially have a more robust fix in the works, which would change the note quite a bit? | |
| 13:38:26 | mriedem | johnthetubaguy: yeah i'm not looking to get a comprehensive fix in for rc1 | |
| 13:38:32 | cdent | or that, okay | |
| 13:38:35 | jroll | heh | |
| 13:38:41 | mriedem | we can backport something if needed | |
| 13:38:45 | johnthetubaguy | +1 | |
| 13:38:55 | mriedem | honestly i doubt very much that anyone is using placement yet, if even on newton yet | |
| 13:39:28 | johnthetubaguy | I think thats fair, based on the massive bugs we found in the last few days | |
| 13:39:53 | jroll | yeah, so to trigger this, you need newton running with placement, with ironic, and delete at least one ironic node | |
| 13:40:01 | jroll | and it will be triggered upon install of current master | |
| 13:40:05 | jroll | right? | |
| 13:41:14 | mriedem | not just install master, | |
| 13:41:20 | mriedem | but bring all of your computes up to ocata | |
| 13:41:28 | mriedem | so full upgrade of all nova services | |
| 13:42:05 | jroll | I don't think so | |
| 13:42:13 | jroll | the bug is in newton if you're running placement | |
| 13:42:25 | jroll | because you have inventories for things that don't exist | |
| 13:42:38 | mriedem | jroll: but the scheduler in newton doesn't use placement | |
| 13:42:38 | jroll | oh wait, the scheduler in ocata checks compute version, doesn't it | |
| 13:43:25 | sean-k-mooney | mriedem: if you have got a moment would you have any objection to me creating an experimatal job for nova to run a subset of what is currently run in the intel NFV ci? we will be moving where the NFV ci is hosted in then next 2-4 months and now seems like a good opurtnity to upstream the tests that dont require special hardware. those that do will remain in the nfv ci. | |
| 13:43:27 | jroll | the scheduler in ocata does, though, I just don't know if it waits for computes to be upgraded before using placement | |
| 13:44:22 | cdent | jroll: it does | |
| 13:44:23 | cdent | recent change | |
| 13:44:44 | jroll | ok cool, so builds don't break until you upgrade everything to ocata | |
| 13:44:55 | jroll | thanks cdent | |
| 13:46:08 | cdent | here's a stab at the reno: https://etherpad.openstack.org/p/bug-1661258-reno | |
| 13:46:13 | cdent | rather hand wavey | |
| 13:47:08 | cdent | or should it be an issue instead of fixes? | |
| 13:47:24 | jroll | maybe both? | |
| 13:48:03 | mriedem | sean-k-mooney: sure | |
| 13:48:34 | mriedem | cdent: jroll: i'd just have 'issues' as in known issue | |
| 13:48:54 | cdent | is leading to a reschedule the only outcome? I thought it could be more dire | |
| 13:50:02 | mriedem | it triggers a reschedule | |
| 13:50:06 | jroll | hm, is there any way to trigger this for not ironic nodes? | |
| 13:50:09 | mriedem | of which you get 3 and then NoValidHost | |
| 13:50:29 | mriedem | jroll: you'd have to delete the compute node records manually for non-ironic | |
| 13:51:16 | jroll | mriedem: right, because n-cpu wouldn't be running for a hyp that went away right | |
| 13:51:36 | jroll | s/went away/was thrown in the dumpster/ | |
| 13:51:43 | mriedem | n-cpu can be running and if it is, it will recreate the compute node record in the db | |
| 13:51:47 | mriedem | but the service record might be gone | |
| 13:51:54 | sean-k-mooney | mriedem: ok ill start working on the job. after a while i would hope we can move it to the check pipeline as non voting and then eventually voteing once it is shown to be stable. | |
| 13:52:26 | jroll | mriedem: right-o | |
| 13:53:20 | jroll | okay, I think I'm happy with that reno | |
| 13:53:28 | cdent | jroll, mriedem I we're happy with that content I can smooth it | |
| 13:53:31 | jroll | and I'm good with the fix as-is, for now, given the reno | |
| 13:53:37 | jroll | cdent: ++ | |
| 13:53:58 | sean-k-mooney | mriedem: im also going to work on some jobs for os-vif and a few other repos. i may have an update at the ptg on nfv testing in general. | |
| 13:54:53 | mriedem | sean-k-mooney: splendid | |
| 13:54:57 | mriedem | thanks for working on that | |
| 13:56:20 | sean-k-mooney | mriedem: well its been kindof buging me that we dont have a linux bridge job for os-vif there are some other gaps in the unit testing in os-vif and nova that i want to take a look at too to tighten up the testing a little | |
| 13:58:32 | openstackgerrit | Chris Dent proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | |
| 13:58:39 | cdent | ^^ adds the reno and changes one bug to partial-bug | |
| 13:58:45 | bauzas | mriedem: FWIW, saw your discussion with dansmith and melwitt yesterday about BFV instances | |
| 13:59:21 | jroll | cdent: trailing space on the reno, not sure if that breaks some check job | |
| 13:59:26 | bauzas | mriedem: so I saw your comment on gerrit and replied to it by saying that we should still verify the disk resource in placement | |
| 13:59:30 | cdent | dag nab it | |
| 13:59:36 | bauzas | mriedem: unless the disk value is 0 | |
| 14:00:02 | openstackgerrit | Chris Dent proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | |
| 14:00:03 | bauzas | mriedem: because once we will have Cinder RPs, then we will still take a allocation for the disk size | |
| 14:01:15 | jroll | cdent: +1'd. now I'm wondering if we want to catch NodeNotFound and have a better error message, but it's probably fine | |
| 14:01:50 | mriedem | jroll: you could still probably get the NodeNotFound for other reasons | |
| 14:01:58 | mriedem | like some kind of race to delete the node and schedule to it at the same time | |
| 14:02:01 | mriedem | i'd just leave it | |
| 14:02:18 | jroll | mriedem: good point | |
| 14:02:38 | mriedem | bauzas: i don't really have that all in my head, but dansmith has a WIP up for some of this | |
| 14:02:55 | mriedem | https://review.openstack.org/#/c/428481/ | |
| 14:07:55 | figleaf | cdent: thanks for picking up https://review.openstack.org/428375 | |
| 14:08:39 | mriedem | +2 from me | |
| 14:08:53 | cdent | figleaf: you're welcome, seemed like that one had some rush on it. If you're busy today I could also pick up the allocation schema stuff too (on which I left some comments about some wrongness) | |
| 14:09:34 | figleaf | cdent: No, not busy. Just getting started, and the caffeine hasn't yet kicked in | |
| 14:09:47 | figleaf | Gonna look at those next | |
| 14:09:48 | cdent | roger that | |
| 14:09:54 | jroll | figleaf: it's 8am already, why so late | |
| 14:10:19 | figleaf | jroll: had to get the rest of the house out to work/school | |
| 14:10:24 | cdent | I guess that means I should think about writing a "placement update" which I've been avoiding because I have no clue, probably because caffeine | |
| 14:10:27 | figleaf | jroll: gotta be able to focus! | |
| 14:10:35 | jroll | figleaf: that was a joke, to be clear :P | |
| 14:10:54 | figleaf | jroll: so was my response :) | |
| 14:11:11 | jroll | :D | |
| 14:11:31 | cdent | s/before/because/ | |
| 14:16:26 | cdent | mriedem: imma fix that nit | |
| 14:17:12 | openstackgerrit | Chris Dent proposed openstack/nova master: Delete a compute node's resource provider when node is deleted https://review.openstack.org/428375 | |
| 14:20:08 | bauzas | cdent: figleaf: open questions in https://review.openstack.org/#/c/428375/9 | |
| 14:20:42 | figleaf | cdent: regarding https://review.openstack.org/#/c/416752/8/nova/api/openstack/placement/handlers/inventory.py, what should be the value for 'max_unit' in the INVENTORY_DEFAULTS (line 109) ? | |
| 14:22:06 | cdent | figleaf: I would think since we want to set a schema minimum of 1 (since less than that is meaningless or wrong) then the default should be 1 as well? | |
| 14:22:09 | cdent | bauzas: looking | |
| 14:23:18 | bauzas | mriedem: when are you removing -W for https://review.openstack.org/#/c/428531/ ? | |
| 14:24:39 | mriedem | bauzas: probably after https://review.openstack.org/#/c/428375/ is merged now | |
| 14:24:46 | mriedem | johnthetubaguy: +2 on ^ now | |
| 14:25:01 | johnthetubaguy | looking... | |
| 14:25:11 | figleaf | cdent: didn | |