| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-23 | |||
| 21:04:51 | dansmith | mriedem: the state->error stuff you mean? | |
| 21:04:57 | mriedem | dansmith: right | |
| 21:05:02 | dansmith | yeah | |
| 21:05:03 | mriedem | saves us the other instance.save() later | |
| 21:05:09 | dansmith | although | |
| 21:05:14 | dansmith | I was wondering if that would mess up the notificaitons | |
| 21:05:23 | dansmith | so you'd get a create notification in error state, | |
| 21:05:34 | dansmith | instead of a create of scheduling and then another updated one with error, then deleted, etc | |
| 21:06:34 | mriedem | i don't know if that's a big problem | |
| 21:07:03 | mriedem | depends on the consumer i guess | |
| 21:07:19 | dansmith | well, if you're currently watching for instance.updated,vm_state=ERROR for your logging thing | |
| 21:07:20 | mriedem | but i'm pretty sure we are not good about notifications as a proper state machine | |
| 21:07:23 | dansmith | and now it's instance.created | |
| 21:07:24 | dansmith | yeah | |
| 21:07:28 | dansmith | obvi | |
| 21:07:35 | mriedem | still thinking this is a bad idea https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L1178 | |
| 21:07:59 | dansmith | we have to pass there or we won't clean up the BRs | |
| 21:08:32 | dansmith | log it maybe if you want | |
| 21:08:47 | mriedem | mnaser: did you say the instance mapping existed for this instance but the cell_mapping was NULL? | |
| 21:08:56 | dansmith | yes | |
| 21:09:08 | mriedem | which could be the case if the instance.save() in _set_vm_state_and_notify blew up | |
| 21:10:09 | mnaser | yes ^ | |
| 21:10:20 | mriedem | dansmith: also, doesn't it seem like we should bury in cell0 here? https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L1244 | |
| 21:10:25 | mriedem | seems like a glaring omission | |
| 21:11:01 | mriedem | well, | |
| 21:11:07 | dansmith | I dunno, | |
| 21:11:10 | dansmith | we've picked a host/cell | |
| 21:11:11 | mriedem | i guess the idea is that if the user deleted the build request, and we don't create it in a cell, | |
| 21:11:13 | mriedem | then you can't list/show it | |
| 21:11:45 | dansmith | ah right this is the pre-create lookup | |
| 21:12:15 | dansmith | could argue that you should still be able to show deleted I guess | |
| 21:12:22 | dansmith | but not sure it's worth changing now | |
| 21:12:46 | mriedem | heh i guess i added that https://github.com/openstack/nova/commit/8ba92778fe14b47ad4ff5b53022e0550a93f37d3 | |
| 21:12:51 | mriedem | i knew it looked familiar | |
| 21:12:57 | dansmith | so on second thought it looks perfect? :) | |
| 21:13:05 | mriedem | it is pretty f'ing nice | |
| 21:13:29 | dansmith | you lost me about two minutes ago since 'Josie' came up on the jukebox | |
| 21:13:47 | dansmith | volume -> 100%, butt -> out of seat | |
| 21:14:29 | mriedem | well it is pretty funky | |
| 21:15:18 | eandersson | What is the general strategy for dealing with hypervisor maintenance (e.g. os updates etc)? | |
| 21:15:43 | eandersson | We wanted to look into a similar implementation to what Amazon does | |
| 21:16:12 | eandersson | Like tagging a hypervisor for downtime | |
| 21:16:31 | mriedem | there used to be a maintenance guide in the ops guide | |
| 21:16:32 | mnaser | i think that's beyond the scope of nova and more like business/operational processes | |
| 21:16:42 | mriedem | but i don't know where the ops guide stuff is being worked on now | |
| 21:17:17 | mriedem | https://github.com/openstack/openstack-manuals/blob/mitaka-eol/doc/ops-guide/source/ops_maintenance.rst | |
| 21:17:50 | eandersson | Yea - I was hoping that maybe someone wrote some exteranl tools for managing this already, or maybe even a feature inbuilt into nova to allow you to set a timer on a compute | |
| 21:18:09 | mnaser | i mean | |
| 21:18:10 | mriedem | didn't we talk about this at some point you and i? | |
| 21:18:14 | eandersson | We did | |
| 21:18:16 | mriedem | you can mark a compute as disabled from scheduling | |
| 21:18:26 | mnaser | and the user will see the status as 'maintenance' or something | |
| 21:18:27 | mnaser | in the api | |
| 21:18:27 | mriedem | https://developer.openstack.org/api-ref/compute/#compute-services-os-services | |
| 21:18:37 | mnaser | i think there is a hoststatus field or something | |
| 21:18:41 | mnaser | unless thats' admin only | |
| 21:18:43 | mriedem | the user doesn't see services | |
| 21:18:53 | mnaser | not but for vms running in that host | |
| 21:18:56 | mriedem | the guests on the host are unaffected if you disable the compute service | |
| 21:18:59 | mnaser | in /servers/<uuid> | |
| 21:19:16 | mriedem | you just can't schedule or migrate new instances to that host while the service is disabled | |
| 21:19:57 | mriedem | you can force the service status to 'down' to evacuate guests from it | |
| 21:20:08 | mriedem | but if it's up and normal otherwise just live migrate | |
| 21:20:42 | mnaser | eandersson: and for customer facing stuff, the nova instance has field 'host_status' which will be 'MAINTENANCE' if the vm is ona disabled host | |
| 21:20:49 | mnaser | so that might be a way to warn consumers that something will happen | |
| 21:20:55 | mnaser | if you wanna say disable it for a few days before | |
| 21:21:09 | mnaser | https://developer.openstack.org/api-ref/compute/#show-server-details | |
| 21:21:13 | mnaser | i swear it's there >:( | |
| 21:21:29 | mriedem | oh right, | |
| 21:21:32 | mriedem | admin only by default | |
| 21:21:35 | mriedem | configurable via policy | |
| 21:21:48 | mnaser | ah yeah, so maybe that might be something you want to mess with policy then | |
| 21:21:49 | mriedem | and only with microversion >= 2.16 which is probably newer than eandersson has | |
| 21:22:07 | eandersson | Yea - we are working hard on upgrading to Queens.. but still stuck on Mitaka for now | |
| 21:23:09 | eandersson | Also, us NUMA pinning everything does not help | |
| 21:23:31 | mriedem | not if you actually want to move things | |
| 21:23:50 | mnaser | while latency goes down with NUMA pinning | |
| 21:23:55 | mnaser | i've noticed overall performance is not as good | |
| 21:24:06 | mnaser | for workloads in general | |
| 21:24:21 | eandersson | Gaming is a bit of a odd one unfortunately | |
| 21:25:13 | eandersson | latency is everything for players (and not only talking about network latency here :D) | |
| 21:35:57 | mriedem | dansmith: i was going to rebase the queued_for_delete series but jaypipes has comments on your migration patch (and the change below it in the series) https://review.openstack.org/#/c/584504/ so you should probably look first | |
| 21:43:36 | openstackgerrit | Dan Smith proposed openstack/nova master: Online data migration for queued_for_delete flag https://review.openstack.org/584504 | |
| 21:43:49 | dansmith | (that was just the rebase) | |
| 21:48:46 | openstackgerrit | Dan Smith proposed openstack/nova master: Online data migration for queued_for_delete flag https://review.openstack.org/584504 | |
| 21:50:34 | openstackgerrit | Eric Fried proposed openstack/nova master: [placement] Add /reshaper handler for POST https://review.openstack.org/576927 | |
| 21:50:35 | openstackgerrit | Eric Fried proposed openstack/nova master: reshaper: Look up provider if not in inventories https://review.openstack.org/585033 | |
| 21:50:36 | openstackgerrit | Eric Fried proposed openstack/nova master: Make get_allocations_for_resource_provider sane https://review.openstack.org/584598 | |
| 21:50:37 | openstackgerrit | Eric Fried proposed openstack/nova master: Report client: Real get_allocs_for_consumer https://review.openstack.org/584599 | |
| 21:50:38 | openstackgerrit | Eric Fried proposed openstack/nova master: Report client: get_allocations_for_provider_tree https://review.openstack.org/584648 | |
| 21:50:39 | openstackgerrit | Eric Fried proposed openstack/nova master: Report client: _reshape helper, placement min bump https://review.openstack.org/585034 | |
| 21:51:45 | jaypipes | dansmith, mriedem: I'm +2 on em. | |
| 21:58:42 | mriedem | that makes 2 of us | |
| 21:59:20 | mriedem | dansmith: there is an email in the ops ML with some cells v2 questions, i replied with what i could but might be a couple of things in there you want to expand on | |
| 22:05:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update queued-for-delete from the ComputeAPI during deletion/restoration https://review.openstack.org/566813 | |
| 22:10:23 | openstackgerrit | Merged openstack/nova master: Add VIFMigrateData object for live migration https://review.openstack.org/515423 | |
| 22:13:41 | mriedem | oh wow something merged today | |
| 22:16:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: doc: link to CERN summit video about upgrading from cells v1 to v2 https://review.openstack.org/585044 | |
| 22:25:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: doc: link to AZ talk from the Rocky summit https://review.openstack.org/585045 | |
| 23:06:45 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Compute: Handle reshaped provider trees https://review.openstack.org/576236 | |