| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-18 | |||
| 20:59:31 | efried | sean-k-mooney: I wish dansmith or mikal or someone had warned us about problems like this. http://lists.openstack.org/pipermail/openstack-dev/2018-April/129294.html | |
| 20:59:57 | mriedem | https://github.com/mfcloud/python-zvm-sdk/blob/master/setup.py#L39 | |
| 21:00:13 | efried | http://lists.openstack.org/pipermail/openstack-dev/2018-April/129350.html | |
| 21:00:50 | mriedem | i guess this is where we'd report a bug that it doesn't work with 3.6 https://bugs.launchpad.net/python-zvm-sdk/+bugs | |
| 21:01:00 | mriedem | and then harass the shit out of jichenjc to get that fixed and released | |
| 21:01:29 | efried | support 3.7 while we're at it, so we don't get behind immediately. | |
| 21:01:45 | sean-k-mooney | mriedem: ya the weird thing is the error message seams to indicate it support 3.5 but only declares 2.7 support in that file | |
| 21:01:57 | mriedem | yeah it definitely supports 3.5 https://github.com/mfcloud/python-zvm-sdk/blob/master/tox.ini#L27 | |
| 21:02:04 | mriedem | what blows up with 3.6? | |
| 21:02:09 | mriedem | it just doesn't install/ | |
| 21:02:10 | mriedem | ? | |
| 21:02:23 | sean-k-mooney | yep tox fails while pip installing it | |
| 21:02:30 | mriedem | what's the error? | |
| 21:02:56 | sean-k-mooney | one sec | |
| 21:04:29 | sean-k-mooney | http://paste.openstack.org/show/726237/ | |
| 21:06:04 | mriedem | bingo https://github.com/mfcloud/python-zvm-sdk/blob/master/zvmsdk/version.py#L28 | |
| 21:07:02 | sean-k-mooney | you know if you replace that 5 with a 6 it will proably work | |
| 21:07:28 | mriedem | try it | |
| 21:07:55 | mriedem | or just try doing that and add a tox36 target and run it's unit tests | |
| 21:08:08 | mriedem | this could be your first (and only ever) zvmcloudconnector patch :) | |
| 21:09:31 | sean-k-mooney | some might say its one too many. | |
| 21:09:54 | sean-k-mooney | that said i just downloaded and ran its py27 unitests and one of them still failed | |
| 21:09:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Heal RequestSpec.is_bfv for legacy instances during moves https://review.openstack.org/583715 | |
| 21:09:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix wonky reqspec handling in conductor.unshelve_instance https://review.openstack.org/583739 | |
| 21:10:28 | mriedem | let me state one more time, for the record, for this week, that i hate all of the request spec compat code in conductor | |
| 21:11:12 | sean-k-mooney | mriedem: duely noted. | |
| 21:11:57 | sean-k-mooney | i just found another bug in there test code... i am going to go down a rabbit whole with this i can tell | |
| 21:12:31 | _ix | Afternoon folks. I realize this channel is less support oriented, but there's no response in #openstack. I'm having an issue in our pike deployment with resource allocations still existing on one compute, but the vm itself resides on another. It's not reflecting additional usage on the original VM, but the scheduler still won't allow me to instantiate there. | |
| 21:12:39 | _ix | How can I clear the allocation from the original hypervisor? | |
| 21:13:05 | _ix | > There are allocations remaining against the source host that might need to be removed | |
| 21:14:21 | sean-k-mooney | _ix: did you migrate or resize the vm at some point? if you have not completed the migrations then it will have both allocations if i rembere correctly | |
| 21:14:39 | _ix | I did, but the migration failed. | |
| 21:15:01 | _ix | I believe it was the case where a live migration was ongoing, and the targeted compute's services were unavailable or became unavailable. | |
| 21:15:24 | _ix | From a user's perspective, there doesn't seem to be a way to back out of the migration. | |
| 21:15:40 | sean-k-mooney | _ix so the vm migrated but the migration timeded out becase the node became unableale? | |
| 21:16:23 | _ix | The vm stayed on the source but received allocation on the target. | |
| 21:16:24 | sean-k-mooney | _ix: well migration is an admin action so "end users" should not be aware of them in general | |
| 21:16:33 | _ix | That's a fair point. | |
| 21:17:10 | sean-k-mooney | mriedem: jaypipes efried do you know how to clean up the stale allocation? | |
| 21:17:29 | _ix | How can I force the issue? I've manually added allocations records to reflect reality in the nova_api.allocations table and marked the failed migration as deleted in nova.migrations. | |
| 21:17:51 | efried | sean-k-mooney: I haven't been following the conversation. You want to manually remove an allocation from placement? | |
| 21:18:02 | _ix | Yes please. | |
| 21:18:15 | sean-k-mooney | efried: live migration failed but allocation on dest remain | |
| 21:18:32 | efried | mriedem is the osc-placement expert. I can only tell you how to do it programmatically :) | |
| 21:19:48 | openstack | Launchpad bug 1753676 in OpenStack Compute (nova) "Live migration not working as Expected when Restarting nova-compute service while migration from source node" [Undecided,Confirmed] | |
| 21:19:48 | _ix | It's an open bug, but it's such an edge case, I totally understand why this might lead to something out-of-state: https://bugs.launchpad.net/nova/+bug/1753676 | |
| 21:20:23 | _ix | I'm pleased to report live migrations tend to work quite well and fast, I might add. | |
| 21:20:34 | sean-k-mooney | _ix: ya migration in pike was kind of buggy. dansmith was working on fixing that i think | |
| 21:20:56 | mriedem | yeah osc-placement | |
| 21:21:16 | mriedem | https://docs.openstack.org/osc-placement/latest/index.html | |
| 21:21:29 | mriedem | https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-allocation-set specifically | |
| 21:21:46 | _ix | mriedem: Thanks! | |
| 21:21:51 | mriedem | if the consumer (instance in this case) shows having allocations against multiple resource providers (compute nodes), then determine the one you want to keep and the one you want to remove, and update the allocations | |
| 21:22:04 | mriedem | you can view the allocations with https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-aggregate-list | |
| 21:22:07 | mriedem | oops | |
| 21:22:12 | sean-k-mooney | mriedem: would the consumer not be the migration in this case | |
| 21:22:13 | mriedem | https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-allocation-show | |
| 21:22:18 | mriedem | sean-k-mooney: not in pike | |
| 21:22:25 | sean-k-mooney | oh ok | |
| 21:22:27 | mriedem | that change happened in queens | |
| 21:22:41 | mriedem | https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/migration-allocations.html | |
| 21:22:55 | mriedem | also, pike GA had a few bugs with cleaning up allocations during failures, | |
| 21:23:01 | mriedem | which we fixed in stable point releases of pike | |
| 21:23:07 | mriedem | _ix: which version of pike are you running? latest stable i hope | |
| 21:23:18 | mriedem | https://docs.openstack.org/releasenotes/nova/pike.html | |
| 21:24:20 | _ix | mriedem: Latest stable, I believe. | |
| 21:24:42 | _ix | I must be missing a client. What enables the placement hierarchy? | |
| 21:25:11 | mriedem | _ix: https://pypi.org/project/osc-placement/ | |
| 21:25:26 | mriedem | it's an osc plugin | |
| 21:25:56 | _ix | I wish I were more familiar. I'll do some reading. Thanks very much! | |
| 21:27:47 | mriedem | np | |
| 21:28:21 | mriedem | placement is a new big required thing to a lot of people in pike so it's ok to ask operational questions here | |
| 21:36:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add regression test for bug 1781710 https://review.openstack.org/583339 | |
| 21:36:58 | openstack | bug 1781710 in OpenStack Compute (nova) "ServersOnMultiNodesTest.test_create_server_with_scheduler_hint_group_anti_affinity failing with "Servers are on the same host"" [High,Fix released] https://launchpad.net/bugs/1781710 - Assigned to Matt Riedemann (mriedem) | |
| 21:36:59 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update RequestSpec.instance_uuid during scheduling https://review.openstack.org/583347 | |
| 21:37:03 | mriedem | jaypipes: melwitt: sean-k-mooney: now rebased on top of the regression test which also helped me find some other places i needed to touch up | |
| 21:38:47 | _ix | So... if a consumer is provided for by.... four providers, that probably isn't so good. | |
| 21:39:41 | mriedem | _ix: nope | |
| 21:40:01 | mriedem | in case it's not clear, you can correlate the provider id to hypervisor ids | |
| 21:40:03 | mriedem | using nova hypervisor-list | |
| 21:40:10 | sean-k-mooney | efried: mriedem pull request sent https://github.com/mfcloud/python-zvm-sdk/pull/7 | |
| 21:40:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update RequestSpec.instance_uuid during scheduling https://review.openstack.org/583347 | |
| 21:41:30 | mriedem | sean-k-mooney: update the setup.py too | |
| 21:42:14 | sean-k-mooney | mriedem: oh ya will do | |
| 21:44:50 | _ix | I've probably spent too much time looking the through the database entries trying to figure out how this was supposed to work. Perhaps I don't see the forest for the trees, but I would guess that the consumers aren't only instances. Is that the case? | |
| 21:48:45 | sean-k-mooney | mriedem: i just did a force push to my forked copy and the pull request updated. im not sure if i like this or now but done | |
| 21:50:21 | jaypipes | mriedem: ack. ++ | |
| 21:53:52 | melwitt | ack | |
| 21:55:12 | efried | _ix: In pike they are; in more recent releases we introduced a special consumer to represent migrations so we didn't have to double up instance allocs like you're seeing. | |
| 22:04:17 | openstackgerrit | sean mooney proposed openstack/os-vif master: add upper_constraints support https://review.openstack.org/583582 | |
| 22:09:26 | sean-k-mooney | efried: addressed you comments and changed to CONSTRAINTS_OPT | |
| 22:12:25 | sean-k-mooney | anyway its 11 so i should proably log off | |
| 22:12:41 | sean-k-mooney | i have missed working upstream | |
| 22:17:58 | sean-k-mooney | efried: stephenfin melwitt mriedem before i go. if we merge https://review.openstack.org/#/c/583582 today ill update https://review.openstack.org/#/c/583319 in the morning and os-vif will be updated for rocky. if their are any other change ye want ill adress them in the morning | |
| 22:50:10 | melwitt | sean-k-mooney: ack | |
| 23:33:38 | openstackgerrit | Merged openstack/nova master: [placement] Add gabbi coverage for an inventory change https://review.openstack.org/583642 | |
| 23:33:49 | openstackgerrit | Merged openstack/nova master: [placement] Add gabbi coverage for inv of missing rp https://review.openstack.org/583643 | |
| 23:33:57 | openstackgerrit | Merged openstack/nova master: [placement] cover bad content-length header https://review.openstack.org/583644 | |
| #openstack-nova - 2018-07-19 | |||
| 01:06:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Use consumer generation in _heal_allocations_for_instance https://review.openstack.org/577905 | |
| 01:39:09 | openstackgerrit | Merged openstack/python-novaclient master: Microversion 2.64 - Use new format policy in server group https://review.openstack.org/578261 | |