| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-21 | |||
| 09:43:51 | mriedem | o/ | |
| 09:44:35 | kashyap | Isn't it terribly early for you there? | |
| 09:44:45 | cdent | jet lag? | |
| 09:44:52 | sean-k-mooney | mriedem: o/ | |
| 09:45:23 | mriedem | jet lag and work on the brain | |
| 09:45:40 | lbragstad | mriedem: same, my sleep schedule is completely screwed | |
| 09:46:29 | sean-k-mooney | mriedem: well do what jay pipes used to do. if your up early get work done early and be done by 1/2 pm and enjoy the rest of your day | |
| 09:48:03 | kashyap | mriedem: ^ Might want to check it out, when you're not sleeping :P | |
| 09:49:25 | kashyap | (FWIW, the author is not a "journalist" writing junk 'pop science'; he's a serious researcher on that topic.) | |
| 09:57:59 | openstack | Launchpad bug 1788014 in OpenStack Compute (nova) "when live migration fails due to a internal error rollback is not handeled correctly." [Undecided,New] | |
| 09:57:59 | mriedem | sean-k-mooney: question in https://bugs.launchpad.net/nova/+bug/1788014 | |
| 10:03:32 | sean-k-mooney | mriedem: hi i think that is the issue yes. i have not had time to pin down the exact cause but i supect it because we are not activating the source binding after deleting the dest | |
| 10:04:08 | mriedem | hmm, i guess i would have expected neutron to automatically activate the source host port bindings when the dest host bindings were deleted | |
| 10:04:24 | mriedem | because when we activate the dest host bindings, neutron automatically de-activates the source host bindings, | |
| 10:04:51 | mriedem | so my thinking was when we delete the dest bindings on rollback, neutron would say, oh i need to activate the only other bindings (source) left | |
| 10:05:57 | mriedem | i could have a wip patch for you to test with if you still have that live migration env available | |
| 10:16:17 | sean-k-mooney | mriedem: i have the devstack vms shut down but i can have it set up quickly again | |
| 10:16:31 | openstackgerrit | Slawek Kaplonski proposed openstack/os-vif master: Avoid os-vif to add ovs ports as trunk by default https://review.openstack.org/594118 | |
| 10:16:37 | sean-k-mooney | im goint to work on the first two neutron bugs first | |
| 10:17:04 | sean-k-mooney | ^ that is confusing.... we dont | |
| 10:20:00 | sapd1 | sean-k-mooney: Are you working on SR-IOV attach/detach? | |
| 10:20:01 | sapd1 | :D | |
| 10:21:55 | sean-k-mooney | sapd1: its on my todo list | |
| 10:26:37 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Set default AZ explicitely for instances without host. Ignore deleted instances when populating with availability zones https://review.openstack.org/594050 | |
| 10:27:57 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Set default AZ explicitely for instances without host. https://review.openstack.org/594050 | |
| 10:31:53 | mriedem | tssurya: on ^, we should just filter out instances w/o a host | |
| 10:32:04 | mriedem | default_availability_zone isn't the right config option for instances | |
| 10:32:19 | mriedem | default_schedule_zone is, but it defaults to None so it wouldn't fix the bug | |
| 10:36:31 | sean-k-mooney | mriedem: isint the default availableity zone nova? | |
| 10:36:46 | mriedem | default default_availability_zone is nova | |
| 10:36:53 | mriedem | if the instance is on a host | |
| 10:37:07 | mriedem | default_schedule_zone is the thing we set on instance.availability_zone if the user didn't request an az | |
| 10:37:09 | mriedem | and that defaults to None | |
| 10:38:39 | sean-k-mooney | oh ok, does horozon handel that differently? | |
| 10:38:51 | sean-k-mooney | or does devstack set them both to nova? | |
| 10:39:02 | mriedem | no | |
| 10:40:14 | mriedem | GET /servers/{id} will return '' for the az if the instance doesn't have a host set https://github.com/openstack/nova/blob/722d5b477219f0a2435a9f4ad4d54c61b83219f1/nova/api/openstack/compute/views/servers.py#L170 | |
| 10:41:32 | sean-k-mooney | mriedem: when can an instance not have a host set. when its shelved? | |
| 10:42:00 | mriedem | if it fails during scheduling | |
| 10:42:04 | mriedem | NoValidHost | |
| 10:42:09 | sean-k-mooney | ah ok | |
| 10:42:14 | mriedem | and yes if it's shelved offloaded | |
| 10:42:41 | mriedem | i'm not sure that we clear out the instance availability_zone on shelve offload though | |
| 10:42:47 | mriedem | if not that's like another bug | |
| 10:42:47 | sean-k-mooney | well it makes sense if its not schduled to a node it should not have az right? | |
| 10:42:53 | mriedem | correct | |
| 10:43:02 | mriedem | that's what i'm saying in the review | |
| 10:44:00 | openstack | Launchpad bug 1788115 in OpenStack Compute (nova) "nova-manage db online_data_migrations hangs on instances with no host set" [Medium,In progress] - Assigned to Jiri Suchomel (jsuchome) | |
| 10:44:00 | mriedem | melwitt: i've marked https://bugs.launchpad.net/nova/+bug/1788115 for rc potential | |
| 10:46:13 | mriedem | yeah we don't clear out the instance.az on shelve offload | |
| 10:46:22 | mriedem | we will update it on unshelve though | |
| 10:46:50 | mriedem | https://github.com/openstack/nova/blob/722d5b477219f0a2435a9f4ad4d54c61b83219f1/nova/conductor/manager.py#L815 | |
| 10:47:48 | mriedem | which reminds me https://review.openstack.org/#/c/559828/ | |
| 10:48:55 | mriedem | as noted in ^ we also don't clear the port binding | |
| 10:49:28 | tssurya | mriedem: back from lunch, oh okay yea would be better to all together filter out the ones where host is NOne | |
| 10:49:59 | mriedem | tssurya: i think that still works for your original bug too | |
| 10:50:01 | mriedem | comments inline on why | |
| 10:59:21 | tssurya | mriedem: I agree, | |
| 11:44:46 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Resource retrieving: add change-before filter https://review.openstack.org/591976 | |
| 11:47:03 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Filter out instances without a host when populating AZ https://review.openstack.org/594050 | |
| 11:53:26 | mriedem | sean-k-mooney: see how ^ floats your boat | |
| 11:53:27 | mriedem | oops | |
| 11:53:34 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Re-activate source host port bindings on live migration rollback https://review.openstack.org/594139 | |
| 11:53:37 | mriedem | sean-k-mooney: ^ | |
| 11:53:45 | mriedem | i want to see what miguel thinks about that too | |
| 11:55:48 | sean-k-mooney | mriedem: ill test that and see. the other thing is im not sure what the state of qemu when this partaclar bug happens due to the fact th monitor closed. | |
| 11:56:02 | sean-k-mooney | that is kind of a seperate bug however | |
| 11:56:27 | sean-k-mooney | at least with this patch if i do a hard reboot i think everything shold work properly again. | |
| 11:56:33 | mriedem | i am a bit surprised that qemu would bomb out after we went into post-copy mode | |
| 11:56:48 | mriedem | we only activate the dest host port bindings during post-copy or post-live migration | |
| 11:57:10 | mriedem | so your qemu failure must have happened after post-copy for us to activate the dest host bindings | |
| 11:57:27 | mriedem | you'd know if you saw "Binding ports to destination host" in the source host compute logs | |
| 11:58:15 | sean-k-mooney | ill reporduce and check that then apply your patch and see what happens | |
| 12:05:30 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: [placement] Use oslotest uuidsentinel https://review.openstack.org/594144 | |
| 12:05:37 | mriedem | cdent: ^ | |
| 12:05:59 | cdent | nice | |
| 12:07:22 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Resource retrieving: add change-before filter https://review.openstack.org/591976 | |
| 12:07:54 | cdent | mriedem: wait | |
| 12:08:10 | cdent | nm | |
| 12:08:27 | cdent | efried: did you see https://review.openstack.org/#/c/594068/ | |
| 12:08:41 | efried | looking... | |
| 12:09:01 | efried | cdent: Thanks, I was just starting to poke on that. | |
| 12:09:57 | cdent | efried: cool, wasn't sure if you had started your own | |
| 12:11:51 | mriedem | clearly that will have to go through the release dance | |
| 12:23:46 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Resource retrieving: add change-before filter https://review.openstack.org/591976 | |
| 12:23:55 | openstackgerrit | Slawek Kaplonski proposed openstack/os-vif master: Avoid os-vif to add untagged ports to ovs ports by default https://review.openstack.org/594118 | |
| 12:26:02 | openstackgerrit | Jiri Suchomel proposed openstack/nova master: Filter out instances without a host when populating AZ https://review.openstack.org/594050 | |
| 12:28:39 | openstackgerrit | Slawek Kaplonski proposed openstack/os-vif master: DNM Testing different CDN projects with DEAD_VLAN_TAG https://review.openstack.org/594153 | |
| 12:58:13 | tssurya | alex_xu, gmann or other api-experts: nova list seems to have this filter "--instance-name" which doesn't seem to be processed anywhere, was wondering where/why it was used ? | |
| 13:00:44 | alex_xu | tssurya: I even don't know we have '--instance-name' filter in the API | |
| 13:01:10 | alex_xu | I don't think we have that in the API | |
| 13:01:30 | tssurya | alex_xu: I didn't find it either; but its listed in the options | |
| 13:01:45 | mriedem | what is it translated to in novaclient? | |
| 13:03:19 | mriedem | added in 2011 by rackspace so it was probably something in rax | |
| 13:03:21 | mriedem | not upstream | |
| 13:04:24 | mriedem | looks like it's meant to filter on OS-EXT-SRV-ATTR:instance_name | |
| 13:04:28 | alex_xu | we have attribute 'OS-EXT-SRV-ATTR:instance_name', try to find out if there is any translate | |
| 13:04:40 | alex_xu | mriedem: yea | |
| 13:04:43 | mriedem | right which would be instance.name | |
| 13:04:51 | mriedem | filtering on --name would be instance.display_name | |