Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-08
14:23:47 mriedem upper-constraints can be bumped
14:23:59 sean-k-mooney mriedem: ya i just wanted to bump upper
14:24:08 mriedem otherwise assume stable GA'ed and is frozen with the minimum
14:38:06 lpetrut Hi, I have a trivial fix for "nova usage-list", which is counting instances twice: https://review.openstack.org/#/c/608685/
14:48:01 mriedem lpetrut: it would be really nice if we could have a functional test to go along with that to show the regression
14:48:35 mriedem paging over simple tenant usage is confusing enough already
14:49:16 lpetrut sure, I'll add a test
14:52:01 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add native implementation OVSDB API https://review.openstack.org/482226
14:54:38 openstackgerrit Jan Gutter proposed openstack/os-vif master: Add support for generic representors https://review.openstack.org/608693
15:01:00 openstackgerrit Markus Hentsch proposed openstack/nova-specs master: Spec for the Nova part of Image Encryption https://review.openstack.org/608696
15:07:30 mriedem jaypipes: email to yikun sent about those allocation ratio specs
15:08:05 jaypipes mriedem: ty
15:08:09 mriedem efried: gibi: fyi since i need to backport https://review.openstack.org/#/c/606106/ i'm not going to try and fit in the newly refactored assertion method usage and just copy the old methods into the test itself, which i will remove from master in a follow up
15:09:18 efried mriedem: ack, sounds fair.
15:24:11 openstackgerrit Martin Midolesov proposed openstack/nova master: Implementing graceful shutdown. https://review.openstack.org/608704
15:24:33 openstackgerrit Matt Riedemann proposed openstack/nova master: Add post-test hook for testing evacuate https://review.openstack.org/602174
15:24:33 openstackgerrit Matt Riedemann proposed openstack/nova master: Add volume-backed evacuate test https://review.openstack.org/604397
15:24:35 openstackgerrit Matt Riedemann proposed openstack/nova master: Add functional regression test for bug 1794996 https://review.openstack.org/606106
15:24:35 openstack bug 1794996 in OpenStack Compute (nova) "_destroy_evacuated_instances fails and kills n-cpu startup if lazy-loading flavor on a deleted instance" [High,In progress] https://launchpad.net/bugs/1794996 - Assigned to Matt Riedemann (mriedem)
15:24:35 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix InstanceNotFound during _destroy_evacuated_instances https://review.openstack.org/606122
15:24:35 openstackgerrit Matt Riedemann proposed openstack/nova master: Run evacuate tests with local/lvm and shared/rbd storage https://review.openstack.org/604400
15:24:35 openstackgerrit Matt Riedemann proposed openstack/nova master: Refactor TestEvacuateDeleteServerRestartOriginalCompute https://review.openstack.org/608705
15:31:19 mriedem gibi: on https://review.openstack.org/#/c/605785/ can we just fail fast in the API/conductor if the admin is trying to force a host during live migration and the instance has allocations on non-standard resource classes? where non-standard is anything other than VCPU/MEMORY_MB/DISK_GB?
15:31:41 mriedem granted, with jay's CPU resource tracking spec, VCPU would eventually be nested too...
15:31:58 jaypipes mriedem: *could* be an inventory on a child provider, yes.
15:32:19 jaypipes mriedem: though it's bauzas' spec that goes into that.
15:32:44 jaypipes mriedem: since he wanted to keep all the NUMA-ness in his spec and out of the cpu-resources spec.
15:33:32 bauzas ... and I need to rebase this spec
15:33:54 bauzas since we said to unplug the NUMA affinity from this spec
15:36:37 openstackgerrit Merged openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612
15:43:13 mriedem sure, i just don't want to get too hung up on trying to bend over backward to honor that force parameter
15:43:18 mriedem which is a bad idea to use in the first place
15:44:14 openstackgerrit Merged openstack/nova master: Fix missing import in test_compute_mgr https://review.openstack.org/608426
16:04:35 mriedem need another core to look at the vmware live migration patch https://review.openstack.org/#/c/270116/
16:04:39 mriedem it's in a runway, +2 from me
16:04:41 mriedem CI is passing
16:04:57 sean-k-mooney mriedem: i was just talking to fungi about https://review.openstack.org/#/c/602384/ and the related bug
16:05:33 sean-k-mooney given its status he indicated we can talk about it here.
16:05:35 fungi yeah, a point-in-time summary of the situation would be helpful to add to that bug
16:05:58 fungi it's unclear to me how this can be a bug in nova but gets fixed by a patch to os-vif
16:06:11 sean-k-mooney fungi: i will update the bug today.
16:06:19 fungi much appreciated
16:07:03 fungi in particular if the vulnerability in the service has to be fixed by updating a dependent library, this is going to be complicated to communicate and may also make backporting harder
16:07:12 sean-k-mooney fungi: effectivly nova will wait for a notification form nuetron to know the port has been wired up on migration.
16:07:32 sean-k-mooney but in this edgecase os-vif is not used to plug the port libvirt is
16:07:51 sean-k-mooney as such we fallback to a time out and migrate without first having neutron wire up the port
16:08:23 sean-k-mooney neutron then wires up the port when the vm starts but that takes a little tiem to happen
16:08:47 sean-k-mooney the fix is to delegate to the os-vif lib to plug the interface in this edgecase also
16:09:17 sean-k-mooney that way the port will be wired up by neutron before we migrate
16:09:36 fungi and this is effectively a design flaw in nova because it assumes the call won't time out? or a bug in neutron for not treating it consistently using os-vif?
16:10:14 sean-k-mooney fungi: its legacy behavior form when nova woululd do firewalling for the port instead of neutron
16:10:54 fungi okay, so this will also eventually be solved when nova removes that deprecated behavior?
16:10:58 sean-k-mooney that said yes its partly a design flaw in nova.
16:11:29 sean-k-mooney yes if nova always used os-vif to plug the interface it would not happen
16:12:16 fungi and so https://review.openstack.org/602384 is basically a workaround to avoid having to switch nova to calling into os-vif for these?
16:12:33 mriedem umm...we use os-vif with nova-net too, so "yes if nova always used os-vif to plug the interface it would not happen" is kind of confusing
16:13:19 sean-k-mooney mriedem: the ovs plugin in os-vif was designed not to plug the interface in this case because libvirt does that
16:14:04 sean-k-mooney so we expressly do not create the ovs port and allow libvirt to do that currently. the change makes os-vif create the port which allows neutron to wire it up before we migrate the vm
16:15:16 sean-k-mooney mriedem: fungi so its not that nova does not call os-vif in this case. it does but os-vif was designed not to plug the interface in this case to maintain parity with how nova plugged interface before os-vif was split out
16:15:32 sean-k-mooney does that make sense?
16:16:00 mriedem shrug
16:16:03 fungi so the bug is in nova making assumptions about os-vif's behavior in this circumstance, or that os-vif doesn't fully implement the behavior nova expects?
16:16:20 mriedem sean-k-mooney: is this going to be a weird one off for ovs in os-vif only?
16:16:34 mriedem like will the behavior be different for all other vif types?
16:17:03 mriedem and when you say libvirt, do you mean the nova libvirt driver or libvirt the service?
16:17:11 sean-k-mooney that is a good question. i think there are a class of bugs related to this.
16:17:21 sean-k-mooney this bug predate os-vif
16:17:40 sean-k-mooney the original behavoir was incorrect
16:18:01 sean-k-mooney i say that becase anytime libvirt plugs the vif this can happen
16:18:48 sean-k-mooney this will not happen for vhost-user port as libvirt does not hanel plugin in that case. simplarly for siov libvirt set the vlan tag not neutron so that is safe
16:19:52 sean-k-mooney for now this is a one off but i need to look at other backends such as linux bridge to confirm this is a one off
16:19:56 fungi is this going to be backportable at least as far as stable/pike of os-vif? since we'll want a 1.7.1 tagged there for nova stable/pike to consume i guess
16:20:09 sean-k-mooney fungi: yes it should be
16:20:57 sean-k-mooney fungi: it requires not code change out side of os-vif and has no dependceis that i can tell to backport the change.
16:21:56 fungi i'm still a little iffy on how to go about describing this situation if we decide to publish an official advisory, particularly in that we consider it a nova bug but didn't patch nova to fix it. does the bug remain in nova even with newer os-vif? or is it simpler to explain it as a shortcoming of os-vif that we fixed to eliminate this behavior?
16:23:14 jaypipes mriedem: did the vmware live migration patch.
16:23:20 sean-k-mooney fungi: a newer os-vif will resolve the issue. my concern with calling this an os-vif only bug is if we go back to before we split out os-vif the bug i belive would still exist in the nova tree
16:24:24 fungi well, the advisory will only concern itself with the state of these repositories as of pike or later since we don't claim to provide security support to eol or em branches/releases
16:25:16 sean-k-mooney fungi: in that case its likely eaiser to discribe it as a os-vif bug given that pike uses os-vif
16:27:51 fungi and when you say "a class of bugs related to this" have any more been reported yet?
16:28:10 sean-k-mooney fungi: given the above if i add a release not to the os-vif change and propose backports. would it be inline with stable/vulnerablity policy to cut a release and bump the upper consttaint in the stable release
16:29:04 fungi mriedem would likely be able to better advise you on whether that particular change is suitable from a stable backport pilicy perspective
16:29:08 sean-k-mooney fungi: no but its posibly that thrid party plugins that manage ovs interfaces will need the same fix
16:30:02 fungi okay, so similar fixes may need to be applied to third-party ovs interface management plugins but not to any others officially managed by openstack as far as you're aware?
16:31:04 sean-k-mooney i just checked the linux bridge plugin and it does not need a similar fix as far as i can tell. so no none that i am aware of
16:31:41 fungi and what's the situation with https://review.openstack.org/602432 ? is that going to be abandoned as unneeded?
16:31:42 sean-k-mooney there are two out of tree plugins that i need to follow up on but i will check them and contact there maintianer if they need the same fix
16:32:24 sean-k-mooney fungi: yes i was going to that said i had planned on following up with the libvirt folks to see why it times out sometimes
16:34:05 sean-k-mooney we have had the issues using the ethernet iterface type in the past and i would like them to confirm why it does not work correctly in this case. that said its not relevent in to the bug disucssion
16:35:01 fungi okay, once you get a summary of the present state added to the bug and we get some confirmation that https://review.openstack.org/602384 is in line with stable policy, the vmt can write up an impact description, request a cve assignment and get the ball rolling on issuing an advisory
16:35:57 fungi and thanks for taking the time to explain this to me in such detail!
16:36:26 mriedem we'll want a release note on the os-vif change, we'll bump upper-constraints on stable but not lower-constraints,
16:36:31 sean-k-mooney fungi: no worries. i probably went into too much detail :)
16:36:43 mriedem it's unclear to me what, if any, side effects we could have on stable with different versions of ovs/libvirt being used
16:37:04 mriedem e.g. will libvirt complain if the port already exists because os-vif created it?
16:37:17 fungi yeah, some input on whether this is deemed safe enough to backport would also be most welcome
16:37:19 mriedem would libvirt create a duplicate?
16:37:20 sean-k-mooney this wont be effected by ovs. it may or may not be effected by libvirt version
16:37:55 sean-k-mooney mriedem: that is a good question and why i create https://review.openstack.org/#/c/602432/2 in the first place
16:38:18 sean-k-mooney i was expecting libvirt to be unhappy but the os-vif change passed tempest

Earlier   Later