Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-09-20
15:11:57 sdague AZs really exist to bound failure domains so that you can HA across them correctly
15:12:11 sdague moving an instance across AZ boundaries completely ruins that strategy
15:12:24 bauzas so, the point is, if you made a typo, then you're screwed up if some users began to use your cloud, and you have to explain to them that you screwed up
15:12:48 bauzas but you shouldn't magically fix your issue
15:13:01 bauzas tl;dr: assume your mistakes
15:13:12 dansmith sdague: that was my point yeah
15:13:28 cfriesen mriedem: will pass on the suggestion
15:13:56 sdague dansmith: ++
15:13:58 mriedem edleafe: think it's sorted out, but still looks like a latent bug
15:14:09 edleafe mriedem: Agree on the change for num_instances
15:14:24 edleafe I can do that quickly - maybe for backport?
15:14:33 mriedem don't use multi-create, and if you do, don't migrate any of those instances if len(hosts) < len(instances)
15:14:42 mriedem edleafe: i think it would be good to have a functional test for thisfirst
15:15:17 mriedem e.g. 2 computes, 2 instances created in a single boot request to compute 1, disable compute 1 and live migrate the isntances to compute 2
15:15:30 mriedem it should fail on the first live migration attempt since you're trying to move 2 instances and we have 1 host
15:15:39 sdague dansmith: I'm +1 on your block of az renames if there are instances in them
15:15:50 sdague honestly, these things should probably be idempotent like flavors
15:15:51 dansmith sdague: cool
15:16:23 dansmith yeah, you'll still have to count instances across cells in the az in question to know whether or not to block it
15:16:31 dansmith just not letting that happen at all is easier still
15:17:14 mriedem edleafe: it's actually a weird check in the filter scheduler driver, i don't really understand why we compare the number of instances to the number of hosts, surely we can create more than one instance per host
15:17:21 openstackgerrit Merged openstack/nova master: doc: Split flavors docs into admin and user guides https://review.openstack.org/501342
15:18:01 edleafe mriedem: are you refrerring to https://github.com/openstack/nova/blob/8a386b055c82df67092a1abc683e7225ef80671e/nova/scheduler/filter_scheduler.py#L86 ?
15:18:12 openstackgerrit Merged openstack/nova master: doc: Add documentation for emulator_thread_policy https://review.openstack.org/501721
15:18:24 mriedem edleafe: yes
15:18:25 edleafe mriedem: if so, that's the number of *selected* hosts, not the total number of hosts
15:18:28 mriedem cfriesen: edleafe: in fact https://review.openstack.org/#/c/491439/
15:18:36 edleafe IOW, we couldn't find hosts for all the instances
15:18:44 esberglu sdague: Can you restore this for us? https://review.openstack.org/#/c/422512/
15:18:52 esberglu If I were to submit a new patch would it restore the changeset?
15:19:13 sdague esberglu: restored
15:19:18 esberglu sdague: tx
15:19:37 sdague esberglu: the patch owner in gerrit, or a core can do restores
15:19:46 sdague but you have to restore before pushing an updated patch
15:20:02 esberglu sdague: Good to know thanks
15:21:43 mriedem claudiub|2: the master branch change for https://review.openstack.org/#/c/505285/ is merged now
15:24:17 edleafe mriedem: I'm confused. If https://review.openstack.org/#/c/491439/ merged, why is the old code still in master?
15:25:23 tasker having trouble live-migrating my last instance out of compute-1. after a suggestion from melwitt, I looked into the scheduler logs and I see that it scheduling the instance, it states that it's looking at the target host, notes that the target host fails and is removed from contention, but doesn't state why it failed -- even with debug logging on.
15:25:34 tasker any thoughts? or is this infrastructure problems?
15:25:56 tasker it's obvious that nova is doing its job, it's just not being verbose enough.
15:26:42 mriedem edleafe: different place
15:27:23 edleafe mriedem: ah
15:27:24 mriedem tasker: depends on if it's the pre-live migration check on the target host, those logs would either be in nova-conductor or nova-compute for the target host
15:27:33 mriedem edleafe: this is why we probably need a functional test for this scenario
15:31:31 cfriesen mriedem: edleave: did bauzas' patch cause the problem? (because the mailing list thread didn't see the IndexError)
15:32:17 openstackgerrit OpenStack Proposal Bot proposed openstack/os-traits master: Updated from global requirements https://review.openstack.org/503646
15:32:20 openstackgerrit OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/502708
15:32:26 mriedem cfriesen: not sure
15:32:42 mriedem it's clear that we need to be consistent in both places though
15:32:50 cfriesen mriedem: for sure
15:38:50 mriedem dansmith: ok that series is all +W
15:48:54 openstackgerrit OpenStack Proposal Bot proposed openstack/os-traits master: Updated from global requirements https://review.openstack.org/503646
15:48:57 openstackgerrit OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/502708
15:52:02 openstackgerrit Chris Dent proposed openstack/nova master: WIP: [placement] manage cache headers https://review.openstack.org/495380
15:53:58 cdent Is now a good time to get this merged, since we are at the beginning of the cycle: https://review.openstack.org/#/c/486829/
15:54:11 gibi could one of the cores look at test only patch? https://review.openstack.org/#/c/496202/ only needs a second +2
15:54:17 cdent fix on MiniDNS
15:55:58 openstackgerrit Evgeny Antyshev proposed openstack/nova master: Vzstorage: synchronize volume connect/disconnect https://review.openstack.org/505708
15:57:34 stephenfin gibi: Looking
15:59:10 gibi stephenfin: thanks
16:05:50 openstackgerrit OpenStack Proposal Bot proposed openstack/os-traits master: Updated from global requirements https://review.openstack.org/503646
16:05:54 openstackgerrit OpenStack Proposal Bot proposed openstack/os-vif master: Updated from global requirements https://review.openstack.org/502708
16:08:21 mriedem gibi: stephenfin: commented on https://review.openstack.org/#/c/496202/
16:08:29 mriedem that seems to be munging together a few different things
16:08:37 openstackgerrit Stephen Finucane proposed openstack/nova master: Add functional migrate force_complete test https://review.openstack.org/496202
16:09:26 stephenfin mriedem: Aye, but they all seemed valid
16:09:41 stephenfin There could be merit in splitting them out though, so I've rebased to take it out of the gate queue
16:09:56 mriedem if we're going to whitelist legacy notifications because they were missed, the change that adds them in should have a test to make sure they are sent
16:10:16 mriedem which i guess is the referenced patch...but those are linked together in any way
16:10:28 mriedem here is an example https://review.openstack.org/#/c/504978/
16:13:05 dansmith mriedem: right on right on right on
16:13:36 gibi mriedem: OK, let's make a separate bug for the missing force_complete test coverage and move the whitelisting into that
16:15:50 mriedem sounds good to me
16:16:01 mriedem dansmith: you're getting older and the patches are staying the same age?
16:16:02 gibi sorry for the mess
16:16:10 dansmith mriedem: lol
16:19:11 bauzas dansmith: sdague: sorry, was running a meeting, but like I said in my comment in https://review.openstack.org/#/c/419502/2 "Modifying an AZ name should not be possible if instances are still in the AZ"
16:19:23 mriedem sdague: jamespage: looks like the devstack patch with the qemu version hack is passing the neutron multinode job that runs live migration
16:19:41 bauzas dansmith: sdague: the only point I have is that if we agree on that, should we have a microversion for that?
16:20:00 dansmith bauzas: we can't support the old thing, so no, IMHO
16:20:27 bauzas dansmith: sdague: I mean, say I wanna update my aggregate and modify the AZ value, should we need a microversion for returning a 40x now ?
16:20:39 bauzas instead of a HTTP200
16:20:54 bauzas in theory, it should, but I guess it's an already broken behaviour
16:21:46 bauzas also, it would require to get the full list of all per-cell instances when updating the aggregate, not a huge deal but still something to do
16:23:05 gibi mriedem, stephenfin: reported the bug https://bugs.launchpad.net/nova/+bug/1718485
16:23:06 openstack Launchpad bug 1718485 in OpenStack Compute (nova) "instance.live.migration.force.complete is not a versioned notification and not whitelisted" [Undecided,New]
16:24:15 mriedem thanks
16:25:21 gibi will push the fix soon
16:32:27 stephenfin gibi: Cool. Ping me when you do, sure
16:34:00 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: check qemu version when calling qemu-img info https://review.openstack.org/505673
16:41:30 mriedem dansmith: i assume the -2 on https://review.openstack.org/#/c/504983/ is just a placeholder for the whole thing to be reviewed and approved?
16:42:03 dansmith mriedem: it was because the top level wasn't wired in, but yeah I figure no reason to land things that aren't reachable until we have some reasonable reviews on the rest of it
16:42:47 mriedem efried: fyi for your ksa endpoint discovery stuff https://review.openstack.org/#/c/485121/
16:42:56 mriedem dansmith: ok, will hit those after lunh
16:42:58 mriedem *lunch
16:44:01 dansmith mriedem: cool thanks
16:44:03 dansmith mriedem: note that I found a bug in a functional test for pagination with that series
16:44:22 dansmith which wasn't an issue before because we were inefficient, but it's nice that it was a test bug and not a functional one
16:45:45 mriedem efried: edmondsw: if the operator has to configure something different because of https://review.openstack.org/#/c/505546/ that's a big no-no for a backport
16:46:08 edmondsw mriedem they don't have to

Earlier   Later