Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-03-12
13:37:36 gibi OK, so in the fix we simply refresh nova's view based on neutron's view and only unbind if the vif is still belongs to the instance
13:37:42 gibi that could work
13:38:32 gibi in a similar scenario with bandwidth when neutron admin forcefully unbind the port then the resource allocation view will become inaccurate
13:39:39 gibi then when the old nova instance is deleted the whole resource allocation is deleted in placement so the inventory become correct eventually
13:41:06 gibi hm, in the bug, the instance is deleted while the compute was still down. I'm not sure when the allocation for such instance is deleted from placement
13:43:46 gibi OK. compute.api._local_delete calls delete_allocation_for_instance() so if the admin unbinds the port _after_ the instance is local deleted then the resource view is in a correct state
13:46:42 gibi mriedem: So I don't see that this sceanrio leads to any issue with bandwidth resource
13:46:48 mriedem this bug predates placement by a long shot
13:46:53 mriedem i'm not saying it does,
13:47:04 mriedem we have just talked about the weird corner case of unbinding ports out of band
13:51:18 gibi mriedem: yeah, the use case seems valid
13:53:50 openstackgerrit Jack Ding proposed openstack/nova master: Preserve UEFI NVRAM variable store https://review.openstack.org/621646
13:56:13 openstack Launchpad bug 1816399 in OpenStack Compute (nova) "The periodic task to clean up expired console_auth tokens is invalid" [Undecided,In progress] - Assigned to Takashi NATSUME (natsume-takashi)
13:56:13 mriedem melwitt: is this really stein rc potential? https://bugs.launchpad.net/nova/+bug/1816399 - isn't that a latent issue since rocky?
16:10:08 openstackgerrit Matt Riedemann proposed openstack/nova-specs master: Re-propose cross-cell-resize spec for Train https://review.openstack.org/642807
16:12:15 dansmith mriedem: regarding this: https://review.openstack.org/#/c/567701, where is the az set on the reqspec during live migration?
16:14:41 mriedem the request spec az is set on server create and never changes
16:15:10 dansmith okay I'm missing something about your assertion in the commit message then
16:16:07 dansmith you say the reqspec.az is what the az filter looks at, and that means the instance can be moved across azs, which I assumed meant we update it so after you move it it's correct,
16:16:20 dansmith but if not, then if you force a move across azs, and then do something like a resize,
16:16:34 dansmith the instance will pop back over to the old az during scheduling, yeah? Is that what you meant?
16:19:33 mriedem replied inline
16:20:38 dansmith mriedem: okay I'm more talking about the case where az *is* set
16:20:56 dansmith agree that if it's not set, it shouldn't matter, that's the easy case
16:21:18 mriedem i wasn't considering the force case here,
16:21:25 mriedem that's a separate f'ed up issue that evacuate will also suffer from
16:21:53 mriedem but yeah, if you request server create in AZ1, admin forces live migrate or evacuate to AZ2, and then the user resizes, they'll wind up back in AZ1
16:21:56 mriedem b/c of the request spec
16:22:07 mriedem another example of why we removed the force parameter
16:22:13 mriedem but it's a separate bug imo
16:24:25 dansmith mriedem: I'm more interested in the v2.1 case, since that's all that osc seems to support right now, which by my reading is kinda equivalent to force=True with a destination right?
16:24:43 dansmith mriedem: I'm actually asking about this for another reason, not for confirmation of your patch itself
16:26:54 mriedem "which by my reading is kinda equivalent to force=True with a destination right" yup
16:27:10 mriedem also,
16:27:16 mriedem https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/server.html#server-migrate is totally borked in this regard,
16:27:24 dansmith yes
16:27:26 mriedem if you're using osc to do live migrations, you're *always* forcing by default
16:27:29 mriedem it's mega fucked
16:28:05 mriedem i've talked with dean about this and unfucking it means essentially a major version bump
16:28:10 mriedem b/c it has to be an interface change
16:28:46 mriedem what you're talking about with force not updating the request spec az is a latent bug yeah
16:29:41 mriedem https://review.openstack.org/#/c/627801/
16:30:20 mriedem maybe should queue that up for the ptg
16:33:48 mriedem mordred: i added an item to the nova train ptg etherpad about finally unbreaking the osc server migrate interface, since we should shit or get off that pot
16:34:05 mriedem ~L104 https://etherpad.openstack.org/p/nova-ptg-train
16:34:41 dansmith mriedem: on the --force removal, we just removed that flag and use the presence of the host parameter to indicate that we're serious?
16:34:59 dansmith because I thought the point was to be able to provide a host and let the scheduler sanity check it, and force was "this is an emergency just do it"
16:35:17 mriedem yeah - the provided host is a requested destination and the only one the scheduler will look at
16:35:44 dansmith so we've lost the emergency button?
16:35:57 mriedem if you're using the latest microversion yes,
16:36:07 mriedem but you can still 2.1 yourself into oblivion if you want
16:36:22 dansmith I vaguely recall these discussions of course, I'm just trying to fit my head around the current state
16:36:31 mriedem we also intentionally broke force for any servers that have nested allocations
16:36:48 mriedem regardless of microversion
16:37:07 dansmith if we start saying 2.1 is how you override things in an emergency then we really need to put the cap on "we're never bumping the minimum" :)
16:37:34 dansmith in the 2.1 case where you provide a host, we're still updating allocations though as far as I can see
16:37:39 dansmith you just mean nested allocs specifically?
16:37:40 mriedem we'll just do something dumb like adding the force parameter back
16:37:44 mriedem forcev2!
16:38:24 mriedem dansmith: yes if you force (2.1 where host is not None), we copy the allocations over to the forced dest compute node provider
16:38:29 mriedem but only if they are flat allocations
16:38:44 mordred mriedem: ok. happy to help fixing that
16:39:02 mriedem dansmith: that's this part https://github.com/openstack/nova/blob/e608568518ed91a0cbf08f779c5adb851762d80a/nova/conductor/tasks/live_migrate.py#L111
16:40:42 dansmith mriedem: ack eyah
16:42:17 mriedem so it sounds like you want me to change https://review.openstack.org/#/c/567701/ to at the very least save off the current instance.availabilty_zone before setting it based on the new dest host selection and then if we hit rollback to set it back to the saved off value?
16:42:27 mriedem which would only happen if i guess the rpc cast blew up?
16:43:15 mriedem dansmith: doing it on the compute when we change instance.host is a non-starter if you care about up-calls
16:44:22 dansmith mriedem: you can do it in the cell on the instance :)
16:44:46 dansmith as long as we pass the az into compute so it doesn't have to look it up, which is what I was saying was likely an rpc change
16:45:21 dansmith mriedem: I don't know what I want really.. what you're doing seems wrong, but what I did was equally wrong although slightly less concerning because of how those actions work
16:45:44 dansmith so I dunno, do we justify it with prior sins?
16:45:45 mriedem i can't backport an rpc change
16:46:32 openstack Launchpad bug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during reschedule - get_host_availability_zone up-call" [Medium,Triaged]
16:46:32 mriedem we already have several places where we know we need to pass the az down to compute to avoid up-calls https://bugs.launchpad.net/nova/+bug/1781286
16:47:40 mriedem which reminds me https://review.openstack.org/#/c/581912/
16:48:20 dansmith how about we just toss a grenade in this mofo and run?
16:48:47 mriedem wfm
16:50:54 cfriesen what's the current behaviour for resizing from a flavor with swap to one without swap? Do we allow it?
16:51:19 mriedem cfriesen: i'm fairly certain you ask this once per quarter
16:51:25 mriedem it's time for a tattoo
16:51:56 cfriesen as of last year it was supposed to work, but one of our testers just said it was busted
16:52:24 cfriesen I'm looking for automated tests of it
16:52:32 openstack Launchpad bug 1762423 in OpenStack Compute (nova) "Resize to swapless flavor still looking for swap" [Low,In progress] - Assigned to jichenjc (jichenjc)
16:52:32 mriedem https://bugs.launchpad.net/nova/+bug/1762423
16:53:56 openstack Launchpad bug 1552777 in OpenStack Compute (nova) "resizing from flavor with swap to one without swap puts instance into Error status" [Medium,Fix released] - Assigned to Kam Nasim (knasim-wrs)
16:53:56 cfriesen according to my comment at the bottom of https://bugs.launchpad.net/nova/+bug/1552777 it was working in Pike
16:55:00 mriedem cfriesen: were you testing nova or starlingx?
16:55:46 cfriesen mriedem: pretty sure it was devstack, but can't say for sure now
16:56:51 mriedem dansmith: unrelated softball https://review.openstack.org/#/c/642611/
16:57:17 dansmith *crack*
17:05:17 openstackgerrit Balazs Gibizer proposed openstack/nova master: Documentation for bandwidth support https://review.openstack.org/642064
17:43:04 mriedem efried: i'm mostly ok with https://review.openstack.org/#/c/640879/ but comments inline - the ironicclient valid_interfaces patch
17:45:27 mriedem i don't know if you can wordsmith something in as a code comment and commit message details, or if it's not worth it...
18:04:28 openstackgerrit Matt Riedemann proposed openstack/nova master: RT: replace _instance_in_resize_state with _is_trackable_migration https://review.openstack.org/560467
18:09:11 openstackgerrit Jeremy Stanley proposed openstack/nova stable/rocky: [DNM] Exercising base-test for a rocky change https://review.openstack.org/642582
18:10:15 mriedem stephenfin: regarding https://review.openstack.org/#/c/641653/ - if that feature *never worked* since queens, doesn't backporting this stuff now essentially mean we're backporting a feature?
18:10:39 openstackgerrit Jeremy Stanley proposed openstack/nova master: [DNM] Exercising base-test for a master change https://review.openstack.org/642580
18:16:21 openstackgerrit Jeremy Stanley proposed openstack/nova stable/rocky: [DNM] Exercising base-test for a master change https://review.openstack.org/642582
18:17:37 dansmith mriedem: hmm.. "claiming will always fail"
18:18:05 dansmith mriedem: I'm trying to figure out what the upgrade impact of this is

Earlier   Later