| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-02-13 | |||
| 16:36:24 | mriedem | i wonder why the nova admin guide docs on schedulers gives a reference to the volume scheduler in cinder https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html | |
| 16:36:37 | mriedem | the link is broken, but i don't know if i should fix it or just remove that part | |
| 16:37:22 | gibi | mriedem: does this trace mean that nova does not handle the exception? https://bugs.launchpad.net/nova/+bug/1749215/comments/3 | |
| 16:37:23 | openstack | Launchpad bug 1749215 in OpenStack Compute (nova) "Allocations not deleted on failed resize_instance" [Medium,In progress] - Assigned to Claudiu Belu (cbelu) | |
| 16:37:53 | mriedem | gibi: depends on what you mean by handle | |
| 16:38:54 | mriedem | that's in the _error_out_instance_on_exception context manager, | |
| 16:39:09 | mriedem | which is setting the instance to ERROR due to an unexpected exception | |
| 16:39:19 | mriedem | that's likely a bug, the instance shouldn't be in ERROR | |
| 16:39:32 | mriedem | for "Resize error: Unable to resize disk down." | |
| 16:39:33 | gibi | mriedem: in my log this is the last place nova mentions the ResizeError | |
| 16:39:33 | openstackgerrit | Merged openstack/nova stable/pike: Make eventlet hub use a monotonic clock https://review.openstack.org/537811 | |
| 16:40:08 | mriedem | gibi: sure, that's because resize_instance is running on the source host, via rpc cast from prep_resize on the dest host | |
| 16:40:09 | gibi | mriedem: I in that context manager this is coming from we does not delete claims | |
| 16:40:14 | mriedem | so if that fails, we should cleanup and exit | |
| 16:40:47 | mriedem | the claim is made in prep_resize on the dest | |
| 16:41:02 | mriedem | before we rpc cast to resize_instance | |
| 16:41:38 | gibi | mriedem: ahh that was the missing piece | |
| 16:41:50 | gibi | mriedem: thanks, now I have a place to look at | |
| 16:42:53 | mriedem | what i'm not sure about is if/when the dest host RT updates its usage after the migration fails to remove the claim | |
| 16:43:52 | mriedem | # Grab all in-progress migrations: | |
| 16:43:52 | mriedem | migrations = objects.MigrationList.get_in_progress_by_host_and_node( | |
| 16:43:53 | mriedem | context, self.host, nodename) | |
| 16:43:53 | mriedem | self._pair_instances_to_migrations(migrations, instances) | |
| 16:43:53 | mriedem | self._update_usage_from_migrations(context, migrations, nodename) | |
| 16:43:59 | mriedem | that should update the dest host RT at some point | |
| 16:44:08 | mriedem | once the migration fails, it will no longer be in that in-progress list | |
| 16:46:37 | gibi | mriedem: in my env I don't see that this Exception handler is called during the failed resize: https://github.com/openstack/nova/blob/fa6c0f9cb14f1b4ce4d9b1dbacb1743173089986/nova/compute/manager.py#L4124 | |
| 16:47:00 | mriedem | gibi: it's not, | |
| 16:47:08 | mriedem | because we've already rpc cast to the source host | |
| 16:47:26 | mriedem | https://github.com/openstack/nova/blob/fa6c0f9cb14f1b4ce4d9b1dbacb1743173089986/nova/compute/manager.py#L4085 | |
| 16:47:39 | mriedem | that's why claudiub's original fix was to change that to an rpc call | |
| 16:47:51 | mriedem | so the failure would come back to prep_resize on the dest and we'd run that cleanup code | |
| 16:48:03 | mriedem | but rpc calls just mean messaging timeouts | |
| 16:48:06 | mriedem | so i'd rather not do that | |
| 16:48:08 | openstackgerrit | Merged openstack/python-novaclient stable/queens: Updated from global requirements https://review.openstack.org/543741 | |
| 16:48:09 | gibi | mriedem: OK, then I understand what I see. | |
| 16:48:18 | gibi | mriedem: thanks again | |
| 16:48:45 | gibi | mriedem: I have to run now but if the problem still open then I can look at it again tomorrow | |
| 17:04:50 | hegemoOn | hello there | |
| 17:05:04 | hegemoOn | is it possible to define the number of queue in an image metadata | |
| 17:05:15 | hegemoOn | when using virtion-scsi ? | |
| 17:09:05 | mriedem | stephenfin: on https://review.openstack.org/#/c/531413/ - i think i might decouple the re-wording part so we can backport that, since i see some other config guide docs that reference that option | |
| 17:11:00 | stephenfin | mriedem: Sure thing. The reservation was because we haven't removed deprecated aliases before, that I'm aware of (there's little cost in keeping them). Worth making sure it was intentional | |
| 17:11:13 | mriedem | i'm sure we've removed deprecated aliases before | |
| 17:11:29 | mriedem | maybe not recently, but it was pretty common cleanup back in the day | |
| 17:12:01 | stephenfin | :D | |
| 17:12:15 | stephenfin | numa-aware-vswitches has me plenty busy :) | |
| 17:14:44 | hegemoOn | in virsh xml you have <driver queues='n'> | |
| 17:14:56 | hegemoOn | for virtio-scsi | |
| 17:15:09 | hegemoOn | i cannot see a way to define n in nova | |
| 17:16:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix and update compute schedulers config guide https://review.openstack.org/544010 | |
| 17:19:28 | openstackgerrit | Chris Dent proposed openstack/nova master: Move db MAX constants to own file https://review.openstack.org/543469 | |
| 17:21:17 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove the deprecated scheduler_driver_task_period option https://review.openstack.org/531413 | |
| 17:21:18 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Clarify the help text for [scheduler]periodic_task_interval https://review.openstack.org/544015 | |
| 17:24:50 | stephenfin | mriedem: Should I have +Wd this, given that you've a -2 on the following patch? https://review.openstack.org/#/c/539738/ | |
| 17:25:00 | stephenfin | I can rebase and take it out of the gate if not | |
| 17:25:32 | mriedem | it's a bug fix | |
| 17:25:35 | mriedem | so i don't think so | |
| 17:25:48 | stephenfin | Phew. Okidok | |
| 17:25:53 | mriedem | i mean, it's not a problem | |
| 17:27:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove warning in feature support matrix page https://review.openstack.org/544017 | |
| 17:30:59 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Check for leaked server resource allocations in post_test_hook https://review.openstack.org/538510 | |
| 17:31:14 | openstackgerrit | James E. Blair proposed openstack/python-novaclient stable/ocata: Zuul: Remove project name https://review.openstack.org/544018 | |
| 17:32:00 | openstackgerrit | Merged openstack/nova stable/pike: Query all cells for service version in _validate_bdm https://review.openstack.org/541036 | |
| 17:33:03 | openstackgerrit | Eric Berglund proposed openstack/nova master: Use correct arguments in task inits https://review.openstack.org/543571 | |
| 17:34:42 | mriedem | lyarwood: dansmith: bauzas: so before i request the next pike release, do i have to bump the minor version to 16.1.0 for https://review.openstack.org/#/c/528330/ because it's a schema migration; i will forever remember the red hat product team lashing i got for *not* doing something like that back in newton | |
| 17:35:49 | dansmith | um | |
| 17:35:51 | dansmith | okay | |
| 17:36:02 | dansmith | bump the minor _to_ what? | |
| 17:36:10 | mriedem | 16.0.4 -> 16.1.0 | |
| 17:36:15 | dansmith | oh I see | |
| 17:36:18 | mriedem | i would normally just do 16.0.5 | |
| 17:36:23 | mriedem | but caught hell for doing that once | |
| 17:36:24 | dansmith | I don't think I know anything about that | |
| 17:40:47 | mriedem | alright 16.1.0 it is https://review.openstack.org/#/c/544020/ | |
| 18:29:43 | lyarwood | mriedem: 16.1.0 is fine by me, this is still an optional schema migration until queens anyway right so anyone not running migrations after updating nova during stable/pike will pick it up when they upgrade to queens. | |
| 18:31:15 | lyarwood | mriedem: FWIW with TripleO/RDO I can't see us ever running schema migrations with each minor update (16.0.4 to 16.1.0 etc) | |
| 18:31:48 | dansmith | lyarwood: apparently a minor version change kicks the "do the db sync" flag on | |
| 18:32:21 | lyarwood | dansmith: not in any of our tooling | |
| 18:32:32 | lyarwood | dansmith: just the poor ops guy who notices the change | |
| 18:32:46 | dansmith | lyarwood: supposedly bauzas beat up mriedem about it last time on that basis | |
| 18:34:29 | lyarwood | owalsh: ^ re minor updates on TripleO, we don't run schema migrations at all right? | |
| 18:35:30 | dansmith | if not then bauzas should pay for mriedem's therapist bills | |
| 18:35:50 | lyarwood | dansmith: I can only assume his point was that we should still highlight to ops etc that the update contains a schema migration by bumping the minor release or something | |
| 18:39:49 | owalsh | lyarwood: correct, just restart services | |
| 18:40:11 | lyarwood | owalsh: cool thanks | |
| 18:40:24 | dansmith | death match round 2, dublin | |
| 18:53:36 | mriedem | fork in the kidneys, check | |
| 19:55:15 | openstackgerrit | Merged openstack/nova master: Replace Chinese quotes to English quotes https://review.openstack.org/543349 | |
| 19:56:31 | dansmith | efried: jaypipes: seen the question on this? https://review.openstack.org/#/c/540111/3 | |
| 19:56:45 | dansmith | I was about to reply, but then realized I was misunderstanding his concern and I think it's probably valid | |
| 19:57:06 | dansmith | hoping that there's some detail of how you see that working that wouldn't actually break it | |
| 19:57:46 | efried | dansmith: It's been on my list to look at, but was rapidly getting buried. Thanks for bringing it back to the top. Looking.... | |
| 19:58:58 | mriedem | been wondering the same type of thing with traits, | |
| 19:59:12 | mriedem | the ironic driver will blow away any traits that aren't on the ironic node | |
| 19:59:16 | mriedem | rather than try to merge the | |
| 19:59:18 | mriedem | *them | |
| 19:59:27 | dansmith | well, there needs to be some amount of that I think, | |
| 19:59:36 | dansmith | although we can't blow them all away in this case I think | |
| 19:59:49 | dansmith | was hoping there was some "only blows away at the given level" detail or something | |