| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-28 | |||
| 20:39:27 | efried | mriedem: requirements bump https://review.opendev.org/#/c/668245/ | |
| 20:39:52 | efried | smcginnis: https://review.opendev.org/#/c/668245/ do you have the power? | |
| 20:40:29 | smcginnis | efried: Just waiting for zuul to report results on that. | |
| 20:40:43 | efried | slave to process you are, sheesh | |
| 20:40:46 | smcginnis | I can +2, but don't want to ninja that one through. | |
| 20:40:47 | smcginnis | :D | |
| 20:42:44 | Sundar | efried, cdent: Is there a master topic/blueprint for all nested magic stuff? I am looking for better query than https://review.opendev.org/#/q/project:openstack/placement+nested | |
| 20:43:06 | efried | Sundar: It's all in a single spec | |
| 20:43:20 | efried | Sundar: https://docs.openstack.org/placement/latest/specs/train/approved/2005575-nested-magic-1.html | |
| 20:43:54 | efried | unless you mean a doc for all currently implemented features. That would be https://docs.openstack.org/placement/latest/usage/provider-tree.html | |
| 20:44:19 | efried | plus the api-reference https://developer.openstack.org/api-ref/placement/ | |
| 20:44:55 | Sundar | We've also had: https://review.opendev.org/657419. Some of your specs, since abandoned (like https://review.opendev.org/658510) were also useful to read | |
| 20:45:26 | Sundar | It would be useful to have a single topic or blueprint for all of these. What do you think? | |
| 20:46:23 | efried | Sundar: That patch was for the arbitrary suffix feature, which is included in the above spec. | |
| 20:46:34 | efried | If you want to track all the patches, you can look at the story | |
| 20:46:40 | efried | https://storyboard.openstack.org/#!/story/2005575 | |
| 20:47:29 | efried | agree that topic branch story/$story-$task isn't as useful as just story/$story for searching purposes. | |
| 20:47:50 | efried | but too late to modify topic for merged patches, sorry. | |
| 20:49:15 | Sundar | NP, got it. Good to know that the spec and storyboard is all there is. Is there a spec/patch on Nova side for how it all comes together -- from flavor to Placement query? | |
| 20:49:43 | Sundar | i.e. 'all there is to read for somebody to understand the whole shebang | |
| 20:55:40 | Sundar | efried: ^ | |
| 21:02:13 | efried | Sundar: There's not likely to be something that high level. And there's as yet no blueprint/spec I'm aware of on the nova side for exploiting these features. | |
| 21:02:41 | melwitt | mriedem: the failed builds thing, my fave | |
| 21:14:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update HostState.*_allocation_ratio earlier https://review.opendev.org/668252 | |
| 21:53:05 | mriedem | wow some fun debugging here | |
| 21:53:24 | mriedem | on initial start of a comptue service, we'll create the compute node, then later try to update it here https://github.com/openstack/nova/blob/324da0532f3b59aa16233a93a260d289e55860fb/nova/compute/resource_tracker.py#L1010 | |
| 21:53:54 | mriedem | if that fails, we've changed old_resources and then a subsequent run will return false from _resource_change and we'll go on to create the resource provider in placement and report inventory | |
| 21:53:58 | mriedem | but the compute node is botched | |
| 21:54:12 | mriedem | and then things in the scheduler blow up if we try to use that node | |
| 22:00:04 | openstackgerrit | Merged openstack/nova stable/queens: libvirt: Do not reraise DiskNotFound exceptions during resize https://review.opendev.org/660363 | |
| 22:20:48 | mriedem | efried: melwitt: would you think that on start of the compute service, if we hit a DBConnectionError here https://github.com/openstack/nova/blob/324da0532f3b59aa16233a93a260d289e55860fb/nova/compute/manager.py#L8129 and startup=true, we should bail and not let the service start? | |
| 22:21:03 | mriedem | i guess it might correct itself... | |
| 22:21:30 | efried | mriedem: it might. But in the meantime, everything is effed. Should at least disable the service at that point. | |
| 22:21:30 | mriedem | maybe that's why we don't bail out | |
| 22:21:47 | mriedem | heh, if you can....need a db connection to do that :) | |
| 22:22:07 | melwitt | I know that in tripleo-land they don't want it to bail out because of puppet starting up services in different orders etc | |
| 22:22:27 | melwitt | they want it to keep retrying the db (for at least a limited time) before bailing out | |
| 22:23:12 | mriedem | yeah, i wonder if https://docs.openstack.org/nova/latest/configuration/config.html#database.use_db_reconnect would help since it's disabled by default i've never tried it | |
| 22:23:32 | melwitt | I've dealt multiple times with tickets getting opened for not services not retrying [temporarily] upon startup | |
| 22:24:58 | melwitt | mriedem: are you saying it is or isn't bailing out today? sorry I don't remember how it behaves at this point. some of our services retry db connection underneath because oslo.service does it. but for some reason that I don't remember, it didn't work on compute | |
| 22:25:15 | mriedem | use_db_reconnect wouldn't help nova since it only tells a decorator in oslo.db it's ok to retry (safe_for_db_retry) which we don't use | |
| 22:26:15 | mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#database.max_retries indicates that we'd have retries by default on connection failures | |
| 22:26:50 | mriedem | with 10 seconds between retries by default (according to retry_interval) | |
| 22:26:55 | melwitt | yeah. last I knew, compute does not retry and I can't remember if I figured out why it doesn't. because from all evidence, it should, the way all our other services do | |
| 22:27:07 | melwitt | let me dig up that old moldy BZ | |
| 22:27:21 | mriedem | it'd be whatever conductor the compute is talking to | |
| 22:28:00 | mriedem | anyway, the connectoin failure doesn't kill the compute on startup so we can get into a weird state with what nova has in the compute nodes table and what is in placement for the provider | |
| 22:28:12 | openstack | Launchpad bug 1834712 in OpenStack Compute (nova) "ResourceTracker._update should restore previous old_resources value if ComputeNode.save fails" [Medium,Triaged] | |
| 22:28:12 | mriedem | i've debugged one issue as a result https://bugs.launchpad.net/nova/+bug/1834712 | |
| 22:28:31 | efried | we need some kind of decorator to... like... @safely_connect | |
| 22:28:37 | mriedem | heh | |
| 22:29:01 | mriedem | i mean, i have a very unrealistic overloaded host situation here | |
| 22:29:02 | melwitt | that bug report makes my head hurt | |
| 22:29:07 | efried | and with that lightning jolt of sarcasm, I'm outta here. See y'all Monday. | |
| 22:29:44 | melwitt | I think that situation would be realistic for our RDO cloud. I've seen so much weird crap happen with it | |
| 22:30:24 | mriedem | yeah this is a critical section it turns out https://github.com/openstack/nova/blob/324da0532f3b59aa16233a93a260d289e55860fb/nova/compute/resource_tracker.py#L1005-L1012 | |
| 22:30:25 | melwitt | like orphaned build request records etc because of spurious db errors | |
| 22:30:50 | melwitt | instance records that live in both cell0 and cell1 (I still don't get how on earth that is possible to happen) | |
| 22:30:51 | mriedem | and _resource_change mutates a thing in the RT that is checked on a later periodic run | |
| 22:31:27 | melwitt | yeah, I see | |
| 22:31:56 | melwitt | I'd be in favor of your rollback idea, or any other way to keep it in sync without killing the service | |
| 22:33:29 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update HostState.*_allocation_ratio earlier https://review.opendev.org/668252 | |
| 22:33:46 | mriedem | yeah killing the service was an idea but not a good one b/c of unknown side effects | |
| 22:34:14 | melwitt | yeah | |
| 23:05:32 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Restore RT.old_resources if ComputeNode.save() fails https://review.opendev.org/668263 | |
| 23:22:22 | openstackgerrit | Merged openstack/nova stable/stein: libvirt: flatten rbd images when unshelving an instance https://review.opendev.org/668118 | |
| #openstack-nova - 2019-06-29 | |||
| 00:28:30 | sean-k-mooney | :) os-tratis 0.15 is released on pypi | |
| 01:29:48 | openstackgerrit | Merged openstack/nova master: Init HostState.failed_builds https://review.opendev.org/668243 | |
| 01:30:42 | openstackgerrit | Merged openstack/nova stable/rocky: libvirt: flatten rbd images when unshelving an instance https://review.opendev.org/668119 | |
| 01:44:20 | efried | mriedem: you still kickin? | |
| 02:00:30 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Add placement request filter for disabled computes https://review.opendev.org/654596 | |
| 02:00:41 | efried | mriedem: ftfy ^ | |
| 02:29:38 | openstackgerrit | Eric Fried proposed openstack/nova master: DRY get_flavor in flavor manage tests https://review.opendev.org/668281 | |
| 02:35:59 | mriedem | efried: <3 | |
| 02:36:10 | efried | :) | |
| 02:36:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/stein: Init HostState.failed_builds https://review.opendev.org/668282 | |
| 02:53:22 | openstackgerrit | Eric Fried proposed openstack/nova master: Add integration testing for heal_allocations https://review.opendev.org/667994 | |
| 11:04:01 | openstackgerrit | Eric Fried proposed openstack/nova master: Add integration testing for heal_allocations https://review.opendev.org/667994 | |
| 15:04:57 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Add irrelevant files in dsvm job again https://review.opendev.org/668293 | |
| #openstack-nova - 2019-06-30 | |||
| 08:48:50 | openstackgerrit | Adrian Chiris proposed openstack/nova stable/queens: Allow driver to properly unplug VIFs on destination on confirm resize https://review.opendev.org/661572 | |
| 08:48:50 | openstackgerrit | Adrian Chiris proposed openstack/nova stable/queens: Move get_pci_mapping_for_migration to MigrationContext https://review.opendev.org/661571 | |
| 22:17:23 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.opendev.org/576020 | |
| 22:17:37 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.opendev.org/576027 | |
| 22:17:49 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.opendev.org/576031 | |
| 22:18:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.opendev.org/576299 | |
| 22:22:47 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.opendev.org/576344 | |
| 22:22:59 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.opendev.org/576673 | |
| 22:23:23 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676 | |
| 22:24:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.opendev.org/576689 | |
| 22:24:37 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.opendev.org/576709 | |
| 22:24:54 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.opendev.org/576712 | |
| 22:25:16 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.opendev.org/566153 | |
| #openstack-nova - 2019-07-01 | |||
| 00:00:11 | adriant | is there any documentation on what the difference (from a Nova perspective) of a volume snapshot vs an instance snapshot? I'm trying to figure out what workflow behind instance action 'createImage' | |
| 00:00:52 | adriant | From what I can tell, if the instance is backed by a persistent volume, it creates a volume snapshot for me | |
| 00:01:51 | adriant | if the instance is backed by an ephemeral volume it triggers the creation of an image (instance snapshot is what the horizon gui calls it). | |
| 00:02:18 | adriant | I've also been told that ' | |
| 00:03:15 | adriant | that an 'instance snapshot' also somehow can take into account multiple volumes attached to an instance and include all of that into a single snapshot, but that functionality isn't documented anywhere, or is it specific to certain volume drivers? | |
| 06:12:09 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Add neutron-tempest-iptables_hybrid job to experimental queue https://review.opendev.org/667154 | |
| 06:12:10 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Revert resize: wait for events according to hybrid plug https://review.opendev.org/667177 | |
| 06:46:42 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Revert resize: wait for events according to hybrid plug https://review.opendev.org/667177 | |