| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-28 | |||
| 21:53:54 | mriedem | if that fails, we've changed old_resources and then a subsequent run will return false from _resource_change and we'll go on to create the resource provider in placement and report inventory | |
| 21:53:58 | mriedem | but the compute node is botched | |
| 21:54:12 | mriedem | and then things in the scheduler blow up if we try to use that node | |
| 22:00:04 | openstackgerrit | Merged openstack/nova stable/queens: libvirt: Do not reraise DiskNotFound exceptions during resize https://review.opendev.org/660363 | |
| 22:20:48 | mriedem | efried: melwitt: would you think that on start of the compute service, if we hit a DBConnectionError here https://github.com/openstack/nova/blob/324da0532f3b59aa16233a93a260d289e55860fb/nova/compute/manager.py#L8129 and startup=true, we should bail and not let the service start? | |
| 22:21:03 | mriedem | i guess it might correct itself... | |
| 22:21:30 | mriedem | maybe that's why we don't bail out | |
| 22:21:30 | efried | mriedem: it might. But in the meantime, everything is effed. Should at least disable the service at that point. | |
| 22:21:47 | mriedem | heh, if you can....need a db connection to do that :) | |
| 22:22:07 | melwitt | I know that in tripleo-land they don't want it to bail out because of puppet starting up services in different orders etc | |
| 22:22:27 | melwitt | they want it to keep retrying the db (for at least a limited time) before bailing out | |
| 22:23:12 | mriedem | yeah, i wonder if https://docs.openstack.org/nova/latest/configuration/config.html#database.use_db_reconnect would help since it's disabled by default i've never tried it | |
| 22:23:32 | melwitt | I've dealt multiple times with tickets getting opened for not services not retrying [temporarily] upon startup | |
| 22:24:58 | melwitt | mriedem: are you saying it is or isn't bailing out today? sorry I don't remember how it behaves at this point. some of our services retry db connection underneath because oslo.service does it. but for some reason that I don't remember, it didn't work on compute | |
| 22:25:15 | mriedem | use_db_reconnect wouldn't help nova since it only tells a decorator in oslo.db it's ok to retry (safe_for_db_retry) which we don't use | |
| 22:26:15 | mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#database.max_retries indicates that we'd have retries by default on connection failures | |
| 22:26:50 | mriedem | with 10 seconds between retries by default (according to retry_interval) | |
| 22:26:55 | melwitt | yeah. last I knew, compute does not retry and I can't remember if I figured out why it doesn't. because from all evidence, it should, the way all our other services do | |
| 22:27:07 | melwitt | let me dig up that old moldy BZ | |
| 22:27:21 | mriedem | it'd be whatever conductor the compute is talking to | |
| 22:28:00 | mriedem | anyway, the connectoin failure doesn't kill the compute on startup so we can get into a weird state with what nova has in the compute nodes table and what is in placement for the provider | |
| 22:28:12 | mriedem | i've debugged one issue as a result https://bugs.launchpad.net/nova/+bug/1834712 | |
| 22:28:12 | openstack | Launchpad bug 1834712 in OpenStack Compute (nova) "ResourceTracker._update should restore previous old_resources value if ComputeNode.save fails" [Medium,Triaged] | |
| 22:28:31 | efried | we need some kind of decorator to... like... @safely_connect | |
| 22:28:37 | mriedem | heh | |
| 22:29:01 | mriedem | i mean, i have a very unrealistic overloaded host situation here | |
| 22:29:02 | melwitt | that bug report makes my head hurt | |
| 22:29:07 | efried | and with that lightning jolt of sarcasm, I'm outta here. See y'all Monday. | |
| 22:29:44 | melwitt | I think that situation would be realistic for our RDO cloud. I've seen so much weird crap happen with it | |
| 22:30:24 | mriedem | yeah this is a critical section it turns out https://github.com/openstack/nova/blob/324da0532f3b59aa16233a93a260d289e55860fb/nova/compute/resource_tracker.py#L1005-L1012 | |
| 22:30:25 | melwitt | like orphaned build request records etc because of spurious db errors | |
| 22:30:50 | melwitt | instance records that live in both cell0 and cell1 (I still don't get how on earth that is possible to happen) | |
| 22:30:51 | mriedem | and _resource_change mutates a thing in the RT that is checked on a later periodic run | |
| 22:31:27 | melwitt | yeah, I see | |
| 22:31:56 | melwitt | I'd be in favor of your rollback idea, or any other way to keep it in sync without killing the service | |
| 22:33:29 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update HostState.*_allocation_ratio earlier https://review.opendev.org/668252 | |
| 22:33:46 | mriedem | yeah killing the service was an idea but not a good one b/c of unknown side effects | |
| 22:34:14 | melwitt | yeah | |
| 23:05:32 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Restore RT.old_resources if ComputeNode.save() fails https://review.opendev.org/668263 | |
| 23:22:22 | openstackgerrit | Merged openstack/nova stable/stein: libvirt: flatten rbd images when unshelving an instance https://review.opendev.org/668118 | |
| #openstack-nova - 2019-06-29 | |||
| 00:28:30 | sean-k-mooney | :) os-tratis 0.15 is released on pypi | |
| 01:29:48 | openstackgerrit | Merged openstack/nova master: Init HostState.failed_builds https://review.opendev.org/668243 | |
| 01:30:42 | openstackgerrit | Merged openstack/nova stable/rocky: libvirt: flatten rbd images when unshelving an instance https://review.opendev.org/668119 | |
| 01:44:20 | efried | mriedem: you still kickin? | |
| 02:00:30 | openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Add placement request filter for disabled computes https://review.opendev.org/654596 | |
| 02:00:41 | efried | mriedem: ftfy ^ | |
| 02:29:38 | openstackgerrit | Eric Fried proposed openstack/nova master: DRY get_flavor in flavor manage tests https://review.opendev.org/668281 | |
| 02:35:59 | mriedem | efried: <3 | |
| 02:36:10 | efried | :) | |
| 02:36:22 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/stein: Init HostState.failed_builds https://review.opendev.org/668282 | |
| 02:53:22 | openstackgerrit | Eric Fried proposed openstack/nova master: Add integration testing for heal_allocations https://review.opendev.org/667994 | |
| 11:04:01 | openstackgerrit | Eric Fried proposed openstack/nova master: Add integration testing for heal_allocations https://review.opendev.org/667994 | |
| 15:04:57 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Add irrelevant files in dsvm job again https://review.opendev.org/668293 | |
| #openstack-nova - 2019-06-30 | |||
| 08:48:50 | openstackgerrit | Adrian Chiris proposed openstack/nova stable/queens: Move get_pci_mapping_for_migration to MigrationContext https://review.opendev.org/661571 | |
| 08:48:50 | openstackgerrit | Adrian Chiris proposed openstack/nova stable/queens: Allow driver to properly unplug VIFs on destination on confirm resize https://review.opendev.org/661572 | |
| 22:17:23 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.opendev.org/576020 | |
| 22:17:37 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.opendev.org/576027 | |
| 22:17:49 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.opendev.org/576031 | |
| 22:18:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.opendev.org/576299 | |
| 22:22:47 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.opendev.org/576344 | |
| 22:22:59 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.opendev.org/576673 | |
| 22:23:23 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676 | |
| 22:24:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.opendev.org/576689 | |
| 22:24:37 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.opendev.org/576709 | |
| 22:24:54 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.opendev.org/576712 | |
| 22:25:16 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.opendev.org/566153 | |
| #openstack-nova - 2019-07-01 | |||
| 00:00:11 | adriant | is there any documentation on what the difference (from a Nova perspective) of a volume snapshot vs an instance snapshot? I'm trying to figure out what workflow behind instance action 'createImage' | |
| 00:00:52 | adriant | From what I can tell, if the instance is backed by a persistent volume, it creates a volume snapshot for me | |
| 00:01:51 | adriant | if the instance is backed by an ephemeral volume it triggers the creation of an image (instance snapshot is what the horizon gui calls it). | |
| 00:02:18 | adriant | I've also been told that ' | |
| 00:03:15 | adriant | that an 'instance snapshot' also somehow can take into account multiple volumes attached to an instance and include all of that into a single snapshot, but that functionality isn't documented anywhere, or is it specific to certain volume drivers? | |
| 06:12:09 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Add neutron-tempest-iptables_hybrid job to experimental queue https://review.opendev.org/667154 | |
| 06:12:10 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Revert resize: wait for events according to hybrid plug https://review.opendev.org/667177 | |
| 06:46:42 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Revert resize: wait for events according to hybrid plug https://review.opendev.org/667177 | |
| 08:18:22 | gibi | stephenfin: hi! could you please look back to https://review.opendev.org/#/c/641679/ ? you were +2 before | |
| 09:06:02 | kashyap | gibi: Hi, can you ACK|NACK this stable/rocky backport, please? -- https://review.opendev.org/#/c/667436/ (libvirt: Rework 'EBUSY' (SIGKILL) error handling code path) | |
| 09:06:22 | kashyap | It already has a +2. (And FWIW, its stable/queens counterpart is merged.) | |
| 09:08:11 | gibi | kashyap: I have no +2 power on stable, but the backport looks OK to me | |
| 09:08:38 | kashyap | gibi: Hmm, very bizarre; johnthetubaguy ^ Want to put the stable/rocky one, too? | |
| 09:08:54 | gibi | kashyap: and I think you ment stable/stain counterpart :) | |
| 09:09:08 | kashyap | gibi: Err, indeed :-) | |
| 09:09:21 | johnthetubaguy | I can take a peak | |
| 09:09:41 | gibi | kashyap: this is the stable core group https://review.opendev.org/#/admin/groups/540,members | |
| 09:09:49 | kashyap | Yeah, saw that | |
| 09:09:57 | kashyap | johnthetubaguy: Thanks; you ACKed one of the backports | |
| 09:10:21 | kashyap | johnthetubaguy: Also, I'm almost done through reworking the OVMF spec, hope to push out a new version EOD or tomm. Want to get it past the "finish line" | |
| 09:10:33 | kashyap | (Addressing your feedback, i.e.) | |
| 09:11:50 | johnthetubaguy | kashyap: ah, cool, it is spec review day tomorrow, although I am on a train to bristol and all day meeting tomorrow, so going through stuff today | |
| 09:12:11 | kashyap | johnthetubaguy: Oh, okay; is tomm the last deadline or what for specs? | |
| 09:12:36 | kashyap | Regardless, /me will work through it. Was busy with some downstream stuff | |
| 09:12:39 | johnthetubaguy | more just a big push, that you might want to get cought up in, if you can | |
| 09:13:43 | johnthetubaguy | kashyap: no worries, we all know that feeling of being pulled in 12 directions | |
| 09:14:20 | johnthetubaguy | actually little francis literately pulls at my leg now when he decides I am not listening | |
| 09:14:55 | kashyap | Hehe | |
| 09:18:52 | openstackgerrit | pengyuesheng proposed openstack/python-novaclient master: Add Python 3 Train unit tests https://review.opendev.org/668395 | |
| 09:20:41 | openstackgerrit | pengyuesheng proposed openstack/python-novaclient master: Blacklist sphinx 2.1.0 (autodoc bug) https://review.opendev.org/668398 | |
| 10:09:11 | ysandeep | #openstack-nova, Is there OpenStack cli alternative for this command "nova boot --flavor m1.medium --block-device source=image,id=<image-id>,dest=volume,size=10,shutdown=preserve,bootindex=0 --nic net-id=<net-id> --key-name mykey test_instance" | |
| 10:37:55 | openstackgerrit | Merged openstack/nova master: Add integration testing for heal_allocations https://review.opendev.org/667994 | |
| 10:46:17 | tssurya | bauzas: stable stein easy review if you have time - https://review.opendev.org/#/c/667948/ | |
| 10:46:35 | tssurya | and the corresponding rocky one: https://review.opendev.org/#/c/667955/1 | |