| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-18 | |||
| 18:51:36 | sean-k-mooney | ah ok i remember talking about not supporting it in the osc version | |
| 18:51:49 | mloza | mriedem: I guess I hitting that orphan issue. I possibly cleared the migration records before. Now wonder I everytime I launch a new VM it goes always goes to a specific host. | |
| 18:51:49 | sean-k-mooney | speaking of which i should update that patch. | |
| 18:52:15 | mloza | What would be a remedy to this issue? | |
| 18:52:35 | sean-k-mooney | mloza: what release are you running | |
| 18:52:41 | mloza | stable/stein | |
| 18:52:57 | mriedem | sean-k-mooney: please tell me you ran pep8 on this https://review.opendev.org/#/c/669738/8.. | |
| 18:52:58 | sean-k-mooney | ok i was wondering if you had the old failed builds behavior. | |
| 18:52:59 | mriedem | https://review.opendev.org/#/c/669738/ | |
| 18:54:11 | sean-k-mooney | i think i did | |
| 18:54:18 | sean-k-mooney | i know i had to fix the odering of the imports | |
| 18:54:34 | mriedem | mloza: this is a work in progress but you might be able to use this https://review.opendev.org/#/c/670112/ - that would also provide valuable feedback on what it reports | |
| 18:55:38 | mriedem | mloza: you can use https://docs.openstack.org/osc-placement/latest/ to see what is consuming resource allocations on your nodes (resource providers) | |
| 18:55:55 | mriedem | i would start by investigating a particular host that you think should be available but placement is saying it's not | |
| 18:56:37 | mloza | will take a look | |
| 18:56:40 | mriedem | so using this https://docs.openstack.org/osc-placement/latest/cli/index.html#resource-provider-show with the --allocations option | |
| 18:57:02 | mriedem | the allocations are keyed by consumer uuid, which in the case of nova since queens can be migrations or instances | |
| 18:57:23 | mriedem | migrations for the source host during a migration, instances for the dest host | |
| 18:57:57 | mriedem | so let's say you evacuated all instances from a down host but placement is still reporting allocations against that host - those allocation consumers are likely migration records | |
| 18:57:59 | mriedem | and were orphaned | |
| 18:58:33 | mriedem | there are a couple of related bugs in this patch https://review.opendev.org/#/c/678100/ | |
| 19:18:35 | mriedem | melwitt: how are you feeling about this? https://review.opendev.org/#/c/541420/ i split the refactor out and without the compat code for old cinder it's quite a bit simpler than the last time you looked. | |
| 20:03:55 | KeithMnemonic1 | I have a customer running into an odd issue with multipath during migration (Pike). To make a long story short the LUNS assignments are getting mixed up somewhere. geguileo mentioned it possibly could be this https://review.opendev.org/#/c/551302/ I did a quick look at cherry-picking and it seems there are a ton of dependencies. Does that seems accurate or does someone have some magic to cherry-pick it to | |
| 20:03:55 | KeithMnemonic1 | pike? | |
| 20:15:29 | KeithMnemonic1 | mriedem, melwitt any thoughts on my question above | |
| 20:16:27 | mriedem | coincidentally that came up yesterday on pike https://review.opendev.org/#/c/670016/ | |
| 20:16:44 | mriedem | ^ will not fix your issue though | |
| 20:17:33 | mriedem | looking at the conflicts on the queens backport i'm not surprised that there would be conflicts going to pike, but w/o doing it myself i do'nt know how bad it is, | |
| 20:17:45 | mriedem | red hat probably doesn't care to backport that since i don't think they are supporting pike | |
| 20:23:47 | mriedem | efried_pto: are you going to be running the meeting tomorrow morning or should someone else? | |
| 20:39:26 | KeithMnemonic1 | mriedem i was using aspiers git-deps and it was a long list. I am not experienced enough to know if there are any shortcuts | |
| 20:48:30 | KeithMnemonic1 | I was looking to confirm with someone if my inital look was correct (that it is too cumbersome) or if it is not that bad | |
| 20:50:10 | mriedem | i can peek in a bit | |
| 20:51:07 | melwitt | mriedem: I dunno, haven't looked at it in a long time. will go through it today | |
| 20:56:37 | melwitt | KeithMnemonic1, mriedem: I can confirm that we haven't backported https://review.opendev.org/#/c/551302/ beyond queens downstream either, so I also don't know what to expect as far as how gnarly the backport will be without actually trying to do it | |
| 20:58:12 | KeithMnemonic1 | I took a look and it seemed gnarly but was hoping for a second opinion | |
| 21:00:13 | mriedem | is this a new failure in building docs locally? | |
| 21:00:13 | mriedem | WARNING: RSVG converter command 'rsvg-convert' cannot be run. Check the rsvg_converter_bin setting | |
| 21:00:59 | KeithMnemonic1 | melwitt a plug for aspiers this is what I used to gauge the effort https://github.com/aspiers/git-deps | |
| 21:04:31 | KeithMnemonic1 | but that pulls in every dependency and i am not experienced enough to know if any could be skipped | |
| 21:05:43 | mriedem | ah i see https://github.com/openstack/nova/commit/16b9486bf7e91bfd5dc48297cee9f54b49156c93 | |
| 21:15:19 | mriedem | we need librsvg2-bin in bindep.txt | |
| 21:19:33 | mriedem | KeithMnemonic1: do you have a paste of the dependencies? i wouldn't be surprised if a bunch of the conflicts are due to like mox removal patches | |
| 21:28:11 | mriedem | melwitt: if you have a fedora or centos system available can you confirm that librsvg2-tools provides the rsvg-convert command? | |
| 21:28:53 | mriedem | which is i think just: yum provides rsvg-convert | |
| 21:28:53 | mriedem | right? | |
| 21:29:13 | melwitt | let me check | |
| 21:31:23 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Re-propose cross-cell-resize spec for Ussuri https://review.opendev.org/683002 | |
| 21:33:15 | melwitt | mriedem: you are correct, comes from librsvg2-tools | |
| 21:33:39 | mriedem | cool, thanks. that's what cinder had in their bindep but i've only got ubuntu and that's all that zuul uses for the pdf jobs | |
| 21:33:43 | melwitt | looked on centos7.3 | |
| 21:34:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add librsvg2* to bindep https://review.opendev.org/683003 | |
| 21:34:55 | mriedem | looks like we just had a zuul restart | |
| 21:40:18 | mriedem | and it's busted, "Unable to freeze job graph: 'dict_keys' object does not support indexing" | |
| 21:40:28 | mriedem | on that note, looks like i'm out of there for the day | |
| 21:49:00 | mriedem_afk | KeithMnemonic1: quickly glancing at that pike backport from queens, there are a bunch of conflicts but the big ones look like they are due to the multiattach volume support added in queens but those aren't functional dependencies, so i don't think those would actually impact backporting the fix to pike except it makes the backport harder | |
| 22:34:46 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.opendev.org/576020 | |
| 22:34:54 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.opendev.org/576020 | |
| 22:36:19 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.opendev.org/576027 | |
| 22:36:30 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.opendev.org/576027 | |
| 22:39:45 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.opendev.org/576031 | |
| 22:39:58 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.opendev.org/576031 | |
| 22:41:13 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.opendev.org/576299 | |
| 22:41:19 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (16) https://review.opendev.org/576299 | |
| 22:42:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.opendev.org/576344 | |
| 22:42:18 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (17) https://review.opendev.org/576344 | |
| 22:43:30 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.opendev.org/576673 | |
| 22:43:37 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (18) https://review.opendev.org/576673 | |
| 22:44:50 | openstackgerrit | Eric Fried proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | |
| 22:45:23 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676 | |
| 22:45:31 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676 | |
| 22:46:09 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.opendev.org/576689 | |
| 22:46:18 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.opendev.org/576689 | |
| 22:47:39 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.opendev.org/576709 | |
| 22:47:45 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.opendev.org/576709 | |
| 22:48:18 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.opendev.org/576712 | |
| 22:48:30 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.opendev.org/576712 | |
| 22:49:16 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add TODO note for mox removal https://review.opendev.org/576758 | |
| 22:49:22 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add TODO note for mox removal https://review.opendev.org/576758 | |
| 23:08:29 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Add a check for --config-drive option on nova boot https://review.opendev.org/653683 | |
| 23:30:47 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: WIP: Avoid redundant initialize_connection on source post live migration https://review.opendev.org/683008 | |
| 23:31:24 | mriedem | KeithMnemonic1: ^ is the pike backport with runtime code conflicts handled, test module conflicts are not handled (yet) but the commit message calls out what caused the conflicts so it's just a matter of resolving those test module conflicts | |
| 23:34:29 | sean-k-mooney | can we actully spligt the cpu resouces series form the vpmem seriese at this point | |
| 23:34:56 | sean-k-mooney | it was intended to prevent merge conflict but i think it has ended up hurting more then it helped | |
| 23:35:39 | sean-k-mooney | although at this point i guess the vpmem serice has merged? | |
| 23:36:08 | sean-k-mooney | looks like we are not on the first useful patch of the pcpu series | |
| 23:36:35 | alex_xu | sean-k-mooney: we already split | |
| 23:37:00 | sean-k-mooney | today? | |
| 23:37:38 | sean-k-mooney | what was the failure out of interest | |
| 23:38:14 | alex_xu | yesterday | |
| 23:39:54 | alex_xu | I only saw two bug, one is test_model_sync, another one is db timeout | |
| 23:40:40 | sean-k-mooney | sorry i though you were refering to https://review.opendev.org/#/c/672693/ | |
| 23:41:07 | sean-k-mooney | its going to fail i nthe gate beacue devstack failed on the second node in the live migration job | |
| 23:42:38 | sean-k-mooney | looks like the nova compute service db entry for the subnode never got created properly | |
| 23:42:49 | alex_xu | yea, I thought those two in vpmems, just checking what fail in cpu resource | |
| 23:43:13 | efried_pto | mriedem: I'll try to be around for the meeting tomorrow; but if I'm not, you wanna run it? | |
| 23:43:32 | mriedem | sure | |
| 23:43:57 | mriedem | alex_xu: zuul had 2 restarts today | |
| 23:44:00 | mriedem | so that reset everything | |
| 23:44:05 | efried_pto | alex_xu: totally bug 1823251 again. | |