| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-02 | |||
| 19:09:51 | sean-k-mooney | i have added nova to the existing bug and added the stein-rc-potential tag but if you would like a new bug filed i can do that too. | |
| 19:10:17 | sean-k-mooney | im going to have dinner but if you want any changes let me know | |
| 19:10:19 | efried | sean-k-mooney: assuming stx doesn't care about our stein-rc-potential tag, I think this should be fine. | |
| 19:10:21 | efried | Thanks sean-k-mooney | |
| 19:10:37 | sean-k-mooney | cfriesen: any comment on ^ | |
| 19:11:14 | cfriesen | looking | |
| 19:11:17 | sean-k-mooney | i think it should be ok but we should discuss it at the ptg too. e.g. how to tack cross project bugs like this between openstack and starlingx | |
| 19:16:38 | cfriesen | sean-k-mooney: looks reasonable, but why not put the result['parent_ifname'] assignment in the try block and get rid of the initial assignment to None and the conditional? | |
| 19:18:30 | openstackgerrit | Ghanshyam Mann proposed openstack/nova-specs master: Spec for API policy updates https://review.openstack.org/547850 | |
| 19:21:39 | cfriesen | efried: one of our people in charge thinks adding nova (and the tag) to that bug is a good solution for tracking joint issues | |
| 19:21:57 | cfriesen | (i.e. StarlingX, just to be clear) | |
| 19:21:57 | efried | wfm. mriedem may have a stronger opinion. | |
| 19:22:13 | efried | cfriesen: what's not clear to me is whether anything needs to be done on the stx side at all. | |
| 19:22:51 | cfriesen | on our end it'll likely just be picking up a new load once it's fixed in nova and validating the issue is gone | |
| 19:25:19 | sean-k-mooney | cfriesen: ya i think from a souce code point of view it will be a cherrypick or rebase | |
| 19:25:35 | sean-k-mooney | there is obviously the testing aspect too. | |
| 19:26:50 | cfriesen | sean-k-mooney: with your fix we're seeing logs every minute: nova.pci.utils [req-d9c8620e-7990-4b3c-a6b0-88a131852e47 - - - - -] No net device was found for VF 0000:3d:02.2: PciDeviceNotFoundById: PCI device 0000:3d:02.2 not found | |
| 19:27:13 | sean-k-mooney | that is from the other capablities fucntion | |
| 19:27:49 | sean-k-mooney | not adding a second message every minute was why i add the pass in teh except block in stead of logging | |
| 19:29:29 | cfriesen | might make sense to quiet that down, but that's not quite so urgent. | |
| 19:29:36 | sean-k-mooney | cfriesen: that comes form here https://github.com/openstack/nova/blob/master/nova/pci/utils.py#L224-L225 | |
| 19:29:45 | cfriesen | it's 48 logs every minute | |
| 19:30:13 | sean-k-mooney | cfriesen: ya that has been there for 2 or 3 releases | |
| 19:30:32 | sean-k-mooney | im going to work on a followup to fix some comments i notice i can remove that warning too. | |
| 19:30:39 | sean-k-mooney | or make it a debug message | |
| 19:30:58 | cfriesen | debug might be good | |
| 19:31:21 | sean-k-mooney | there is nothing that an operator can do to scilence it if they have non nic VF currently and its not helpful in that case | |
| 19:31:33 | cfriesen | agreed | |
| 19:32:20 | sean-k-mooney | anyway the fact that your nova-compute agent didnt die means the fix is at least minimally working. | |
| 19:35:05 | sean-k-mooney | i might look at optimising this code a bit too. currently we call _get_pcidev_info from _get_pci_passthrough_devices before applying the pci whitelist so we look at way more device then we need too | |
| 19:50:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix comment in test_attach_with_multiattach_fails_not_available https://review.openstack.org/649440 | |
| 19:52:35 | openstackgerrit | Merged openstack/nova master: Fix a deprecation warning https://review.openstack.org/649234 | |
| 19:54:45 | efried | Seems to hit several different jobs, but always grenade | |
| 19:54:45 | efried | mriedem: How about this one: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ModuleNotFoundError%3A%20No%20module%20named%20'memcache'%5C%22 | |
| 19:56:45 | efried | mriedem: https://review.openstack.org/#/c/649096/ ? | |
| 19:57:54 | mriedem | efried: known issue | |
| 19:58:02 | mriedem | yeah related to that bug | |
| 19:58:09 | mriedem | i need to update the e-r query, or you can | |
| 19:58:25 | mriedem | to include build_name:"neutron-grenade-multinode" OR build_name:"neutron-grenade-dvr-multinode" | |
| 19:58:34 | efried | mriedem: there's also grenade-py3 | |
| 19:58:46 | efried | let me try | |
| 19:58:55 | mriedem | the query is already restricted to just grenade-py3 | |
| 19:59:16 | mriedem | so need message:"..." AND tags:"screen" AND (build_name:"grenade-py3" OR build_name:"..."...) | |
| 19:59:19 | efried | k, hadn't pulled it up yet. | |
| 19:59:29 | openstack | Launchpad bug 1822873 in devstack "stack fails if NOVA_NUM_CELLS > 1 and n-novnc enabled" [Undecided,New] | |
| 19:59:29 | mriedem | melwitt: dansmith: duh duh duh https://bugs.launchpad.net/devstack/+bug/1822873 | |
| 19:59:32 | efried | the bug says the module not found is etcd | |
| 19:59:49 | mriedem | efried: it's a whole bunch of packages, | |
| 19:59:57 | mriedem | i spent half of yesterday digging into that | |
| 20:00:01 | dansmith | mriedem: duh duh duhntcare | |
| 20:00:02 | efried | okay, I see the message is just ModuleNotFound, cool. | |
| 20:00:05 | mriedem | but it's also just busted networks getting to pypi | |
| 20:00:36 | mriedem | efried: just need to update the existing query for http://status.openstack.org/elastic-recheck/#1820892 | |
| 20:00:42 | efried | mriedem: on it | |
| 20:02:08 | mnaser | mriedem: if you launch it in sjc1, you'd be launching it against stein :) | |
| 20:06:25 | efried | mriedem: Okay, so would there have been a better way for me to determine that this was 1820892 ? | |
| 20:17:04 | melwitt | mriedem: dammit novnc | |
| 20:17:37 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Docs: emulator threads: clarify expected behavior https://review.openstack.org/649416 | |
| 20:29:03 | eandersson | Does anyone recall the reason why the functionality to search based on metadata was removed from the api? | |
| 20:29:06 | eandersson | > search_options["metadata"] = '{"my_key" : "bla" }' | |
| 20:29:12 | eandersson | https://review.openstack.org/#/c/408571/ | |
| 20:29:18 | eandersson | Also, is the alternative to use tags? | |
| 20:43:31 | efried | eandersson: I have no idea, but did you look at the spec? http://specs.openstack.org/openstack/nova-specs/specs/ocata/implemented/add-whitelist-for-server-list-filter-sort-parameters.html | |
| 20:44:32 | efried | That appears to answer the first question. alex_xu could probably answer the second. | |
| 20:45:12 | mriedem | mnaser: i only see ca-ymq-2 | |
| 20:45:48 | mnaser | huh? sjc1 is a separate region | |
| 20:46:05 | mnaser | OS_REGION_NAME=sjc1 | |
| 20:46:51 | eandersson | I can't find any good documentation for tags, but maybe I just suck at googling :D | |
| 20:48:36 | eandersson | nova cli has things like > nova server-tag-add | |
| 20:48:57 | eandersson | but openstackcli has no mention of tags | |
| 20:49:00 | eandersson | > https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/server.html#server-create | |
| 20:49:10 | mriedem | mnaser: oh i thouht it was a zone | |
| 20:49:27 | mriedem | eandersson: openstack server set i think | |
| 20:49:35 | mriedem | https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/server.html#server-set | |
| 20:49:44 | mriedem | oh nvm | |
| 20:49:49 | mriedem | no tags support there either yet | |
| 20:49:55 | mriedem | consulting https://etherpad.openstack.org/p/compute-api-microversion-gap-in-osc | |
| 20:50:22 | mriedem | https://review.openstack.org/#/c/569386/ | |
| 20:50:51 | eandersson | That explains it | |
| 20:51:51 | mriedem | eandersson: if you're going to be in denver https://www.openstack.org/summit/denver-2019/summit-schedule/events/23665/closing-compute-api-feature-gaps-in-the-openstack-cli | |
| 20:52:02 | eandersson | I will | |
| 20:52:19 | mriedem | i've started the etherpad for that session https://etherpad.openstack.org/p/DEN-osc-compute-api-gaps | |
| 21:09:50 | NewBruce | hej mriedem - we look to have stumbled on a bug in Neutrons Port Binding API, in a particular area of the code which you will probably be more intimately familliar with than I | |
| 21:10:03 | NewBruce | https://bugs.launchpad.net/nova/+bug/1822884 (cc sean-k-mooney) | |
| 21:10:04 | openstack | Launchpad bug 1822884 in OpenStack Compute (nova) "live migration fails due to port binding duplicate key entry in post_live_migrate" [Undecided,New] | |
| 21:12:04 | openstackgerrit | melanie witt proposed openstack/nova stable/stein: Add doc on VGPU allocs and inventories for nrp https://review.openstack.org/649454 | |
| 21:12:51 | NewBruce | if you have a minute, id love your opinion - im trying to decide how much more debugging is worth before we move over to cold migrations | |
| 21:16:40 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Docs: emulator threads: clarify expected behavior https://review.openstack.org/649416 | |
| 21:17:51 | mriedem | NewBruce: i don't know what the differences are between RDO and OSA which would cause issues here | |
| 21:18:18 | mriedem | as you said, "This is unexpected, as even in the RDO to RDO case, both nodes are Rocky and so the new process should be in use." | |
| 21:18:31 | mriedem | are you sure you're not hitting a case where one node is really queens? | |
| 21:18:31 | NewBruce | hrmm, no - that seems to be our sticking point - i was chatting with sean-k-mooney and also mnaser on this | |
| 21:19:17 | NewBruce | yeah, absolutely sure - as I am using a single test source node which is absolutely Rocky as well as the target | |
| 21:21:17 | mriedem | is 'Port Bindings Extended' showing up in the neutron api extension list in both the rdo and osa case? | |
| 21:22:07 | mriedem | if rdo is rocky, i don't know why rdo->rdo would use the old flow | |
| 21:22:16 | mriedem | unless the 'Port Bindings Extended' neutron extension is not showing up | |
| 21:23:48 | NewBruce | i can confirm that on the target it is enabled | |
| 21:23:52 | mriedem | the live migration task should only do the new flow with the 2nd deactivated port binding if (1) the neutron 'Port Bindings Extended' API extension is available, and (2) both the source and dest compute service versions are >= 37 | |
| 21:23:58 | mriedem | *35 | |
| 21:24:27 | eandersson | mriedem, solution was to just list all VMs in the project and then just filter on metadata :p | |