| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-29 | |||
| 19:00:03 | openstackgerrit | Gage Hugo proposed openstack/nova master: Update config doc policy file type https://review.opendev.org/673349 | |
| 19:14:46 | aspiers | sean-k-mooney: thanks | |
| 19:15:00 | aspiers | something in my devstack changed and now I get this when I try to delete instances: | |
| 19:15:03 | aspiers | ERROR oslo_messaging.rpc.server RemoteError: Remote error: IncompatibleObjectVersion Version 1.1 of ConsoleAuthToken is not supported, supported version is 1.0 | |
| 19:15:45 | aspiers | I know that's an OVO thing, but I have no idea how to fix it except for redeploying devstack which would take ages and hose a bunch of setup I want to keep - any ideas? | |
| 19:16:47 | aspiers | ah, I see the patch which added 1.1 recently - that explains why it happened, but not how I can migrate my objects | |
| 19:24:36 | aspiers | doh, I just had to restart *all* nova services | |
| 19:24:42 | aspiers | must have been a mismatch with n-cpu | |
| 19:35:03 | openstackgerrit | Eric Fried proposed openstack/nova-specs master: tox -e fast-specs https://review.opendev.org/673356 | |
| 19:35:10 | efried | sean-k-mooney, mriedem: as we discussed the other day ^ | |
| 19:36:58 | efried | stephenfin: that probably wants your sphinx expertise ^ | |
| 19:37:12 | openstackgerrit | Erik Olof Gunnar Andersson proposed openstack/nova master: Cleanup when hitting MaxRetriesExceeded from no host_available https://review.opendev.org/672855 | |
| 19:39:51 | openstack | bug 1837955 in OpenStack Compute (nova) "MaxRetriesExceeded sometime fails with messaging exception" [Medium,In progress] https://launchpad.net/bugs/1837955 - Assigned to Erik Olof Gunnar Andersson (eandersson) | |
| 19:39:51 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional regression test for bug 1837955 https://review.opendev.org/673357 | |
| 19:39:54 | mriedem | eandersson: efried: sean-k-mooney: ^ here is the functional test, i'll rebase the fix on top of it - was kind of a pain to make sure it would work on queens | |
| 19:42:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Cleanup when hitting MaxRetriesExceeded from no host_available https://review.opendev.org/672855 | |
| 19:42:58 | mriedem | rebased | |
| 20:08:00 | openstackgerrit | Eric Fried proposed openstack/nova master: Cleanup when hitting MaxRetriesExceeded from no host_available https://review.opendev.org/672855 | |
| 20:09:24 | eandersson | Thanks efried | |
| 20:09:40 | efried | thanks for the fix | |
| 20:09:58 | efried | now we just need someone to send mriedem's test | |
| 20:28:29 | sean-k-mooney | dumb question but we have a bunch of tempest test that end in TestJSON what makes them different form normal tempest tests. are then calling the api with json payloads or somthing? | |
| 20:43:01 | sean-k-mooney | ... i is confused | |
| 20:43:32 | sean-k-mooney | the tempest-fully-py3 job supports depends-on against neutron patches right | |
| 20:44:07 | sean-k-mooney | because my patch failed with an error message i deleted form the neutron code base so it really looks like it didnt work | |
| 20:45:41 | sean-k-mooney | yat http://logs.openstack.org/32/602432/12/check/tempest-full-py3/39b553b/job-output.txt.gz#_2019-07-23_17_51_24_263888 it looks like it didnt check out my patch | |
| 20:46:28 | sean-k-mooney | yep it ran with master ... http://logs.openstack.org/32/602432/12/check/tempest-full-py3/39b553b/job-output.txt.gz#_2019-07-23_17_54_02_352110 | |
| 21:01:33 | openstackgerrit | sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432 | |
| 21:16:38 | eandersson | btw is it just RamFilter that is being removed? Or is AggregateRamFilter being removed as well? | |
| 21:16:52 | eandersson | Because I see AggregateRamFilter is still a thing in Stein | |
| 21:17:46 | sean-k-mooney | both | |
| 21:18:13 | sean-k-mooney | and the core and disk filter both aggreagte and non aggrate forms | |
| 21:18:52 | sean-k-mooney | eandersson: all 6 filter have been depreacted since ocata | |
| 21:19:10 | sean-k-mooney | we removed the caching scudler i think in stien which was the last valid usecase for them | |
| 21:19:23 | eandersson | I see | |
| 21:20:19 | sean-k-mooney | we still actully have the filter on master but i dont think they work properly anymore | |
| 21:20:55 | eandersson | I did like how you could orchestrate aggregates | |
| 21:21:25 | sean-k-mooney | you can still do that in a sligtly different way via placmenet | |
| 21:21:47 | sean-k-mooney | but its perfromce before did nto scale well and there were a bunch of edgcases | |
| 21:22:04 | sean-k-mooney | it looks like https://github.com/openstack/nova/commit/78645e61c63bf042453d1f822ae8b3f1ee6a311b missed the aggrate versions | |
| 21:22:18 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects for NUMA live migration https://review.opendev.org/634827 | |
| 21:22:18 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP-until-series-is-ready] Introduce live_migration_claim() https://review.opendev.org/635669 | |
| 21:22:19 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: add support for augmenting migrate_data with info from claims https://review.opendev.org/634828 | |
| 21:22:20 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: NUMA live migration support https://review.opendev.org/634606 | |
| 21:22:20 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: RPC changes to prepare for NUMA live migration https://review.opendev.org/634605 | |
| 21:22:20 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: add support for updating NUMA-related XML on the source https://review.opendev.org/635229 | |
| 21:22:21 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Functional test for NUMA live migration https://review.opendev.org/672595 | |
| 21:22:21 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration https://review.opendev.org/640021 | |
| 21:23:10 | artom | Probably broke a whole bunch of unit tests with that, will let CI tell me what's up. | |
| 21:25:26 | artom | But the problems that were left at the end of Stein have been fixed | |
| 21:32:46 | sean-k-mooney | the main one was saving the updated xml right | |
| 21:33:17 | sean-k-mooney | when i tested it at the end of stien it more or less worked bar the quirk | |
| 21:34:47 | artom | The two problems were: 1. not saving the new instance_numa_topology 2. If the resource audit ran during the LM, it wouldn't pick up the migrating instance, and would remove its resources from usage | |
| 21:35:29 | sean-k-mooney | oh ok and they are fixed in the patches you just pushed | |
| 21:36:18 | sean-k-mooney | im going to redeploy my 2 node ovn setup to something more more useful | |
| 21:36:23 | artom | Yeah. But like I said, I probably broke other things, just by changing some method names and removing that PinMapping object | |
| 21:36:33 | sean-k-mooney | should i pull down your latest patches and give them a try? | |
| 21:36:45 | artom | Not yet :) | |
| 21:36:56 | artom | I'll email the ML when it's ready (should be this week) | |
| 21:37:17 | artom | I know Windriver (and Cisco too, apparently?) would like to test it out | |
| 21:37:19 | sean-k-mooney | ok i might try and test numa + sriov next week | |
| 21:37:51 | sean-k-mooney | i havent powered up my physical server since the sriov stuff merged but it would be nice to test both together | |
| 21:39:20 | sean-k-mooney | ideally if we can get your numa stuff merged soon ish i can then test stephens cpu pinning change too | |
| 21:41:17 | artom | His thing was surprisingly quick | |
| 21:41:25 | artom | Given the upgrade impact, I was expecting it to take longer | |
| 21:41:56 | sean-k-mooney | the PCUs in placement changes | |
| 21:42:13 | artom | Yeah | |
| 21:42:44 | sean-k-mooney | well the fact we dont have numa this cycle signifcatly redues the upgrade impact | |
| 21:42:59 | sean-k-mooney | but i als dont know if the reshap is fully working yet | |
| 21:43:20 | sean-k-mooney | i think it is but there was still one edgecase that stephen was looking into | |
| 21:43:55 | sean-k-mooney | not with the reshap but with the schduler handeling old instance or host topologies | |
| 21:51:12 | mriedem | sean-k-mooney: eandersson: the Aggregate filters are not deprecated | |
| 21:51:35 | mriedem | otherwise the docs would say so https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#compute-filters | |
| 21:52:15 | sean-k-mooney | mriedem: the aggreate filters dont work anymore because the limmits set in the aggreate metadata are ignored by placmenet | |
| 21:52:37 | sean-k-mooney | so you have to set the limits on all the compute nodes or viat the placement api instead | |
| 21:52:52 | mriedem | i realize, i wrote these docs https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#allocation-ratios | |
| 21:52:57 | eandersson | They kinda work | |
| 21:53:09 | eandersson | I mean they work for us because computes are misconfigured | |
| 21:53:45 | mriedem | i don't know why we didn't deprecate the aggregate filters when the others were deprecated, probably different timelines | |
| 21:54:46 | sean-k-mooney | when we found that bug and melwitt sent http://lists.openstack.org/pipermail/openstack-dev/2018-January/126283.html i thought that was use signeling the depreacation | |
| 21:54:58 | mriedem | sean-k-mooney: tempest TestJson is a remnant from when we had xml api testing | |
| 21:55:02 | sean-k-mooney | we also have that gian warning https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#scheduling-considerations | |
| 21:55:10 | sean-k-mooney | mriedem: ah ok | |
| 21:55:29 | mriedem | sean-k-mooney: well i guess no one bothered to follow up and actually deprecate those filters | |
| 21:55:30 | sean-k-mooney | i rembere there was an xml api and i never used it | |
| 21:55:45 | mriedem | i deprecated the core/ram/disk filters since those were clearly no longer useful outside of the caching scheduler | |
| 21:55:53 | sean-k-mooney | mriedem: ok am can we do it in train and kill them in U then? | |
| 21:56:14 | mriedem | i would ask in the ML b/c that is a bunch of context i don't have loaded into my head right now | |
| 21:56:30 | sean-k-mooney | right and as you noted ironic stoped reporting the cpus ram and disk so they activly were broken with ironic as of stien | |
| 21:56:47 | mriedem | noted in my commit message you mean | |
| 21:56:51 | sean-k-mooney | yes | |
| 21:57:16 | mriedem | eandersson: also fyi https://review.opendev.org/#/c/640898/ - an osc command to update allocation ratios on provider aggregates | |
| 21:57:39 | mriedem | it's been hung up in how to actually write the command using osc conventions | |
| 22:01:48 | sean-k-mooney | speaking of osc my patch for adding evacuate to osc https://review.opendev.org/#/c/643578/ | |
| 22:02:21 | sean-k-mooney | i havent looked at it in a while but ill try to get back to it soon ish | |
| 22:02:24 | mriedem | there are unanswered comments in there | |
| 22:02:42 | mriedem | mainly that dansmith wanted to continue calling it evacuate to avoid different names all over the place | |
| 22:02:43 | sean-k-mooney | i think ill mail the list and ask for feedback on the way forward vs nameing | |
| 22:02:51 | sean-k-mooney | ya | |
| 22:03:21 | sean-k-mooney | i am ok to do that i did care more earlier but i would prefer to land a command rather then none | |