| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-23 | |||
| 16:48:50 | stephenfin | mriedem: How does one kick off tempest tests when they wouldn't normally run? https://review.opendev.org/#/c/677475/ | |
| 16:49:10 | stephenfin | (because irrelevant-files, I'm assuming) | |
| 16:49:42 | artom | dansmith, is that what you meant? Move the "stop ignoreing live migrations in the resource tracker" to its own small patch on top? | |
| 16:50:14 | dansmith | artom: yes, but, I need to re-read this claims stuff | |
| 16:50:24 | dansmith | maybe I'm making CLAIMS that are incorrect | |
| 16:50:30 | artom | dansmith, ack. Sounds like a good place for me to get lunch | |
| 16:50:58 | artom | dansmith, HI-OH | |
| 16:51:01 | openstackgerrit | Stephen Finucane proposed openstack/nova master: setup.cfg: Cleanup https://review.opendev.org/677969 | |
| 16:51:16 | artom | dansmith, want me to push the bottom fixes? | |
| 16:51:23 | dansmith | I think it's because all the regular claim code has been removed | |
| 16:51:24 | dansmith | artom: yes | |
| 16:52:45 | dansmith | I thought the only bits left of the claiming stuff don't actually really interact that much with the resource tracker, | |
| 16:53:01 | dansmith | and rather just see if the things being asked for exist on the compute.. meaning the pci device and the numa arrangement | |
| 16:53:17 | dansmith | so I'm not sure why the reporting stuff is really related directly | |
| 16:54:40 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects for NUMA live migration https://review.opendev.org/634827 | |
| 16:54:40 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Introduce live_migration_claim() https://review.opendev.org/635669 | |
| 16:54:41 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: add support for updating NUMA-related XML on the source https://review.opendev.org/635229 | |
| 16:54:41 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: add support for augmenting migrate_data with info from claims https://review.opendev.org/634828 | |
| 16:54:42 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration https://review.opendev.org/640021 | |
| 16:54:42 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: NUMA live migration support https://review.opendev.org/634606 | |
| 16:54:43 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Functional test for NUMA live migration https://review.opendev.org/672595 | |
| 16:54:47 | efried | dansmith: I should only have to restart n-cpu to pick up a change in oslo.service, yah? | |
| 16:54:53 | efried | at least, the change we care about | |
| 16:55:08 | dansmith | efried: if you're just testing that service then obviously :) | |
| 16:55:35 | efried | I didn't know if other services and their e.g. RPC objects would care. | |
| 16:56:05 | efried | dansmith: Then I've confirmed the oslo.service fix doesn't fix the problem. | |
| 16:56:31 | efried | (which doesn't mean it's wrong, just not complete) | |
| 16:57:17 | dansmith | artom: okay, I see now, I forgot how this works, but I see that if we don't remove that skip we won't do the actual checking of anything, so ignore me | |
| 16:57:20 | dansmith | efried: sweet | |
| 16:58:35 | artom | dansmith, yeah, it's coupling great? | |
| 16:58:40 | artom | *isn't | |
| 16:59:06 | artom | Right, pushed, bottom 4 should be good, now lunch | |
| 17:01:51 | ozzzo | It looks like nova is keeping mysql connections open until they time out. Is there a way to stop these error messages? | |
| 17:02:00 | ozzzo | Aborted connection 10726 to db: 'nova' user: 'nova' host: 'asdf' (Got timeout reading communication packets) | |
| 17:16:08 | mnaser | efried: if you have a few seconds, i wanted to know if you had thoughts about https://review.opendev.org/#/c/676540/ -- should i mark it no longer as deprecated as it's clearly being used? | |
| 17:17:46 | efried | mnaser: or propose whatever alternative we would prefer. mriedem ^ | |
| 17:18:39 | mnaser | i mean if it's clearly being consumed, then i don't mind making the change to stop marking is an deprecated.. | |
| 17:25:43 | openstackgerrit | Merged openstack/nova stable/rocky: Remove experimental job on openSUSE 42.3 https://review.opendev.org/677176 | |
| 17:25:50 | openstackgerrit | Merged openstack/nova stable/queens: Remove experimental job on openSUSE 42.3 https://review.opendev.org/677177 | |
| 17:25:57 | openstackgerrit | Merged openstack/nova stable/pike: Remove experimental job on openSUSE 42.3 https://review.opendev.org/677178 | |
| 17:28:37 | efried | mriedem, dansmith: Posted details on the SIGHUP patch https://review.opendev.org/#/c/641907/ | |
| 17:29:01 | efried | We are indeed breaking the privsep pipe (/me says as if I know what I'm talking about) | |
| 17:29:15 | efried | unclear where that would need to be fixed. | |
| 17:29:42 | mriedem | stephenfin: yeah irrelevant-files | |
| 17:36:47 | sean-k-mooney | stephenfin: add a followup [DNM] patch that add whitespace to a file that would trigger tempest or temporally remove one fo the fiels from irrelevent-files. i generall prefer to have a DNM testing patch as i can just run the job i care about. | |
| 17:37:39 | mriedem | efried: mnaser: i think we likely should undeprecate the direct file download stuff since clearly there are users and those users have made attempts to upstream an alternative solution which hasn't been approved, but not wholesale rejected either from what i understand, so it's probably not good to just rip it out from them when we're not providing an alternative | |
| 17:38:16 | efried | wfm | |
| 17:38:32 | sean-k-mooney | stephenfin: that chante will alter they way devstack install nova as nova installs all packages in requirements.txt and test-requiremnts.txt system wide by default so it should pass in the gate | |
| 17:38:59 | sean-k-mooney | stephenfin: however if you enable installing in virtual envs it may not work anymore | |
| 17:39:53 | sean-k-mooney | stephenfin: without a change to devstack nova would be relying on other pojects to install pymsql or psycopg2 for postgress | |
| 17:43:14 | efried | mriedem: any advice on next steps for copying mysql logs? https://review.opendev.org/#/c/678051/ | |
| 17:45:54 | efried | rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.1] | |
| 17:45:54 | efried | rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] | |
| 17:45:54 | efried | sudo: no tty present and no askpass program specified | |
| 17:45:54 | efried | Warning: Permanently added '166.78.116.9' (ECDSA) to the list of known hosts. | |
| 17:45:54 | efried | mriedem: | |
| 17:47:12 | sean-k-mooney | i dont think you want to copy them to the executer liek that i think you want to drop them into the working directory on the vm and have the normal log copy task copy them at the end | |
| 17:48:14 | sean-k-mooney | i may be wrong about that but i think that is what i have seen done before | |
| 17:49:53 | artom | sean-k-mooney, do you know if we would break anything if we started doing claims for all live migrations? I'm thinking of PCI stuff... | |
| 17:50:13 | artom | Presumably claims for PCI aren't broken in and of themselves, since they're used to boot instances and for cold migrations | |
| 17:50:31 | sean-k-mooney | we do claims for pci deivces for migration already | |
| 17:50:40 | artom | *live*? | |
| 17:50:43 | sean-k-mooney | yes | |
| 17:51:07 | sean-k-mooney | well for neutron sriov ports | |
| 17:51:17 | sean-k-mooney | we dont support live migration with other pci devices | |
| 17:51:30 | sean-k-mooney | but for sriov migration we added cliams | |
| 17:51:45 | sean-k-mooney | *sriov live migration | |
| 17:51:51 | artom | Are we talking about the same claims? | |
| 17:51:59 | sean-k-mooney | in the resouce tracker | |
| 17:52:03 | artom | Yeah | |
| 17:52:10 | sean-k-mooney | we do it as part of pre live migration at dest | |
| 17:52:11 | artom | Could you point me to the review/code? | |
| 17:52:22 | sean-k-mooney | lookign for it now | |
| 17:53:30 | sean-k-mooney | https://review.opendev.org/#/c/616120/ | |
| 17:53:41 | sean-k-mooney | and i think we use them in https://review.opendev.org/#/c/620115/28 | |
| 17:54:32 | sean-k-mooney | the pci manager works differently then the numa toplogy in the RT | |
| 17:54:59 | sean-k-mooney | but ehere is the claim in the compute manager | |
| 17:55:01 | sean-k-mooney | https://review.opendev.org/#/c/620115/28/nova/compute/manager.py | |
| 17:55:45 | openstack | bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed] https://launchpad.net/bugs/1823251 | |
| 17:55:45 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Try to repro bug 1823251 with mysql logs https://review.opendev.org/678051 | |
| 17:56:43 | artom | sean-k-mooney, it's not actually creating a MoveClaim object though, is it? | |
| 17:57:33 | artom | sean-k-mooney, I'm thinking of a scenario where we live-migration a NUMA instance with neutron SRIOV ports | |
| 17:57:50 | sean-k-mooney | no its not as far as i rememeber | |
| 17:57:53 | artom | Will the MoveClaim for the NUMA part conflict with the PCI claim for the SRIOV part | |
| 17:58:23 | sean-k-mooney | no we claim them in the rt and store the pci addresses in the live migration data object | |
| 17:58:31 | sean-k-mooney | i dont think we are using move claims at all | |
| 17:59:46 | sean-k-mooney | we might use move claims for cold migration i have not check that in a long time | |
| 17:59:52 | mriedem | efried: i think it's failing because the dest doesn't exist when your task runs | |
| 18:00:03 | mriedem | that's done later here https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_51/678051/3/check/nova-tox-collect-py27/f64b3ba/ara-report/file/5ba0f501-9e58-4f71-859a-ef084a19e0be/#line-11 | |
| 18:00:33 | artom | sean-k-mooney, if you have the hardware to test those 2 together it would put many minds at east, I think | |
| 18:00:33 | efried | mriedem: infra said it was because sudo is disabled for tox by default. Latest respin (above) reparents to openstack-tox-with-sudo. | |
| 18:00:58 | sean-k-mooney | i plan to do that yes | |
| 18:01:20 | sean-k-mooney | ill do it monday or tuesday if you like | |
| 18:01:34 | efried | mriedem: but maybe that too. Though I would have thought they would set up log dirs before invoking posts. | |
| 18:02:50 | sean-k-mooney | i have 1 of my sriov systems setup with 18.04 so i need to install 18.04 on the second one an then install centos on a third host to test something else | |
| 18:03:19 | sean-k-mooney | ill kick of those now and i should be able to test it monday if you bug me or tuesday if you dont | |
| 18:05:39 | artom | sean-k-mooney, appreciated :) | |
| 18:06:30 | mriedem | efried: maybe it is https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_51/678051/3/check/nova-tox-collect-py27/f64b3ba/ara-report/file/c8a99499-4049-4878-b0a9-13a9292437f4/#line-1 | |
| 18:06:48 | mriedem | efried: so maybe your dest needs to be '{{ zuul_log_path }}' | |
| 18:06:53 | mriedem | er + /mysql/ | |