| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-23 | |||
| 17:55:45 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Try to repro bug 1823251 with mysql logs https://review.opendev.org/678051 | |
| 17:56:43 | artom | sean-k-mooney, it's not actually creating a MoveClaim object though, is it? | |
| 17:57:33 | artom | sean-k-mooney, I'm thinking of a scenario where we live-migration a NUMA instance with neutron SRIOV ports | |
| 17:57:50 | sean-k-mooney | no its not as far as i rememeber | |
| 17:57:53 | artom | Will the MoveClaim for the NUMA part conflict with the PCI claim for the SRIOV part | |
| 17:58:23 | sean-k-mooney | no we claim them in the rt and store the pci addresses in the live migration data object | |
| 17:58:31 | sean-k-mooney | i dont think we are using move claims at all | |
| 17:59:46 | sean-k-mooney | we might use move claims for cold migration i have not check that in a long time | |
| 17:59:52 | mriedem | efried: i think it's failing because the dest doesn't exist when your task runs | |
| 18:00:03 | mriedem | that's done later here https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_51/678051/3/check/nova-tox-collect-py27/f64b3ba/ara-report/file/5ba0f501-9e58-4f71-859a-ef084a19e0be/#line-11 | |
| 18:00:33 | artom | sean-k-mooney, if you have the hardware to test those 2 together it would put many minds at east, I think | |
| 18:00:33 | efried | mriedem: infra said it was because sudo is disabled for tox by default. Latest respin (above) reparents to openstack-tox-with-sudo. | |
| 18:00:58 | sean-k-mooney | i plan to do that yes | |
| 18:01:20 | sean-k-mooney | ill do it monday or tuesday if you like | |
| 18:01:34 | efried | mriedem: but maybe that too. Though I would have thought they would set up log dirs before invoking posts. | |
| 18:02:50 | sean-k-mooney | i have 1 of my sriov systems setup with 18.04 so i need to install 18.04 on the second one an then install centos on a third host to test something else | |
| 18:03:19 | sean-k-mooney | ill kick of those now and i should be able to test it monday if you bug me or tuesday if you dont | |
| 18:05:39 | artom | sean-k-mooney, appreciated :) | |
| 18:06:30 | mriedem | efried: maybe it is https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_51/678051/3/check/nova-tox-collect-py27/f64b3ba/ara-report/file/c8a99499-4049-4878-b0a9-13a9292437f4/#line-1 | |
| 18:06:48 | mriedem | efried: so maybe your dest needs to be '{{ zuul_log_path }}' | |
| 18:06:53 | mriedem | er + /mysql/ | |
| 18:07:24 | mriedem | or {{ zuul_output_dir }}/logs/mysql | |
| 18:07:28 | mriedem | https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_51/678051/3/check/nova-tox-collect-py27/f64b3ba/ara-report/file/f31abc72-92ae-4e7d-b37c-8c124f7f32bd/#line-1 | |
| 18:09:55 | efried | mriedem: okay, can try something along those lines if this one borks again. | |
| 18:10:25 | mriedem | ok | |
| 18:10:31 | mriedem | you know about the ara report right? | |
| 18:10:39 | mriedem | to see what runs before and after you with what values | |
| 18:13:52 | sean-k-mooney | artom: once i have it up an running i can give you remote acess to poke around if you want. i have a static ip so i can easily port forwad you to i the hosts. ubuntu is install now so i just need to add a couple of kernel arges to make sriov work then stack with your code | |
| 18:14:07 | artom | sean-k-mooney, ack, that works | |
| 18:15:20 | artom | sean-k-mooney, I'm thinking of this bit in the resource tracker: https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L283-L295 | |
| 18:15:35 | artom | For NUMA LM, we call that during check_live_migration_at_destination | |
| 18:16:47 | efried | mriedem: no, I knew it was there, but never really understood what it meant. | |
| 18:18:55 | mriedem | btw, speaking of that nova.image.download module https://logs.opendev.org/84/676684/1/check/neutron-grenade-multinode/b0a76ac/logs/screen-n-api.txt#_Aug_15_13_25_04_744293 | |
| 18:18:59 | mriedem | i'm not sure why that's happening | |
| 18:19:05 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L294-L295 might be a proablem | |
| 18:19:31 | sean-k-mooney | since we have already claimed them on the dest it could claim them twice | |
| 18:19:50 | mriedem | oh gdi of course https://review.opendev.org/#/c/676684/ | |
| 18:19:51 | mriedem | hehh | |
| 18:19:56 | sean-k-mooney | i think currenlty we only use that for cold migration right | |
| 18:21:30 | artom | sean-k-mooney, *currently* | |
| 18:21:45 | artom | But NUMA LM is based on doing a MoveClaim | |
| 18:22:09 | sean-k-mooney | right so we would have to modify this or it wont work with sriov live migration | |
| 18:22:18 | sean-k-mooney | or rather might not | |
| 18:23:18 | sean-k-mooney | will you be passing in somthing to indicate its a live migration | |
| 18:23:20 | artom | Or just ignore pci_requests entirely if it's a live migration? | |
| 18:23:31 | sean-k-mooney | that is what i was going to suggest | |
| 18:23:33 | artom | Yeah, the migration will hava a type set | |
| 18:23:34 | openstackgerrit | Andreas Jaeger proposed openstack/nova master: PDF documentation build https://review.opendev.org/676730 | |
| 18:23:54 | sean-k-mooney | its nice that it is claiming alias based pci devices | |
| 18:24:02 | sean-k-mooney | we dont currently support that in the sriov migration | |
| 18:24:13 | sean-k-mooney | but that also means we woudl not update the xml | |
| 18:24:43 | artom | Yeah, we can't claim resource we have no intention of using | |
| 18:24:44 | sean-k-mooney | so what we need to to is 1 check that there is not alias based pci device and two not call the cpi claim bit if its a live migration | |
| 18:25:06 | artom | Why is 2 alone not enough? | |
| 18:25:30 | sean-k-mooney | it is but the other is a safty mesure | |
| 18:25:48 | sean-k-mooney | e.g. we shoudl not attemt to migrate if you have alias based pci passthough | |
| 18:25:54 | sean-k-mooney | its not supported at all | |
| 18:26:08 | sean-k-mooney | we might already check that somewhere | |
| 18:26:16 | sean-k-mooney | if we dont we should in the conductor | |
| 18:26:35 | artom | Seems like that should be separate from NUMA LM tho | |
| 18:26:44 | sean-k-mooney | ya it is. | |
| 18:27:10 | artom | NUMA LM just cares about not stepping on SRIOV LM's toes | |
| 18:27:15 | sean-k-mooney | so for numa the if(migration.type!=live) shoudl be enough | |
| 18:28:12 | artom | We have functional tests for SRIOV LM? | |
| 18:28:20 | artom | I wonder if we could try both in a func test | |
| 18:28:41 | sean-k-mooney | we dont | |
| 18:29:06 | artom | Func tests for neutron SRIOV at all? | |
| 18:29:27 | sean-k-mooney | nope | |
| 18:29:38 | sean-k-mooney | we have for flavor based pci passthough | |
| 18:29:43 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/tests/functional/libvirt/test_pci_sriov_servers.py | |
| 18:29:52 | sean-k-mooney | but no fucntional test for neutorn sriov | |
| 18:30:03 | artom | So manual testing it is | |
| 18:30:07 | sean-k-mooney | so we would have to extend the neutron mock | |
| 18:30:12 | artom | Yeah | |
| 18:30:15 | sean-k-mooney | but we could add them if we did | |
| 18:30:57 | sean-k-mooney | i dont think we have any livemigration functional tests that use the libvirt fake driver | |
| 18:31:09 | artom | We didn't | |
| 18:31:16 | artom | NUMA LM's going to be the first | |
| 18:31:37 | artom | You meant fakelibvirt, right? | |
| 18:31:42 | sean-k-mooney | so your going to implement the multiple port bidnign workflow in the neutron fixtures | |
| 18:31:46 | sean-k-mooney | yes | |
| 18:31:47 | artom | Because we have a couple that use fake virt | |
| 18:31:49 | sean-k-mooney | fakelibvirt | |
| 18:32:03 | artom | sean-k-mooney, Ha. Ha. Ha. | |
| 18:32:26 | sean-k-mooney | artom: you will need that form live migration testing i think | |
| 18:32:37 | sean-k-mooney | you would for sriov lm | |
| 18:32:50 | artom | sean-k-mooney, heh, one thing at a time :) | |
| 18:32:55 | sean-k-mooney | for numa you could fall back to the old codepath | |
| 18:33:06 | artom | If we decide neutron sriov func test is something we want, I don't mind doing it | |
| 18:33:19 | artom | But for NUMA LM it's irrelevant | |
| 18:36:14 | sean-k-mooney | it would be nice to have functional test for this at some point but i think we shoudl wait till U to do that | |
| 18:36:28 | sean-k-mooney | at least that we we can do some firstpart testing beyond unit tests | |
| 18:37:24 | mriedem | sean-k-mooney: https://review.opendev.org/#/c/678098/2 needs to mention the reason for the difference in the backport (in the commit message) | |
| 18:37:27 | mriedem | otherwise it looks OK | |
| 18:37:47 | sean-k-mooney | ok ill respin it now | |
| 18:38:51 | sean-k-mooney | it was this https://github.com/openstack/os-vif/commit/ee124d2e98c2660ced7d71c1bd10525bf095a939 | |
| 18:38:58 | mriedem | Id8b71172fb06b435cf169a7e55c11233f22fa65b | |
| 18:38:58 | mriedem | yeah i just ofund that | |
| 18:44:02 | openstackgerrit | sean mooney proposed openstack/os-vif stable/stein: only disable mac ageing for ovs hybrid plug https://review.opendev.org/678098 | |
| 18:44:10 | sean-k-mooney | something like ^ | |