| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-08-18 | |||
| 11:07:54 | lyarwood | melwitt / johnthetubaguy / dansmith ; https://review.opendev.org/#/c/699291/ - thoughts on finally landing this from mriedem? | |
| 11:08:50 | jsuchome | lyarwood: hi, me again ... any chance for that job testing direct-rbd-download? | |
| 11:09:54 | lyarwood | jsuchome: yup still on my list | |
| 11:18:33 | admin0 | so 2 observations .. if i use a qcow image, they do not boot at all .. if i change to raw images, they show active in the horizon, and i can see the console, but they are stuck on boot "booting from hard drive .. " | |
| 11:20:15 | lyarwood | admin0: and these are cirros images? | |
| 11:20:20 | admin0 | yes | |
| 11:20:33 | admin0 | one is bionic, but the output is kind of similar | |
| 11:20:37 | admin0 | they are stuck on boot | |
| 11:21:06 | lyarwood | admin0: that smells like there's something really wrong with your ceph env then | |
| 11:22:17 | lyarwood | jsuchome: I think I'm going to try to test your change locally first instead of insisting on a job to exercise it | |
| 11:22:26 | lyarwood | jsuchome: so I don't hold this up any longer | |
| 11:23:39 | jsuchome | cool | |
| 11:25:38 | lyarwood | that I expect to fail tbh | |
| 11:25:53 | lyarwood | as the rbd devstack plugin is held togther with tape and chewing gum | |
| 11:28:33 | openstackgerrit | Lee Yarwood proposed openstack/nova master: [goal] Prepare for job migration to Ubuntu Focal (20.04) https://review.opendev.org/738126 | |
| 12:01:27 | mnaser | lyarwood: we actually use it quite often for CI and it's been running quite OK for us | |
| 12:01:36 | mnaser | happy to help clean it up if need be | |
| 12:03:09 | lyarwood | mnaser: yeah apologies I didn't mean anything by that just that it has failed with a few non-standard deployment configs I've given it in the past :) | |
| 12:03:21 | lyarwood | this however worked AFAICT | |
| 12:04:03 | mnaser | Oh yeah no i agree that is is held together by tape though | |
| 12:04:13 | mnaser | I think it deploys Ceph 12.x | |
| 12:04:25 | mnaser | 3 major releases behind | |
| 12:05:03 | mnaser | Which isn’t ideal but eh, I don’t think anyone would like it if i made it implemented the latest release using cephadm which requires docker :p | |
| 12:05:47 | lyarwood | there appears to be some logic for that already in the plugin iirc | |
| 12:06:01 | lyarwood | but not for the latest and greatest release | |
| 12:06:05 | lyarwood | just the container part | |
| 13:06:05 | stephenfin | sean-k-mooney: Have you any idea what sets this field during live migration? https://github.com/openstack/nova/blob/master/nova/objects/migrate_data.py#L41 | |
| 13:09:07 | kashyap | He should definitely know ... in the not-too-distant past I've seen Sean talk about VIF types. | |
| 13:11:13 | sean-k-mooney | oh hi one sec | |
| 13:11:28 | sean-k-mooney | stephenfin: yes i responed in the code review | |
| 13:11:46 | stephenfin | code review? | |
| 13:11:47 | sean-k-mooney | its set in the conducrot live migration task | |
| 13:12:04 | sean-k-mooney | there is a patch up related to a bug | |
| 13:12:20 | sean-k-mooney | i assume you were reviewing https://review.opendev.org/#/c/742180/ | |
| 13:12:33 | sean-k-mooney | for https://bugs.launchpad.net/nova/+bug/1888395 | |
| 13:12:33 | openstack | Launchpad bug 1888395 in OpenStack Compute (nova) "shared live migration of a vm with a vif is broken in train" [Undecided,In progress] - Assigned to Kirill Egorov (kegorov-progmaticlab) | |
| 13:12:43 | sean-k-mooney | the current patch makes incorrect assumtions | |
| 13:12:51 | stephenfin | ah, no, I'm trying to add functional tests for SR-IOV live migration | |
| 13:12:57 | sean-k-mooney | ah ok | |
| 13:13:15 | sean-k-mooney | stephenfin: https://github.com/openstack/nova/blob/d4c857dfcb1ccfa5410de55671e69c722bbc990e/nova/conductor/tasks/live_migrate.py#L377-L383 | |
| 13:13:49 | sean-k-mooney | we create new vifs that have the old vif object inside them then we populate the vif_type form the dest port binding | |
| 13:14:04 | sean-k-mooney | stephenfin: so its set here https://github.com/openstack/nova/blob/d4c857dfcb1ccfa5410de55671e69c722bbc990e/nova/conductor/tasks/live_migrate.py#L433-L436 | |
| 13:14:27 | sean-k-mooney | but since that uses setattr you would find it with grep | |
| 13:14:31 | stephenfin | setattr | |
| 13:14:35 | stephenfin | gfdi | |
| 13:15:02 | stephenfin | sean-k-mooney++ Thanks. Try as I may, I couldn't find that /o\ | |
| 13:15:03 | sean-k-mooney | three are only 4 fileds you could replace that with 4 lines | |
| 13:15:30 | stephenfin | I think I might. setattr kills discoverability | |
| 13:16:24 | sean-k-mooney | if there were more filed i might argue for settattr but its just vnic_type, vif_type, profile and details | |
| 13:17:04 | sean-k-mooney | thats not going to change without a neutron api change and a new microversion which will require a nova code change to use anyway | |
| 13:20:11 | sean-k-mooney | stephenfin: if your going to refactor it might be nice to drop this too https://github.com/openstack/nova/blob/d4c857dfcb1ccfa5410de55671e69c722bbc990e/nova/conductor/tasks/live_migrate.py#L372-L379 | |
| 13:20:25 | stephenfin | agreed | |
| 13:26:18 | sean-k-mooney | lyarwood: sorry you pinged me eairlier but i didnt have irc up on my monitor this morning. did you figure it out? | |
| 13:26:53 | sean-k-mooney | oh im not connected downstream either i must not have connect to the vpn | |
| 13:41:28 | sean-k-mooney | stephenfin: https://review.opendev.org/#/c/728037/6 is in merge conflict so ill adress your nit and ping you to re review once done. | |
| 13:42:07 | sean-k-mooney | if im rebasing i might as well fix them now | |
| 13:44:24 | stephenfin | sean-k-mooney: cool | |
| 13:45:19 | stephenfin | sean-k-mooney: another question for you. The NeutronFixture isn't returning a vif_type key in the response to 'POST /v2.0/ports/{port_id}/bindings' | |
| 13:45:48 | stephenfin | Is that neutron's responsibility? Feel free to tell me to RTFM (I don't have an SR-IOV deployment to hand to experiment :() | |
| 13:46:20 | sean-k-mooney | yes the vif type can technicaly change on a migration but the vif type is set by the ml2 driver when it binds the port | |
| 13:46:37 | sean-k-mooney | stephenfin: can you show me the line | |
| 13:47:00 | stephenfin | This piece you linked earlier https://github.com/openstack/nova/blob/d4c857dfcb1ccfa5410de55671e69c722bbc990e/nova/conductor/tasks/live_migrate.py#L433-L436 | |
| 13:47:13 | sean-k-mooney | i ment the line in the fixture | |
| 13:47:22 | sean-k-mooney | i wanted to see what you have acess too | |
| 13:47:28 | stephenfin | Ohh | |
| 13:47:42 | stephenfin | Sec while I figure that out | |
| 13:47:55 | sean-k-mooney | you could copy it form the soruce vif | |
| 13:48:01 | sean-k-mooney | in the migrate vif objects | |
| 13:48:25 | sean-k-mooney | it can change but only if your migrating from like linux bridge hosts to ovs | |
| 13:48:47 | sean-k-mooney | for sriov it should not change although its techincially possible | |
| 13:48:52 | stephenfin | sean-k-mooney: https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L1659-L1676 | |
| 13:49:01 | stephenfin | That's what the fixture is doing. It's rather dumb | |
| 13:49:06 | sean-k-mooney | if you went from say a hardware offloaded ovs to sriov host | |
| 13:49:24 | stephenfin | Okay, I have the port_id so I can use the stored details | |
| 13:49:26 | sean-k-mooney | ya it does jus tthe minium it needed too | |
| 13:49:38 | stephenfin | with a big note saying that stub doesn't allow you to switch backends | |
| 13:49:47 | stephenfin | if that sounds reasonable | |
| 13:49:56 | sean-k-mooney | self._port_bindings[port_id][host] | |
| 13:50:22 | sean-k-mooney | so you you likely can just look it up assuming [host] also has another entry for the souce node | |
| 13:51:01 | sean-k-mooney | so loop over self._port_bindings[port_id] and fine the one where the key is not euqal to host | |
| 13:51:18 | sean-k-mooney | then grab the info from that | |
| 13:51:38 | stephenfin | Good call | |
| 13:53:22 | sean-k-mooney | that wont always work but it should work in the migration case | |
| 13:54:25 | sean-k-mooney | so you will need to guard it with an if and only do it if the vif_type is not in the binding data i guess | |
| 13:56:22 | sean-k-mooney | actully it might work because reasons. | |
| 13:56:40 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L1573 that is where the vif_type is currently coming form in the fixture | |
| 13:57:29 | sean-k-mooney | well i guess htere are other places too | |
| 13:57:43 | sean-k-mooney | like here https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L1357 | |
| 13:58:21 | sean-k-mooney | we build new ports form port 2 ^ https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L1836-L1837 | |
| 13:59:36 | sean-k-mooney | the current sriov_port does not have the vif_type set | |
| 13:59:39 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L1448-L1469 | |
| 14:02:02 | stephenfin | turns out we don't currently set that self._port_bindings field for the first binding | |
| 14:02:06 | stephenfin | big TODOs from gibi | |
| 14:03:27 | sean-k-mooney | actully looking at the neutron code the VIF_TYPE for sriov depned on the vnic_type | |
| 14:03:28 | sean-k-mooney | https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/mech_sriov/mech_driver/mech_driver.py#L81-L85 | |
| 14:03:51 | luyao | stephenfin: I split the vpmem enhencement patch to three and polish them with Alex's help, I'll appreciate that you review it again if you have time, thanks. https://review.opendev.org/#/c/714653/ | |
| 14:04:14 | stephenfin | luyao: Thanks for the reminder. I'll try get to that this evening or tomorrow :) | |
| 14:04:34 | luyao | stephenfin: Thank you in advance. :) | |
| 14:04:44 | sean-k-mooney | stephenfin: so its "hostdev_physical" for PFs and "hw_veb" for everyting else | |
| 14:05:15 | sean-k-mooney | VIF_TYPE_MACVTAP = 'macvtap' is not used for sriov its for the macvtap backend | |