| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-25 | |||
| 14:08:50 | bauzas | mriedem: btw. saw your req_spec changes | |
| 14:08:54 | bauzas | mriedem: thanks for this | |
| 14:09:00 | mriedem | and commenting out the ensure* line in nova/conductor/tasks/migrate.py would make the test fail | |
| 14:09:21 | mriedem | bauzas: np, i want all that compat code dead in stein | |
| 14:09:58 | bauzas | mriedem: I'll try to review those quickly | |
| 14:10:01 | mriedem | dansmith: is what i described for a cold migate negative test good enough for you for the move patch? | |
| 14:10:47 | dansmith | mriedem: combined with a positive test where we successfully migrate between two hosts that satisfy the requirement I guess | |
| 14:11:05 | bauzas | btw. what's super important to review for the 3st milestone ? | |
| 14:11:13 | bauzas | for the moment, looking at https://review.openstack.org/#/c/576927/19 | |
| 14:11:14 | stephenfin | mriedem: Wait, actually, it wouldn't complain if I simply removed the config. No config = do nothing special (to preserve backwards compatibility) | |
| 14:11:28 | dansmith | stephenfin: you'll have to change the config I think | |
| 14:11:55 | dansmith | stephenfin: also, probably worth doing the same negative test but where you fill up the other node such that it's not a candidate right? | |
| 14:12:13 | dansmith | like, boot enough things tied to the tunneled numa node that it can't take any more and make sure it properly rejects a new one | |
| 14:12:20 | stephenfin | The latter is probably a better test | |
| 14:12:35 | mriedem | stephenfin: good point, would have to change the config from physnet foo to physnet bar or something on the 2nd host | |
| 14:12:40 | stephenfin | though I'm going to have to rely on the unspecified "things schedule to host numa node 0" behavior for that to work | |
| 14:12:41 | dansmith | I think you probably want both | |
| 14:12:48 | dansmith | right | |
| 14:13:41 | mriedem | if you're going to do multiple tests, i would suggest trying to mix in the various moves, cold migrate, evacuate and unshelve | |
| 14:13:46 | stephenfin | Hmm, if I could attach an interface and try that, it would test so much more. As sean-k-mooney is learning though, there are holes all over the neutron/libvirt fakes | |
| 14:14:27 | mriedem | positive test could be the cold migrate where the hosts have the same config, full fail could be unshelve where the compute host was filled up while the server was shelved offloaded, and negative 2nd host different physnet for evacuate | |
| 14:14:33 | mriedem | that would cover those 3 move ops | |
| 14:14:56 | mriedem | i'm ok with dealing with subsequent port attachments in testing in a follow up | |
| 14:15:15 | mriedem | if you could get those 3 scenarios (1 pos, 2 neg, 3 different move ops) in a patch i think that's pretty solid for now | |
| 14:15:21 | dansmith | mriedem: we want a positive and negative case of each type of migration, IMHO | |
| 14:15:21 | sean-k-mooney | stephenfin: the most annoying thing about the fake dirver is how many thimes we convert back and fort between python objects and xml when using it | |
| 14:15:44 | mriedem | dansmith: ok, if that can get done today... | |
| 14:16:04 | dansmith | mriedem: well, positive and negative of at least one of the same migration ops, I think | |
| 14:16:17 | dansmith | otherwise I think we're cheating | |
| 14:17:27 | mriedem | that should be easy enough with cold migrate as i described above | |
| 14:17:44 | mriedem | pos: start 2 computes with same config to match the request, cold migrate between them | |
| 14:17:55 | dansmith | yup | |
| 14:17:57 | mriedem | neg: start compute 1, boot, change config, start compute 2, migrate, fail | |
| 14:18:05 | mriedem | stephenfin: ^ | |
| 14:18:05 | dansmith | yup | |
| 14:18:55 | stephenfin | mriedem: What would I be changing it to to cause it to fail? | |
| 14:19:02 | dansmith | config | |
| 14:19:17 | stephenfin | *what would I be changing the config to | |
| 14:19:28 | dansmith | change the physnet from foo to bar | |
| 14:19:43 | dansmith | so that there are no nodes reporting physnet foo when you go to migrate | |
| 14:20:02 | stephenfin | That's no an issue though (by design) | |
| 14:20:25 | dansmith | oh right because the op can avoid reporting each physnet | |
| 14:20:30 | dansmith | then you have to do it via fill I guess | |
| 14:20:40 | stephenfin | yup, and to avoid breaking upgrades | |
| 14:22:33 | sean-k-mooney | stephenfin: change the numa mappings | |
| 14:23:10 | dansmith | oh, | |
| 14:23:22 | dansmith | stephenfin: can you report physnet bar on a numa node with no memory to make it fail easy? | |
| 14:23:57 | dansmith | so that when you go to request that physnet, but with one guest node of 1G, you don't fit without having to boot anything on it to fill it? | |
| 14:24:59 | stephenfin | That could work. Lemme try that | |
| 14:25:23 | dansmith | yeah then you could do your config switch trick by just moving the physnet to the stupid node | |
| 14:26:07 | pooja_jadhav | mriedem: h | |
| 14:26:13 | pooja_jadhav | mriedem: hi | |
| 14:27:12 | mriedem | hi | |
| 14:29:07 | sean-k-mooney | stephenfin: i have got to the point now where the failure is now in the assert fails. e.g. rebuild works ... | |
| 14:29:22 | stephenfin | sean-k-mooney: Fancy pushing it up? | |
| 14:30:35 | sean-k-mooney | want me to commit it as a seperate patch and we can quash them later or replace your existing patch | |
| 14:30:55 | stephenfin | sean-k-mooney: Nah, squash and stick in co-authored-by sure | |
| 14:35:26 | openstackgerrit | Merged openstack/nova stable/pike: Default embedded instance.flavor.disabled attribute https://review.openstack.org/580523 | |
| 14:36:34 | openstackgerrit | sean mooney proposed openstack/nova master: WIP: Add functional tests for rebuild with NUMA networks https://review.openstack.org/585385 | |
| 14:42:18 | pooja_jadhav | mriedem: I have done shared storage to test live migrate API, but still its trying shh the dest host.. not getting what action needs to be taken here? | |
| 14:45:14 | pooja_jadhav | mriedem: For logs - http://paste.openstack.org/show/726619/ | |
| 14:45:38 | pooja_jadhav | mriedem: Can u please help me to resolve this issue? | |
| 14:54:05 | mriedem | bauzas: if you're looking for anything to review for FF, my portbinding live migration series has 3 changes left to complete the bp and the bottom patch has a +2 https://review.openstack.org/#/c/551371/ | |
| 14:54:18 | mriedem | needless to say i'd like to see the end of this one https://review.openstack.org/#/q/topic:bp/neutron-new-port-binding-api+(status:open+OR+status:merged) | |
| 14:54:38 | bauzas | mriedem: yeah, okay | |
| 14:54:45 | bauzas | mriedem: thanks | |
| 14:54:57 | bauzas | mriedem: easier to review than the reshape API change :( | |
| 14:57:21 | mriedem | sean-k-mooney: mdbooth: so i'm assuming it's still a known problem if you hard reboot a guest, it's eth config for sriov ports attached could randomly change since we rebuild the guest xml, is that right? similar to how mountpoints for block devices can change on hard reboot, and why we have virtual device tags for that | |
| 14:57:52 | mriedem | reply when you can, but i've got to run out for a few hours | |
| 14:58:10 | sean-k-mooney | mriedem: ya we dont guarentee nic ordering because libvirt does not so it can change on a reboot | |
| 14:59:55 | sean-k-mooney | mriedem_away: if you need to know the ordering then ya you use tags and discover it from metadata. it only really is an issue if you reboot after attaching or detaching an interface. | |
| 15:00:19 | mriedem_away | yup that's what i thought, just making sure | |
| 15:01:02 | stephenfin | If I cold migrate, am I guaranteed to end up on a different host? | |
| 15:01:15 | stephenfin | assuming I don't specify a specific host to cold migrate to (if that's a thing) | |
| 15:01:17 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Extract base functional test case from test_direct https://review.openstack.org/585777 | |
| 15:01:18 | openstackgerrit | Chris Dent proposed openstack/nova master: [placement] Use base test in placement functional tests https://review.openstack.org/585778 | |
| 15:01:23 | sean-k-mooney | stephenfin: i think so | |
| 15:02:24 | sean-k-mooney | stephenfin: i think all migrates will give that guarentee. resize i think could land on the same host but cold migrate should result in an actual migration | |
| 15:04:37 | stephenfin | sean-k-mooney: OK, my mental model was correct so. Good to hear :) | |
| 15:05:11 | dansmith | stephenfin: resize and migrate can land on the same host if so configured | |
| 15:05:26 | dansmith | we have a flag specifically for testing for that reason, but by default it should be disabled | |
| 15:05:43 | pooja_jadhav | mriedem: Hi, r u busy? | |
| 15:05:47 | dansmith | stephenfin: allow_resize_to_same_host | |
| 15:06:27 | stephenfin | dansmith: And that's unset so I'm golden | |
| 15:06:34 | stephenfin | Just making sure I'm not hardcoding some assumptions | |
| 15:06:39 | dansmith | aye | |
| 15:06:39 | stephenfin | *false assumptions | |
| 15:06:41 | sean-k-mooney | dansmith: i knew about allow_resize_to_same_host but i didnt know there was one for migrate | |
| 15:06:53 | dansmith | sean-k-mooney: migrate and resize are exactly the same operation | |
| 15:07:02 | dansmith | migrate just doesn't take a new flavor | |
| 15:07:21 | dansmith | they're two api entry points to the exact same code under the covers | |
| 15:07:44 | sean-k-mooney | dansmith: yes but i had assumed that that flag only effect resize because of its name. i guess i should have checked :) | |
| 15:07:51 | dansmith | nope | |
| 15:07:59 | pooja_jadhav | mriedem: If we are using shared storage then what should the value of block migration? | |
| 15:09:58 | openstackgerrit | xulei proposed openstack/nova stable/queens: Disable limits if force_hosts or force_nodes is set https://review.openstack.org/584616 | |
| 15:10:55 | sean-k-mooney | pooja_jadhav: that depend on if all nodes can reach that shared storage and what shared storage you are using. | |
| 15:13:47 | sean-k-mooney | pooja_jadhav: that flag is intended for migration between host when you are using the image backend for nova without shared storage. but if you are using the image backend but have the images store on nfs then you would need to set it if you were migrating to a different nfs share and but i belive you should not set it if you are migrate to a host on the same nfs backend | |
| 15:13:57 | pooja_jadhav | sean-k-mooney: ohk, I am using nfs shared storage and both (source and dest host) are on same shared storage. if both hosts are on same storage then why its going for ssh to dest host? | |
| 15:14:27 | sean-k-mooney | pooja_jadhav: to copy the guest ram | |
| 15:15:59 | pooja_jadhav | sean-k-mooney: which flag, can u more elaborate please :) | |