Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-07-25
14:12:48 dansmith right
14:13:41 mriedem if you're going to do multiple tests, i would suggest trying to mix in the various moves, cold migrate, evacuate and unshelve
14:13:46 stephenfin Hmm, if I could attach an interface and try that, it would test so much more. As sean-k-mooney is learning though, there are holes all over the neutron/libvirt fakes
14:14:27 mriedem positive test could be the cold migrate where the hosts have the same config, full fail could be unshelve where the compute host was filled up while the server was shelved offloaded, and negative 2nd host different physnet for evacuate
14:14:33 mriedem that would cover those 3 move ops
14:14:56 mriedem i'm ok with dealing with subsequent port attachments in testing in a follow up
14:15:15 mriedem if you could get those 3 scenarios (1 pos, 2 neg, 3 different move ops) in a patch i think that's pretty solid for now
14:15:21 sean-k-mooney stephenfin: the most annoying thing about the fake dirver is how many thimes we convert back and fort between python objects and xml when using it
14:15:21 dansmith mriedem: we want a positive and negative case of each type of migration, IMHO
14:15:44 mriedem dansmith: ok, if that can get done today...
14:16:04 dansmith mriedem: well, positive and negative of at least one of the same migration ops, I think
14:16:17 dansmith otherwise I think we're cheating
14:17:27 mriedem that should be easy enough with cold migrate as i described above
14:17:44 mriedem pos: start 2 computes with same config to match the request, cold migrate between them
14:17:55 dansmith yup
14:17:57 mriedem neg: start compute 1, boot, change config, start compute 2, migrate, fail
14:18:05 dansmith yup
14:18:05 mriedem stephenfin: ^
14:18:55 stephenfin mriedem: What would I be changing it to to cause it to fail?
14:19:02 dansmith config
14:19:17 stephenfin *what would I be changing the config to
14:19:28 dansmith change the physnet from foo to bar
14:19:43 dansmith so that there are no nodes reporting physnet foo when you go to migrate
14:20:02 stephenfin That's no an issue though (by design)
14:20:25 dansmith oh right because the op can avoid reporting each physnet
14:20:30 dansmith then you have to do it via fill I guess
14:20:40 stephenfin yup, and to avoid breaking upgrades
14:22:33 sean-k-mooney stephenfin: change the numa mappings
14:23:10 dansmith oh,
14:23:22 dansmith stephenfin: can you report physnet bar on a numa node with no memory to make it fail easy?
14:23:57 dansmith so that when you go to request that physnet, but with one guest node of 1G, you don't fit without having to boot anything on it to fill it?
14:24:59 stephenfin That could work. Lemme try that
14:25:23 dansmith yeah then you could do your config switch trick by just moving the physnet to the stupid node
14:26:07 pooja_jadhav mriedem: h
14:26:13 pooja_jadhav mriedem: hi
14:27:12 mriedem hi
14:29:07 sean-k-mooney stephenfin: i have got to the point now where the failure is now in the assert fails. e.g. rebuild works ...
14:29:22 stephenfin sean-k-mooney: Fancy pushing it up?
14:30:35 sean-k-mooney want me to commit it as a seperate patch and we can quash them later or replace your existing patch
14:30:55 stephenfin sean-k-mooney: Nah, squash and stick in co-authored-by sure
14:35:26 openstackgerrit Merged openstack/nova stable/pike: Default embedded instance.flavor.disabled attribute https://review.openstack.org/580523
14:36:34 openstackgerrit sean mooney proposed openstack/nova master: WIP: Add functional tests for rebuild with NUMA networks https://review.openstack.org/585385
14:42:18 pooja_jadhav mriedem: I have done shared storage to test live migrate API, but still its trying shh the dest host.. not getting what action needs to be taken here?
14:45:14 pooja_jadhav mriedem: For logs - http://paste.openstack.org/show/726619/
14:45:38 pooja_jadhav mriedem: Can u please help me to resolve this issue?
14:54:05 mriedem bauzas: if you're looking for anything to review for FF, my portbinding live migration series has 3 changes left to complete the bp and the bottom patch has a +2 https://review.openstack.org/#/c/551371/
14:54:18 mriedem needless to say i'd like to see the end of this one https://review.openstack.org/#/q/topic:bp/neutron-new-port-binding-api+(status:open+OR+status:merged)
14:54:38 bauzas mriedem: yeah, okay
14:54:45 bauzas mriedem: thanks
14:54:57 bauzas mriedem: easier to review than the reshape API change :(
14:57:21 mriedem sean-k-mooney: mdbooth: so i'm assuming it's still a known problem if you hard reboot a guest, it's eth config for sriov ports attached could randomly change since we rebuild the guest xml, is that right? similar to how mountpoints for block devices can change on hard reboot, and why we have virtual device tags for that
14:57:52 mriedem reply when you can, but i've got to run out for a few hours
14:58:10 sean-k-mooney mriedem: ya we dont guarentee nic ordering because libvirt does not so it can change on a reboot
14:59:55 sean-k-mooney mriedem_away: if you need to know the ordering then ya you use tags and discover it from metadata. it only really is an issue if you reboot after attaching or detaching an interface.
15:00:19 mriedem_away yup that's what i thought, just making sure
15:01:02 stephenfin If I cold migrate, am I guaranteed to end up on a different host?
15:01:15 stephenfin assuming I don't specify a specific host to cold migrate to (if that's a thing)
15:01:17 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Extract base functional test case from test_direct https://review.openstack.org/585777
15:01:18 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Use base test in placement functional tests https://review.openstack.org/585778
15:01:23 sean-k-mooney stephenfin: i think so
15:02:24 sean-k-mooney stephenfin: i think all migrates will give that guarentee. resize i think could land on the same host but cold migrate should result in an actual migration
15:04:37 stephenfin sean-k-mooney: OK, my mental model was correct so. Good to hear :)
15:05:11 dansmith stephenfin: resize and migrate can land on the same host if so configured
15:05:26 dansmith we have a flag specifically for testing for that reason, but by default it should be disabled
15:05:43 pooja_jadhav mriedem: Hi, r u busy?
15:05:47 dansmith stephenfin: allow_resize_to_same_host
15:06:27 stephenfin dansmith: And that's unset so I'm golden
15:06:34 stephenfin Just making sure I'm not hardcoding some assumptions
15:06:39 stephenfin *false assumptions
15:06:39 dansmith aye
15:06:41 sean-k-mooney dansmith: i knew about allow_resize_to_same_host but i didnt know there was one for migrate
15:06:53 dansmith sean-k-mooney: migrate and resize are exactly the same operation
15:07:02 dansmith migrate just doesn't take a new flavor
15:07:21 dansmith they're two api entry points to the exact same code under the covers
15:07:44 sean-k-mooney dansmith: yes but i had assumed that that flag only effect resize because of its name. i guess i should have checked :)
15:07:51 dansmith nope
15:07:59 pooja_jadhav mriedem: If we are using shared storage then what should the value of block migration?
15:09:58 openstackgerrit xulei proposed openstack/nova stable/queens: Disable limits if force_hosts or force_nodes is set https://review.openstack.org/584616
15:10:55 sean-k-mooney pooja_jadhav: that depend on if all nodes can reach that shared storage and what shared storage you are using.
15:13:47 sean-k-mooney pooja_jadhav: that flag is intended for migration between host when you are using the image backend for nova without shared storage. but if you are using the image backend but have the images store on nfs then you would need to set it if you were migrating to a different nfs share and but i belive you should not set it if you are migrate to a host on the same nfs backend
15:13:57 pooja_jadhav sean-k-mooney: ohk, I am using nfs shared storage and both (source and dest host) are on same shared storage. if both hosts are on same storage then why its going for ssh to dest host?
15:14:27 sean-k-mooney pooja_jadhav: to copy the guest ram
15:15:59 pooja_jadhav sean-k-mooney: which flag, can u more elaborate please :)
15:17:18 sean-k-mooney block migrate. that said it also depend on how you have nfs set up. if each host has a different directory on the nfs share block migrate should be safe
15:17:41 sean-k-mooney if all hosts share /var/lib/libvirt then you dont want to block migrate
15:17:49 pooja_jadhav sean-k-mooney: cuurent passing this body in curl command - {"os-migrateLive": {"block_migration": "false", "host": "Neha-VirtualBox"}}
15:18:23 pooja_jadhav but this parameter "block_migration" is mandatory
15:19:15 sean-k-mooney pooja_jadhav: if you are using novas image backend and the guest is not booted from volume then yes it is
15:23:34 pooja_jadhav you mean to say the instance is booted from image which I am trying to migrate to dest host?
15:27:56 sean-k-mooney stephenfin: i think i know why the rebild is passing but i have not checked yet. we skip calling the schduler if the image does not change. so we would skip call you numa affinity code right?
15:28:14 stephenfin sean-k-mooney: Yes, I would
15:28:38 stephenfin I thought we were calling it with a new image though?
15:28:40 stephenfin Or maybe not
15:29:05 sean-k-mooney i dont know i was eating soup when i taught of it so havent checked.
15:29:13 sean-k-mooney ill be back in 5
15:29:26 stephenfin Ha!
15:29:28 stephenfin OK :)
15:31:42 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove usage of migrateToURI{2} APIs https://review.openstack.org/567258
15:31:55 sean-k-mooney ok back :) so ya post = {'rebuild': {'imageRef': server['image']['id']}} same image
15:32:37 kashyap cfriesen_: When you can, a fresh pair of eyes are appreciated; spent too much time fiddling with the above today while resolving the rebase conflict.

Earlier   Later