Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-07-25
15:07:51 dansmith nope
15:07:59 pooja_jadhav mriedem: If we are using shared storage then what should the value of block migration?
15:09:58 openstackgerrit xulei proposed openstack/nova stable/queens: Disable limits if force_hosts or force_nodes is set https://review.openstack.org/584616
15:10:55 sean-k-mooney pooja_jadhav: that depend on if all nodes can reach that shared storage and what shared storage you are using.
15:13:47 sean-k-mooney pooja_jadhav: that flag is intended for migration between host when you are using the image backend for nova without shared storage. but if you are using the image backend but have the images store on nfs then you would need to set it if you were migrating to a different nfs share and but i belive you should not set it if you are migrate to a host on the same nfs backend
15:13:57 pooja_jadhav sean-k-mooney: ohk, I am using nfs shared storage and both (source and dest host) are on same shared storage. if both hosts are on same storage then why its going for ssh to dest host?
15:14:27 sean-k-mooney pooja_jadhav: to copy the guest ram
15:15:59 pooja_jadhav sean-k-mooney: which flag, can u more elaborate please :)
15:17:18 sean-k-mooney block migrate. that said it also depend on how you have nfs set up. if each host has a different directory on the nfs share block migrate should be safe
15:17:41 sean-k-mooney if all hosts share /var/lib/libvirt then you dont want to block migrate
15:17:49 pooja_jadhav sean-k-mooney: cuurent passing this body in curl command - {"os-migrateLive": {"block_migration": "false", "host": "Neha-VirtualBox"}}
15:18:23 pooja_jadhav but this parameter "block_migration" is mandatory
15:19:15 sean-k-mooney pooja_jadhav: if you are using novas image backend and the guest is not booted from volume then yes it is
15:23:34 pooja_jadhav you mean to say the instance is booted from image which I am trying to migrate to dest host?
15:27:56 sean-k-mooney stephenfin: i think i know why the rebild is passing but i have not checked yet. we skip calling the schduler if the image does not change. so we would skip call you numa affinity code right?
15:28:14 stephenfin sean-k-mooney: Yes, I would
15:28:38 stephenfin I thought we were calling it with a new image though?
15:28:40 stephenfin Or maybe not
15:29:05 sean-k-mooney i dont know i was eating soup when i taught of it so havent checked.
15:29:13 sean-k-mooney ill be back in 5
15:29:26 stephenfin Ha!
15:29:28 stephenfin OK :)
15:31:42 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove usage of migrateToURI{2} APIs https://review.openstack.org/567258
15:31:55 sean-k-mooney ok back :) so ya post = {'rebuild': {'imageRef': server['image']['id']}} same image
15:32:37 kashyap cfriesen_: When you can, a fresh pair of eyes are appreciated; spent too much time fiddling with the above today while resolving the rebase conflict.
15:34:25 stephenfin sean-k-mooney: I think I did that so I didn't have to do more mocking of the image service. I'm surprised the check is that "clever" though. I was expecting it to simply check if imageRef or something was set
15:36:50 sean-k-mooney stephenfin: it might not be the issue but its somthing im going to look into. its the best lead i have so far
15:57:51 melwitt sean-k-mooney: I proposed https://review.openstack.org/585530 for the noop plugin thing
15:58:56 melwitt stephenfin ^
15:59:26 sean-k-mooney melwitt: are you sure that is the issue. i was not seeing that issue in the functional test i created
15:59:44 melwitt pretty sure based on past commits that added the previous plugins
15:59:55 melwitt but I didn't know how to test it
16:00:06 sean-k-mooney did you see https://review.openstack.org/#/c/585693/
16:00:13 melwitt yeah, just opened it
16:00:29 sean-k-mooney i was not able to reproduce the error but i was able to load all the plugins correctly without the error
16:00:48 melwitt yeah. it's something to do with the packaging install I think
16:01:14 sean-k-mooney as in when you pip install?
16:01:27 melwitt yeah, is my guess
16:01:43 stephenfin melwitt, sean-k-mooney: probably this https://hynek.me/articles/testing-packaging/
16:01:46 melwitt when the setup.cfg is processed
16:02:55 stephenfin tl;dr: What melwitt said
16:03:37 sean-k-mooney stephenfin: ok how do we create a test for this to ensure we dont miss it in the future
16:03:55 sean-k-mooney stephenfin: or are you suggesting adding a src dir?
16:04:17 stephenfin sean-k-mooney: That I'm not sure about. No one else puts stuff inside a src dir so we can't really do that either
16:04:27 stephenfin sean-k-mooney: Assume it's a once off move on?
16:04:31 stephenfin *and move on
16:05:09 sean-k-mooney stephenfin: i guess we can add a test to one of the ci jobs.
16:06:07 sean-k-mooney ill add it to my list but let go with melwitt fix for now then.
16:06:25 melwitt we could make a dumb test that walks the entries in setup.cfg and raises if something is missing
16:06:56 melwitt I don't know of a smarter way to do it off the top of my head
16:07:46 sean-k-mooney melwitt: that is what i was thinking ya. specifically parsting the os_vif entrypoint and checking the plugin class path are rooted in a dir listed in packages
16:08:41 stephenfin You could do fancy things with pkg_resources too. I really don't think it's worth the effort though. If it bites us again, maybe then
16:10:03 sean-k-mooney i dont like leaveing gotchas like that lieing around. its not super high on my list but its now on my list
16:13:25 jangutter wow. that is doubleplusspam.
16:13:34 sean-k-mooney yeah spam
16:21:04 openstackgerrit Eric Fried proposed openstack/nova master: [placement] Add /reshaper handler for POST https://review.openstack.org/576927
16:21:05 openstackgerrit Eric Fried proposed openstack/nova master: reshaper: Look up provider if not in inventories https://review.openstack.org/585033
16:21:06 openstackgerrit Eric Fried proposed openstack/nova master: Make get_allocations_for_resource_provider sane https://review.openstack.org/584598
16:21:07 openstackgerrit Eric Fried proposed openstack/nova master: Report client: Real get_allocs_for_consumer https://review.openstack.org/584599
16:21:08 openstackgerrit Eric Fried proposed openstack/nova master: Report client: get_allocations_for_provider_tree https://review.openstack.org/584648
16:21:09 openstackgerrit Eric Fried proposed openstack/nova master: Report client: _reshape helper, placement min bump https://review.openstack.org/585034
16:21:10 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Report client: update_from_provider_tree w/reshape https://review.openstack.org/585049
16:21:11 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Compute: Handle reshaped provider trees https://review.openstack.org/576236
16:36:16 sean-k-mooney stephenfin: its passing the schduler because the network_metadata only contains info for the first instance on rebuild
16:36:43 stephenfin what do you mean?
16:37:25 sean-k-mooney stephenfin: on rebuild the network_metadata in the numa topology filter only list 1 of the 2 physnets requested
16:37:37 sean-k-mooney stephenfin: the one form the attached interface is missing
16:39:57 stephenfin sean-k-mooney: Yeah, I'm seeing something possibly related here. In my case, the info_cache.network_info is empty when I attempt to populate the request spec
16:40:11 stephenfin I'm trying to figure out what populates that in he first place in case we have that mocked out too
16:41:49 sean-k-mooney stephenfin: this https://review.openstack.org/#/c/585339/5/nova/objects/request_spec.py
16:42:59 sean-k-mooney i wonder if i need to call that again on attach.
16:43:56 sean-k-mooney there getting quicker at killing the spam
16:44:05 stephenfin sean-k-mooney: It should be called here https://review.openstack.org/#/c/585339/5/nova/conductor/manager.py
16:45:07 openstackgerrit Stephen Finucane proposed openstack/nova master: WIP: Add functional tests for cold migrate with NUMA networks https://review.openstack.org/585819
16:45:38 stephenfin sean-k-mooney: But in the test above, it seems instance.info_cache.network_info is unpopulated. I don't know why
16:45:41 sean-k-mooney ok ill see if that does the right thing. that code is being called
16:48:58 sean-k-mooney so ya the instance.info_cache.network_info only has 1 vif so it was not updated when we did the attach
16:49:37 stephenfin sean-k-mooney: Yup, likely the same thing I'm seeing so. I wonder what causes that to be refreshed?
16:50:07 stephenfin I guess we need to call ComputeManager._heal_instance_info_cache
16:51:10 sean-k-mooney its proably missing in the fake driver
16:55:41 sean-k-mooney stephenfin: looks like that is normally called by neutron when it updates the port
17:01:25 stephenfin sean-k-mooney: I can only see it when we attach a fixed IP, no?
17:02:10 sean-k-mooney i dont see any calls to it directly in the codebase. its spawned as a periodic task so not sure how to call it
17:02:31 sean-k-mooney i just ran wait a breakpoint and it was never called during the test run
17:06:55 stephenfin sean-k-mooney: Yeah, and it is getting set in a production system. Dodgy mocks again
17:07:52 sean-k-mooney ya any idea of how to run them?
17:08:14 sean-k-mooney if i create an instance of the compute manager can i just call it?
17:09:13 stephenfin Not a clue. I guess I need to figure out how to set a watch on an attribute of an o.v.o object so I can see the callers?
17:09:46 openstackgerrit melanie witt proposed openstack/nova stable/ocata: [stable only] Always commit quota decrement if build request found https://review.openstack.org/582413
17:11:59 xgerman_ hi, I am from Octavia and have a question: I had a compute host shutdown. Issued a nova delete and was checking in neutron if a port got detached… but that didn’t happen for 5 minutes. Wonder if there is a nova setting or if I need to increase that timeout…
17:12:09 stephenfin sean-k-mooney: fwiw, I know it works in "production". By default, DevStack deploys with an flat (l2) "public" network and vxlan "private" network. If I configure the two of those to use different NUMA nodes, boot the instance with a port from public, add a port from private and rebuild, it fails with NoValidHost
17:12:28 stephenfin So this has to be a mock not doing something but I just don't know what :)
17:13:30 sean-k-mooney stephenfin: my guess is that the comptue service we start is not running periodic tasks or the neutron mock is not sending the correct event
17:14:08 sean-k-mooney on the plus side i think i figured out how to call the heal method.
17:14:38 stephenfin I don't think it's the periodic task. If I put a breakpoint there (well, LOG.warning), the info_cache.network_info attribute looks correct beforehand
17:15:34 stephenfin sean-k-mooney: This looks promising https://github.com/openstack/nova/blob/master/nova/network/base_api.py#L33-L59
17:24:05 sean-k-mooney stephenfin: we could just add a floating ip to the instance i guess.
17:24:20 stephenfin sean-k-mooney: Exactly what I'm trying to do at the moment
17:24:30 stephenfin well, fixed
17:24:58 sean-k-mooney do you know what context object we need

Earlier   Later