Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-08-25
09:58:51 itandops There is a database with empty values nova_cell0
09:59:37 itandops It seems to be the root cause of this issue. Any suggestion to solve this ?
10:03:35 lyarwood itandops: `Lost connection to MySQL server during query` it isn't that the db is empty
10:06:03 itandops lyarwood: I used 2 controllers running nova API.
10:07:16 lyarwood itandops: okay well there's a clear db connectivity issue there for you to debug now :)
10:15:44 sean-k-mooney gibi are you going to try and resolve the sriov attach/detach feature before m3
10:16:50 sean-k-mooney gibi: im still trying to find time to work on numa in placement but im fully expecting that to slip to next cycle. but im going to try and at least have a poc of it before repoposing the spec next cycle
10:19:01 gibi sean-k-mooney: I will try to push that feature later this week but I think at some point I have to accept that it will slip
10:19:54 gibi Yumeng: if there is a clear list of open issues in the spec then that is enough
10:20:53 sean-k-mooney ok, do you know what the status of provider.yaml is? https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/provider-config-file
10:22:00 gibi sean-k-mooney: I will go back to provider.yaml today, just finished reviewing the cyborg evac support. (going through the runway items)
10:22:35 gibi so I can tell more in couple of hours
10:23:47 sean-k-mooney ok it looks like the first patch merged and the second has a +2 from alex the 3rd or 4th patch makes the feature functional
10:24:27 sean-k-mooney so that look like its making progress ill try to review the runway items but maybe not today.
10:25:17 jsuchome ok gibi, lyarwood, https://review.opendev.org/#/c/574301/ is green again, chance for new reviews?
10:25:58 gibi jsuchome: ohh, I was +2 on that before, so I hope I can make a quick rereview of that now
10:27:00 lyarwood jsuchome: ack looking
10:27:27 lyarwood gibi: I went ahead and moved a few things around with that, there's another change underneath from me now moving rbd_utils out of nova.virt.libvirt for example
10:27:45 lyarwood gibi: and this change also introduces some generic configurables now
10:27:58 gibi lyarwood: yeah, I just opened your change below
10:28:48 lyarwood jsuchome: https://review.opendev.org/#/c/728095/ can you update the releasenote to include the new configurables please
10:31:09 lyarwood bauzas: https://review.opendev.org/#/q/I32908b77c18f8ec08211dd67be49bbf903611c34 - would you mind hitting these today?
10:31:24 bauzas lyarwood: I can take a look
10:32:30 lyarwood thanks
10:33:03 lyarwood https://review.opendev.org/#/q/I7eb86edc130d186a66c04b229d46347ec5c0b625 is the other change that's part of this bugfix
10:49:10 openstackgerrit Luyao Zhong proposed openstack/nova master: Track error migrations in resource tracker https://review.opendev.org/745281
10:49:11 openstackgerrit Luyao Zhong proposed openstack/nova master: Rollback when reset running live migration https://review.opendev.org/747899
10:49:11 openstackgerrit Luyao Zhong proposed openstack/nova master: Improve orphans tracking in resource tracker https://review.opendev.org/745282
11:00:52 openstackgerrit Luyao Zhong proposed openstack/nova master: Rollback when reset running live migration https://review.opendev.org/747899
11:03:22 luyao stephenfin, alex_xu: Thank you for your comments. if you get time, vpmem enhancement is ready for new round review. https://review.opendev.org/#/c/714653/
11:09:41 openstackgerrit Jiri Suchomel proposed openstack/nova master: Bring back allowed_direct_url_schemes in support of RBD image download https://review.opendev.org/728095
11:21:05 sean-k-mooney it would be really nice if you could resize a shelved instance
11:23:17 gibi sean-k-mooney: you can automat that on the client side :) unshelve; resize; shelve
11:23:23 sean-k-mooney im going to file a bug for numa blancing later today and work on a patch to resolve it later in the week
11:23:35 sean-k-mooney gibi: thats what im doing with horizon manually
11:23:46 sean-k-mooney while i finished reading my email
11:23:54 sean-k-mooney which is now alos done
11:24:29 sean-k-mooney gibi: it just feels like something that would be easy to do in the shevled offloaded state as its just a db update
11:24:51 sean-k-mooney resize while shelve offloaded that is
11:25:04 gibi hm, that makes more sense yes
11:25:12 sean-k-mooney but thats an api change and needs a spec so not this cycle
11:41:47 gibi swp20: left some comments in https://review.opendev.org/#/c/719163/18/nova/scheduler/client/report.py@2170
11:42:51 openstackgerrit Balazs Gibizer proposed openstack/nova master: Undeprecate the vmwareapi driver https://review.opendev.org/742407
11:45:52 sean-k-mooney gibi: i know vmware jobs are now working again but do they trigger automatically or is it only when the recheck comment is left
11:46:15 gibi sean-k-mooney: it is triggered automatically
11:46:29 gibi I just pushed a new ps and it will vote on it
11:46:41 sean-k-mooney ok cool
11:46:59 sean-k-mooney initally it looked like it was manual but i guess they made it automatic over the cycle
11:47:42 sean-k-mooney if its still it seams to be relitivly stable so if they continue to maintain it then un deprecating it makes sense.
11:48:17 gibi looking at the recent history it of the job I think it is fairly stable. I also see some patches targeting the vmware driver so there is real maintenance on the code now
11:48:45 gibi e.g. https://review.opendev.org/#/c/742398/
11:49:30 sean-k-mooney ya
11:49:54 sean-k-mooney reading the commit message for your change it makes sense to me
11:59:48 openstackgerrit sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432
12:19:33 lyarwood stephenfin: https://review.opendev.org/#/q/topic:bp/nova-image-download-via-rbd any chance you could add this to your queue, I'll get to the vTPM stuff tomorrow if that's okay
12:28:55 gibi stephenfin, sean-k-mooney: could you confirm that the "any number" here is a doc bug? https://github.com/openstack/nova/blame/master/doc/source/user/flavors.rst#L549-L551 I belive if hw:numa_nodes is not specified then it is defaulted to 1
12:36:25 jsuchome gibi, lyarwood so who could we ask for another +2?
12:36:50 gibi jsuchome: I think lyarwood just asked stephenfin above
12:38:48 lyarwood yup I did
12:39:46 sean-k-mooney gibi: it depends on the context but ill check now
12:40:44 sean-k-mooney gibi: the docs are correct
12:41:17 gibi sean-k-mooney: they why I remember that nova does not automatically split resources between host NUMA nodes?
12:41:27 gibi for a single instance
12:41:46 sean-k-mooney gibi: it does not but it does not crete a numa toplogy by default either
12:42:15 sean-k-mooney hw:numa_cpus and hw:numa_mem are only used if hw:numa_nodes>=2
12:42:40 gibi OK, so if nothing numa related are requested then we split resource between host numa nodes
12:42:43 sean-k-mooney and by default vms without a numa toplogy float over all host numa nodes
12:42:57 sean-k-mooney gibi: no we dont split them
12:42:58 gibi but if we request huge pages then a numa topology is created
12:43:11 sean-k-mooney we simply do not limit them to a specific numa node
12:43:15 gibi and then numa_nodes defaults to 1 if not specified
12:43:18 sean-k-mooney the kernel will split and move tehm as it sees fit
12:43:25 sean-k-mooney yes
12:43:44 sean-k-mooney if you request hugepage or cpu pinning or pmem we default to 1 numa node
12:43:50 gibi OK, cool
12:43:56 gibi then it is not a doc bug
12:44:00 gibi it is just a complication
12:44:06 gibi thanks
12:44:12 sean-k-mooney no worries
12:44:40 sean-k-mooney the other complication is while we support numa affinity policies with pci device/sriov ports we dont create a numa toplogy when you use them by default
12:44:56 sean-k-mooney we just enforce those policies if you also request numa for an instance
12:45:05 gibi I see
12:46:24 sean-k-mooney the descrioption of FLAVOR-NODES is not really correct
12:46:33 sean-k-mooney for a different reason
12:46:41 sean-k-mooney no of these values are related to host toplogy
12:47:02 sean-k-mooney hw:numa_nodes is the number of virtual numa_nodes that we present to the guest
12:47:34 sean-k-mooney the relationship with the host numa nodes is a libvirt driver implemeatnion detail and is not guarenteed by the api currenly
12:47:53 sean-k-mooney althogh it is the behavior the numa toplogy filter will give you
12:47:57 gibi it talks about where the cpus are run on and in that sense it is correc that they can run on any host numa nodes
12:48:21 sean-k-mooney yes
12:49:53 sean-k-mooney but we technically would be allowed to change the meaning of hw:numa_nodes in libvirto to mean spread to at most N host numa nodes instead of exactly N numa nodes with out violating that api guarentees today. although the proposed placment work will actully guarentee a 1:1 mapping
12:51:02 gibi ahha
12:51:12 sean-k-mooney well actully it does not have too but we likely will to keep the same behavior
12:52:31 sean-k-mooney the one thing we are not allowed to do by the current api semantics is take a vm that reqest 1 numa node and spread it across several host numa nodes
12:53:20 sean-k-mooney i.e. we cant split virutal numa nodes across host numa nodes without regressing performance
12:53:35 gibi yes, that would be a serious bug
12:54:01 sean-k-mooney the other way around is ok but if we allow it in the futrue we would likely want to contol it via policy
12:54:30 sean-k-mooney e.g. hw:numa_packing=true or something like that and default to false
12:55:01 sean-k-mooney anyway we dont need to worry about that right now anyway
12:55:34 gibi thanks for the explanation

Earlier   Later