| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-03 | |||
| 14:54:24 | mriedem | there is no backport of the revert | |
| 14:54:29 | mriedem | so this https://review.openstack.org/#/c/524618/ | |
| 14:55:25 | efried | ^ needs to be reverted is what you're saying? | |
| 14:55:30 | mriedem | in the bug they said this happens if compute starts before controller, so i have to assume that includes placement isn't up yet, so we're hitting a flow where the RT is posting inventory for the first time and the compute node RP can't be created | |
| 14:56:03 | mriedem | i would like to avoid a revert if possible | |
| 14:56:10 | mriedem | i'm trying to inspect the change and flow to see if there is something we're missing | |
| 14:57:46 | efried | mriedem: Is it weird that https://review.openstack.org/#/c/562739/ is a revert in ocata, but pike was skipped? | |
| 14:57:59 | mriedem | we only saw the ci failures in ocata | |
| 14:58:04 | mriedem | i haven't seen those in pike | |
| 14:58:18 | efried | okay, I just thought that wasn't allowed. | |
| 14:58:42 | mriedem | i bet i know what it is | |
| 14:58:44 | mriedem | bazinga! | |
| 14:58:46 | efried | I guess it wasn't a cherry-pick of a revert; it was a revert of a cherry-pick. | |
| 14:59:29 | mriedem | set_inventory_for_provider -> _ensure_resource_provider -> _create_resource_provider -> safe_connect returns None because it can't talk to placement yet | |
| 14:59:37 | mriedem | https://review.openstack.org/#/c/524618/2/nova/scheduler/client/report.py@516 | |
| 14:59:40 | mriedem | so we put None in the cache | |
| 14:59:43 | mriedem | kafuckingboom | |
| 14:59:43 | efried | That sounds totally right. | |
| 14:59:57 | melwitt | mriedem: do you have any idea if this bug is still a thing? it's ancient https://bugs.launchpad.net/nova/+bug/1366168 | |
| 15:00:00 | openstack | Launchpad bug 1366168 in OpenStack Compute (nova) "delete private flavor will cause access problem with existed instance" [Medium,Confirmed] | |
| 15:00:42 | mriedem | melwitt: would have to look later, we need a fix for this placement thing in stable | |
| 15:00:51 | melwitt | k, mah bad | |
| 15:00:54 | mriedem | mp | |
| 15:00:56 | mriedem | *np | |
| 15:02:25 | mriedem | I'll wip up a thing | |
| 15:02:48 | mriedem | why we hit this in ocata ci and not pike, i'm not sure | |
| 15:02:59 | mriedem | maybe devstack changed the order of install stuff in pike for the fleetify | |
| 15:11:41 | openstackgerrit | Merged openstack/nova stable/pike: libvirt: Make `cpu_model_extra_flags` case-insensitive for real https://review.openstack.org/565670 | |
| 15:16:32 | melwitt | frickler: hi, could you please lift the -2 on this devstack change https://review.openstack.org/484973? the change that depends on it has been approved https://review.openstack.org/333990 | |
| 15:16:52 | kashyap | mriedem: Thanks for the review here, I wrote that warning in a hurry (hence Workflow -1): https://review.openstack.org/#/c/565242/1/nova/virt/libvirt/driver.py | |
| 15:17:29 | melwitt | frickler: and then the follow up patch (that depends-on the nova change) can be approved to re-enable the novnc tests https://review.openstack.org/527812 | |
| 15:17:32 | kashyap | I want get the wording right. I will update that, and address the unit tests. | |
| 15:26:04 | efried | gibi: o notification deity supreme... | |
| 15:26:12 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle @safe_connect returns None side effect in _ensure_resource_provider https://review.openstack.org/566096 | |
| 15:26:40 | efried | gibi: Do you know of any examples you can point to where an op/admin wrote a script/daemon that listens for notifications and triggers some action? | |
| 15:27:24 | gibi | efried: there should be a couple of services listening to notifications, let me dig up links | |
| 15:29:14 | gibi | efried: searchlight: https://github.com/openstack/searchlight/blob/master/searchlight/elasticsearch/plugins/nova/notification_handler.py | |
| 15:29:41 | mriedem | designate sink | |
| 15:29:43 | mriedem | ceilometer | |
| 15:29:45 | mriedem | mistral | |
| 15:29:56 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: Configure multiple vGPU types https://review.openstack.org/564214 | |
| 15:29:57 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: mdevs returning parent and vendor PCI info https://review.openstack.org/562304 | |
| 15:29:58 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: Get inventories per vGPU type https://review.openstack.org/564215 | |
| 15:30:05 | efried | gibi: Thanks. | |
| 15:31:53 | gibi | efried: https://github.com/openstack/watcher/blob/master/watcher/decision_engine/model/notification/nova.py | |
| 15:32:24 | gibi | efried: I mriedem listed the others in the meanwhile | |
| 15:32:39 | efried | gibi, mriedem: Thanks y'all. | |
| 15:38:38 | openstack | Launchpad bug 1766661 in OpenStack Compute (nova) "'host' property is missing for Cinder volume connector when boot from volume" [Medium,Confirmed] | |
| 15:38:38 | mriedem | TheJulia: left some comments in https://bugs.launchpad.net/nova/+bug/1766661 | |
| 15:38:54 | mriedem | tl;dr nova knows how to get the host connector from the db if needed | |
| 15:39:11 | openstackgerrit | Hongbin Lu proposed openstack/nova-specs master: Choose default network on ambiguity https://review.openstack.org/520247 | |
| 15:39:33 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306 | |
| 15:42:58 | exarlos | Hi all. I had a user attempt a resize of a VM (horizon) and it has failed and reduced the VM to an error state | |
| 15:43:13 | exarlos | ocata / ubuntu / lvm storage | |
| 15:43:39 | exarlos | It mentions not being able to stat the /var/lib/nova/instances folder | |
| 15:44:09 | mriedem | https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7412 | |
| 15:44:34 | mriedem | is it volume-backed or not? | |
| 15:45:17 | exarlos | mriedem: local lvm disks as the backend | |
| 15:45:26 | mriedem | then you'll eventually be failing because ^ | |
| 15:45:46 | mriedem | i think cfriesen might have a patch floating around for that | |
| 15:46:17 | mriedem | https://review.openstack.org/#/c/337334/ | |
| 15:46:21 | exarlos | mriedem: It has worked many times in the past. :-( | |
| 15:46:52 | mriedem | have you patched nova to make that happen? | |
| 15:46:52 | melwitt | so, you're saying regression from newton -> ocata? | |
| 15:48:29 | exarlos | I hadn't patched. There is a allow_resize_to_same_host option in nova.conf that permitted this to happen. | |
| 15:50:01 | mriedem | given https://github.com/openstack/nova/blob/stable/ocata/nova/virt/libvirt/driver.py#L7412 i don't see how that's possible | |
| 15:50:46 | exarlos | Let me check local code. | |
| 15:50:59 | exarlos | Honestly though :-) | |
| 15:53:10 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove support for monitoring Intel CMT `perf` events https://review.openstack.org/565242 | |
| 15:53:17 | bauzas | jaypipes: efried: naichuans (I know it's late for you): I have a thought about providing traits for vGPU types | |
| 15:53:45 | efried | bauzas: Listening. Though I confess I'm behind on those reviews. | |
| 15:53:55 | bauzas | jaypipes: efried: naichuans: the problem is that vGPU types differ in between virt drivers, and even can be depending on the GPU host driver (eg. nvidia vs. intel) | |
| 15:54:06 | bauzas | efried: no worries, it's unrelate | |
| 15:55:00 | efried | bauzas: The thing doing the dev discovery == the thing setting up the resource providers == the thing doing the provisioning == the virt driver. | |
| 15:55:05 | bauzas | jaypipes: efried: naichuans: so, say, I'd like to provide a flavor asking for *this* type, should we provide specific traits like https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#supported-gpus-grid-vgpu ? | |
| 15:55:07 | efried | That's how it boils down for me. | |
| 15:55:49 | bauzas | efried: ok, so you're not concerned by the virt driver setting for example a trait like "nvidia-11" while Xen would set a different one ? | |
| 15:55:56 | efried | not at all. | |
| 15:56:03 | efried | well | |
| 15:56:03 | bauzas | if so, that's super cool, it prevents me to write a mapping table | |
| 15:56:17 | exarlos | 15.0.7 is the version of nova components i'm running. | |
| 15:56:19 | efried | bauzas: If we can standardize some traits that all virt drivers could reasonably use when talking about the same device, that's the best thing. | |
| 15:56:33 | bauzas | efried: see for example https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#vgpu-types-tesla-m60 | |
| 15:56:37 | exarlos | Looks like the code is different, so probably I am a bit behind. | |
| 15:56:55 | efried | bauzas: IOW, the value the virt driver discovers may be different from one virt driver to the next, but as long as it can map it to the standard trait, we're good. | |
| 15:57:08 | bauzas | efried: the GPU type named "M60-0B" is named "nvidia-11" by the nvidia gpu driver | |
| 15:57:29 | bauzas | efried: so the mediated devices etc. use that "nvidia-11" name | |
| 15:57:44 | bauzas | efried: but if you use xen, it's totally named differently | |
| 15:57:55 | efried | bauzas: But is it still a 1-1 mapping? | |
| 15:58:18 | bauzas | efried: https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#how-to-discover-a-gpu-type | |
| 15:58:49 | bauzas | efried: in libvirt, yes, a 1:1 | |
| 15:58:49 | efried | bauzas: eek. | |
| 15:59:01 | efried | Let's talk through the use case real quick. | |
| 15:59:02 | exarlos | mriedem: So, whilst I look at a few things, how best do I pull this back from an error state, and set it so it's bootable again? :-/ | |
| 15:59:19 | bauzas | efried: blame the gpu driver. :p | |
| 15:59:36 | efried | bauzas: From the user pov, is it likely that he's going to put together a flavor that asks for a vgpu and then not care whether he gets a libvirt or xen instance to satisfy it? | |
| 16:00:00 | bauzas | efried: that's a flavor, right? so I don't expect at all end users requesting that | |
| 16:00:17 | efried | sorry, I never remember the right term. | |
| 16:00:21 | bauzas | efried: I rather see operators setting conf file accordingly with flavors | |
| 16:00:45 | bauzas | they could name the flavor GRID M60-0Q I'm fine | |