| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-29 | |||
| 19:06:05 | openstackgerrit | Merged openstack/nova master: Remove unused function https://review.openstack.org/570865 | |
| 19:07:17 | mriedem | jmlowe: what does "openstack resource provider inventory list b0f5d569-687f-43a2-9bdb-309a29597125" show? | |
| 19:08:29 | mriedem | looking at that allocation output, you have 3 instances running on that host with a total of 126976 memory mb consumed | |
| 19:08:52 | jmlowe | http://paste.openstack.org/show/722293/ | |
| 19:09:46 | mriedem | ok and that says there is 130850 mb total of memory | |
| 19:11:19 | mriedem | jaypipes: efried: should max_unit ever be > total - reserved? | |
| 19:11:41 | dansmith | mriedem: I would think it could be | |
| 19:11:42 | efried | mriedem: I think it should be allowed. | |
| 19:11:59 | dansmith | mriedem: consider the case of ironic | |
| 19:12:03 | dansmith | well, that'd be equal I guess | |
| 19:12:07 | efried | Because you should be able to change total and/or reserved without having to worry about also "fixing" max_unit at the same time. | |
| 19:12:12 | dansmith | yeah | |
| 19:12:18 | mriedem | fine fine | |
| 19:12:35 | dansmith | well, yeah, ironic would be total=1, reserved=1, max_unit=1, total-reserved=0 | |
| 19:12:48 | dansmith | for the cleaning state | |
| 19:14:22 | jmlowe | there are 3 instances running on that host, 158726ec-f09d-443c-a700-cce79cd065e0, 5053ebcc-9145-4297-b60f-f2c74ce95aef, 950be30e-a275-4533-9d42-53df6145545b | |
| 19:14:32 | mriedem | jmlowe: how many servers show up on that host if you do "nova hypervisor-servers r06c2b03"? | |
| 19:14:33 | mriedem | same 3? | |
| 19:14:50 | jmlowe | yep, check nova and virsh | |
| 19:15:13 | mriedem | ok, i'm not sure why free_ram_mb shows up as -122863 | |
| 19:15:13 | jaypipes | efried: np | |
| 19:16:18 | jmlowe | this host did once have a allocation for an instance that wasn't present | |
| 19:16:30 | mriedem | based on my calculations you should have 17mb of free ram on that host | |
| 19:16:50 | mriedem | 130850 total - 3857 reserved - 126976 (sum memory mb of allocations of the 3 instances on that host) | |
| 19:17:24 | jmlowe | Is there some cache somewhere that needs updated? | |
| 19:18:26 | mriedem | don't think so, i'm probably missing something in the free ram calculation in the resource tracker, like what the virt driver computes for overhead | |
| 19:19:03 | mriedem | although the libvirt driver doesn't do anything wrt memory_mb overhead | |
| 19:19:21 | esberglu | melwitt: Not sure if you saw my message last week with the summit | |
| 19:19:32 | esberglu | PowerVM vSCSI CI is enabled | |
| 19:19:46 | esberglu | So you should be able to remove the procedural -2 from https://review.openstack.org/#/c/526094/ | |
| 19:21:59 | mriedem | jmlowe: i see this comment but i'm not sure what it means about policy https://github.com/openstack/nova/blob/stable/pike/nova/compute/resource_tracker.py#L913 | |
| 19:22:16 | mriedem | i assume oversubscription? | |
| 19:22:22 | cfriesen | mriedem: jmlowe: is anything else on that compute node consuming ram outside of openstack/qemu/etc? | |
| 19:22:55 | jmlowe | free shows 82G free | |
| 19:28:36 | cfriesen | jmlowe: I think that's largely irrelevent for 4KB pages since they're only allocated on demand. | |
| 19:29:07 | dansmith | cfriesen: other stuff on the box consuming memory shouldn't be represented in placement in any way | |
| 19:29:13 | dansmith | other than reserved if configured in nova | |
| 19:32:05 | melwitt | esberglu: yep, just need to take a look through and will remove the -2 | |
| 19:32:24 | esberglu | Sounds good thanks | |
| 19:33:00 | cfriesen | libvirt.host.Host.get_memory_mb_used() looks at actual memory usage on the host | |
| 19:33:06 | cfriesen | dansmith: ^ | |
| 19:34:03 | dansmith | cfriesen: right but that shouldn't be what we're reporting as reserved | |
| 19:34:15 | mriedem | what we report as reserved comes from config | |
| 19:34:26 | dansmith | right ^ | |
| 19:34:32 | mriedem | what we report as total comes from nova.virt.libvirt.host.Host.get_memory_mb_total | |
| 19:35:00 | mriedem | what we report as used is based on the instances running on the host via the nova db, not what the driver tells us | |
| 21:00:45 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Standardize CPU resource tracking https://review.openstack.org/555081 | |
| 21:03:38 | mriedem | melwitt: not sure if you've thought about it, but we have two things in runway slots that either expire today or tomorrow; was wondering if you wanted to do a 'grace period' extension for those given the summit last week | |
| 21:05:40 | melwitt | mriedem: yeah, I was thinking about something like that. I think it makes sense given the summit week | |
| 21:07:29 | melwitt | maybe just extend all 3 by a week from today? they all have similar original end dates | |
| 21:16:23 | mriedem | sure | |
| 21:25:10 | efried | ++ | |
| 21:27:37 | melwitt | cool, will update | |
| 21:34:44 | openstackgerrit | Zack Cornelius proposed openstack/nova master: Implement file backed memory for instances in libvirt https://review.openstack.org/567876 | |
| 21:34:45 | openstackgerrit | Zack Cornelius proposed openstack/nova master: Refactor libvirt get_memory_used_mb() https://review.openstack.org/571030 | |
| 22:06:12 | mriedem | efried: jaypipes: is it your understanding that if i have a pci request with some entry that's not found in the pci whitelist for a given compute host, the scheduler will filter out that host? | |
| 22:06:26 | mriedem | and that entry can be something totally random, like just foo=bar | |
| 22:06:53 | efried | mriedem: Been a while since I looked at that travesty^W code, but yes, that's my understanding. | |
| 22:06:58 | mriedem | ok | |
| 22:07:18 | mriedem | hard for me to reason by digging through the spaghetti | |
| 22:07:25 | mriedem | i could write a functional test to make sure, but... | |
| 22:07:37 | efried | wait, maybe I misunderstood. You're asking about a *whitelist* entry of foo=bar+ | |
| 22:07:38 | efried | ? | |
| 22:09:25 | efried | mriedem: The super simplified version, IIRC, is that we take whitelist, intersect it with what's reported by get_available_resource, and then look for the requested dev in that intersection. If not found, that host is filtered out. | |
| 22:09:56 | efried | And it's pretty easy for that first intersection to wind up empty | |
| 22:11:07 | mriedem | yeah, for context, i'm reviewing sahid's trusted VF series, which links up a port whose binding profile can have a "trusted" key in it, and that goes into the InstancePCIRequest if trusted is truthy | |
| 22:11:22 | mriedem | now if we have all non-libvirt, or all queens computes, | |
| 22:11:37 | mriedem | there are not going to be any hosts that understand what that trusted tag is in in the pci request via the port binding profile, | |
| 22:11:45 | mriedem | and i wanted to know if the scheduler will kick that request out | |
| 22:11:58 | mriedem | since none of the computes would have the matching 'trusted' tag in their pci whitelist yet | |
| 22:17:30 | efried | mriedem: Oh, okay. I'm pretty sure we ignore keys we don't recognize. | |
| 22:18:15 | efried | mriedem: So adding a random key on an old compute wouldn't cause things to be kicked out where they were previously accepted. | |
| 22:18:20 | efried | I *think*. | |
| 22:18:33 | mriedem | i don't think so, looking at https://review.openstack.org/#/c/565808/4/nova/pci/utils.py | |
| 22:18:53 | mriedem | pci_dev comes from the pci whitelist right? | |
| 22:19:00 | mriedem | and specs is from the InstancePCIRequest.specs | |
| 22:19:11 | efried | you're going to make me look at this code, aren't you | |
| 22:19:25 | mriedem | so if the key was in specs but not the pci whitelist, i think we'd reject it because | |
| 22:19:25 | mriedem | pci_dev_v != v | |
| 22:20:08 | mriedem | also | |
| 22:20:08 | mriedem | pci_dev_v != v | |
| 22:20:10 | mriedem | oops | |
| 22:20:17 | mriedem | 'spec': fields.ListOfDictOfNullableStringsField(), | |
| 22:20:20 | mriedem | NICE! | |
| 22:20:33 | mriedem | ListOfSchmorgazboard | |
| 22:27:02 | efried | mriedem: Okay, so the stuff on L53-6 does what I said, I think. That is, if you ask for k=v in the spec, the dev in the (intersected whitelist + virt-reported) dev list must have that k=v. But any k in the dev list that's not present in the spec is ignored. | |
| 22:28:02 | efried | ...and I believe pci_dev is the (intersected) thingy. | |
| 22:33:38 | mriedem | oh yeah, extra garbage in the pci passthrough_whitelist is ignored, | |
| 22:33:44 | mriedem | but stuff that's in the spec must be in the whitelist | |
| 22:33:48 | mriedem | which is what i want | |
| 22:36:07 | openstackgerrit | Matt Riedemann proposed openstack/nova master: pci: don't consider case when match tags specs https://review.openstack.org/565808 | |
| 22:36:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: configure trust mode for vfs https://review.openstack.org/458514 | |
| 22:36:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: add vf_trusted field for network metadata https://review.openstack.org/566343 | |
| 22:36:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: metadata: add vf_trusted field to device metadata https://review.openstack.org/566344 | |
| 22:36:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: network: update pci request spec to handle trusted tags https://review.openstack.org/458820 | |
| 22:53:26 | mriedem | so, are the pci passthrough_whitelist entries supposed to be lists, or dicts, or either as long as they are serialized json? | |
| 22:54:49 | mriedem | i guess each entry is a serialized dict and we convert it into a list | |
| 22:55:29 | mriedem | er, list or dict, and can be multiples of them | |
| 23:03:56 | efried | mriedem: Yeah, it can be any of those things. It's super confusing. | |
| #openstack-nova - 2018-05-30 | |||
| 01:14:20 | openstackgerrit | Merged openstack/nova master: Fix doc mistakes https://review.openstack.org/570894 | |