| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-23 | |||
| 13:20:38 | pvc_ | bauzas Failed to acquire pid file '/var/run/libvirtd.pid': Resource temporarily unavailable :( | |
| 13:20:52 | pvc_ | bauzas fio error: cad68f60-930c-4d9b-b954-3e0cd855651e: error getting device from group 58: Input/output error | |
| 13:33:09 | pvc_ | anyone can help? | |
| 13:53:03 | pvc_ | hi sean-k-moone do i need to hide the hypervisor of the image? | |
| 13:53:12 | pvc_ | hi sean-k-mooney do i need to hide the hypervisor of the image? | |
| 13:53:13 | alex_xu | jaypipes: for https://review.openstack.org/#/c/555081, are you saying that the user must specify guest numa topology when using resources:PCPU=1 or resources:VCPU=1 | |
| 13:54:42 | sean-k-mooney | alex_xu: im not suer if cpu pinning auto creates a numa toplogy today but it is does not its one of the few numa specifc things that does not | |
| 13:56:16 | pvc_ | sean-k-mooney i have an error on my XML | |
| 13:56:53 | pvc_ | 2bf12bf5 - default default] Error launching a defined domain with XML: <domain type='kvm'> | |
| 13:57:29 | alex_xu | sean-k-mooney: yes, I also think that. If the flavor doesn't include any guest numa topo, then we will get a None value for the InstanceTopologyObj. But jaypipes still want to use InstnaceTopology to store the cpu pinning. that is my confuse. | |
| 13:58:49 | sean-k-mooney | alex_xu: well cpus have numa affintiy so i would be fine with saying if your request pinning you now have a numa toploy of 1 numa node for the vm unless you set a numa toploygy explcitly | |
| 13:58:57 | sean-k-mooney | alex_xu: we do this for hugepages | |
| 13:59:25 | sean-k-mooney | personly i have normally argued against that but we have too much presdent to change it at this point | |
| 13:59:26 | openstackgerrit | Dan Smith proposed openstack/nova master: Make CellDatabases fixture reentrant https://review.openstack.org/611665 | |
| 13:59:27 | openstackgerrit | Dan Smith proposed openstack/nova master: Modify get_by_cell_and_project() to get_not_deleted_by_cell_and_projects() https://review.openstack.org/607663 | |
| 13:59:28 | openstackgerrit | Dan Smith proposed openstack/nova master: Minimal construct plumbing for nova list when a cell is down https://review.openstack.org/567785 | |
| 13:59:28 | openstackgerrit | Dan Smith proposed openstack/nova master: Refactor scatter-gather utility to return exception objects https://review.openstack.org/607934 | |
| 13:59:28 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova show when a cell is down https://review.openstack.org/591658 | |
| 13:59:29 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova service-list when a cell is down https://review.openstack.org/584829 | |
| 13:59:37 | pvc_ | is that related sean-k-mooney you think? | |
| 14:00:13 | sean-k-mooney | pvc_: i dont know but im busy with 3 other things. i do not have time to help futher sorry. | |
| 14:01:09 | alex_xu | sean-k-mooney: yea, that should be ok, that is just a clarify I ask on the spec, since it isn't clear about that | |
| 14:02:41 | sean-k-mooney | alex_xu: for what its worth the free cpus are already tracked in the numa toployg blob in the nova db so i dont hink jay was proposing changing that | |
| 14:02:44 | alex_xu | jaypipes: ^ probably that is what I'm asking, are you plan to change the guest without numa topo to single numa cell topo | |
| 14:03:02 | jaypipes | alex_xu: *currently* there is no way for a user to get pinned CPUs without the instance_extra.numa fields containing a serialized blob of InstanceNUMATopology object. | |
| 14:03:47 | jaypipes | alex_xu: because, as you know, we couple the CPU pinning, memory page and NUMA topology stuff all together in the InstanceNUMATopology object :( | |
| 14:03:55 | sean-k-mooney | jaypipes: do we currently invent a singel numa node topology today. its been to long since i looked at the details of that code to rember that off the top of my head | |
| 14:04:04 | jaypipes | alex_xu: the cpu-resource-tracking spec proposes absolutely no changes to any of that. | |
| 14:04:43 | sean-k-mooney | jaypipes: i have you spec on my list to review but i assumed we would still contiue to do whatever we do today on that front | |
| 14:05:19 | jaypipes | sean-k-mooney: mriedem has basically shot down the possibility of cpu-resource-tracking happening in stein anyway, so I haven't been spending much time on it. :( | |
| 14:05:44 | alex_xu | jaypipes: yes, but your spec didn't say I must specify HW:NUMA_xxx stuff when I use resources:PCPU=1 | |
| 14:05:46 | mriedem | once again i am the killer of all hopes and dreams and kittens | |
| 14:06:02 | jroll | always and forever | |
| 14:06:12 | mriedem | if others think we can pull that change off in stein and are planning on devoting review time to it, then go nuts | |
| 14:06:17 | jaypipes | alex_xu: I was asked by bauzas to take all NUMA stuff out of the cpu-resource-tracking spec so he could address it in his numa spec. | |
| 14:06:58 | alex_xu | jaypipes: and what does mean for CONF.shared_cpu_set, it is for the VCPU will pinning to the CPU set of CONF.shared_cpu_set, and then I must specify HW:NUMA_xxx with resources:VCPU=1? | |
| 14:08:00 | jaypipes | alex_xu: I don't understand your question. could you rephrase? | |
| 14:08:07 | alex_xu | let me try :) | |
| 14:08:13 | jaypipes | alex_xu: CONF.cpu_shared_set already exists, btw | |
| 14:09:12 | sean-k-mooney | alex_xu: jaypipes so jsut looking at https://github.com/openstack/nova/blob/297de7fb9fbabe74b5305ef0aa82e196d5f48d5e/nova/virt/hardware.py#L1543-L1554 we create a singel node numa toplogy for the guest today if using pinning unless you say otherwise | |
| 14:09:49 | jaypipes | sean-k-mooney: right, because we've coupled CPU pinning and NUMA together into the InstanceNUMAToplogy object. | |
| 14:09:55 | sean-k-mooney | alex_xu: so if you just set resources:PCPU=1 then i would assume we would create a singel numa toplogy | |
| 14:10:20 | sean-k-mooney | jaypipes: ya. if you want to decouple them and fix it im happy with that idea too | |
| 14:10:37 | jaypipes | sean-k-mooney: mriedem would never approve such a gigantic change. :P | |
| 14:10:43 | alex_xu | sean-k-mooney: no, we return early at https://github.com/openstack/nova/blob/297de7fb9fbabe74b5305ef0aa82e196d5f48d5e/nova/virt/hardware.py#L1538, actualy it is NOne | |
| 14:11:18 | sean-k-mooney | alex_xu: that for shared cpus | |
| 14:11:43 | jaypipes | alex_xu: resources=VCPU:1 does not equal cpu_policy:shared | |
| 14:11:59 | sean-k-mooney | pinned cpus have cpu_policy==dedicated | |
| 14:12:00 | jaypipes | alex_xu: just another example of terrible coupling in this interface :( | |
| 14:12:06 | jaypipes | if cpu_policy == fields.CPUAllocationPolicy.SHARED: | |
| 14:12:16 | jaypipes | ^^ that is not the same as resources=VCPU:1 | |
| 14:12:27 | alex_xu | jaypipes: your spec is about decouple cpu pinning and numa. so conf.cpu_shared_set defined the pcpus which the shared VCPU is running. if the conf.cpu_shared_set=7-15, dose it means nova-compute will pin the guest vcpus to the physical cpu 7 to 15? | |
| 14:13:00 | sean-k-mooney | alex_xu: it will float them over that rage | |
| 14:13:07 | sean-k-mooney | *range | |
| 14:13:17 | sean-k-mooney | but it wont 1:1 pin the shared cpus | |
| 14:13:25 | jaypipes | alex_xu: no, my spec is not about decoupling CPU pinning and NUMA... my spec is about handling the allocation of dedicated CPUs in a deliberate way. My spec does not touch assignment of host processor to guest vCPU thread, which is what you are referring to. | |
| 14:13:27 | sean-k-mooney | that will be left to the kernel | |
| 14:15:10 | alex_xu | sean-k-mooney: jaypipes so for the request resources:VCPU=1, this VCPU sitll can running on the pcpu which defined in conf.cpu_dedicated_set... | |
| 14:16:27 | sean-k-mooney | alex_xu: with jays spec no. if i rememebre correctly jay was proposeing depercating the hw:cpu_policy extra spec and vcpu would courrespond to the shared set and PCPU reousfce woudl be from dedicated set | |
| 14:17:11 | jaypipes | alex_xu: if the virt driver isn't changed to assign one of the dedicated host CPUs, yep. But from placement (and resource tracking) perspective, we don't care about that. All we care about is that some amount of dedicated (or shared) CPU resources are being deducted from the appropriate inventory of that class of resource (either VCPU or PCPU) | |
| 14:18:14 | sean-k-mooney | alex_xu: jays spec is basicaly discribing how we will keep a tally count of PCPU and VCPU in placement | |
| 14:19:01 | sean-k-mooney | the asiginment of vms to host dedicated or shared cpu sets will be handeled by the virt driver not placment using the exisitng numa toplogy blob in the nova db as we do today | |
| 14:19:26 | sean-k-mooney | placement will just make sure we have enough cpus to fulltile the request without tracking which ones are free | |
| 14:19:44 | sean-k-mooney | thats the virt driver/ resouce trakers jobs | |
| 14:19:45 | jaypipes | sean-k-mooney: and yes, you're right that my spec proposes deprecating the cpu_policy extra spec. | |
| 14:21:15 | sean-k-mooney | i think i left a comment about may using it to translate the flavor VCPU filed into resouces:VCPU=X or resources:PCPU=x to ease transition but long term it would nolonger be needed | |
| 14:21:30 | alex_xu | ah....I probably I see...give me more seconds... | |
| 14:24:41 | mriedem | is tpatil intel? | |
| 14:24:51 | mriedem | oh NTT | |
| 14:25:29 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: Move live_migration.pre.start to the start of the method https://review.openstack.org/612714 | |
| 14:25:30 | openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: Ensure attachment cleanup on failure in driver.pre_live_migration https://review.openstack.org/612715 | |
| 14:25:39 | artom | mriedem, ^^ it has begun *dun dun dun* | |
| 14:26:06 | pvc_ | hi anyone | |
| 14:26:14 | pvc_ | how can i remove a pci devices? | |
| 14:26:29 | pvc_ | nova_libvirt searching for it but it is not existing | |
| 14:28:54 | openstackgerrit | Matthew Booth proposed openstack/nova master: Fix test bug when host doesn't have /etc/machine-id https://review.openstack.org/612717 | |
| 14:32:07 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add regression test for bug 1550919 https://review.openstack.org/591733 | |
| 14:32:07 | openstack | bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9) | |
| 14:32:37 | alex_xu | sean-k-mooney: jaypipes with that spec, the request with resources:PCPU=1 and without any HW:NUMA_.. stuff, that vcpu is also floating on all the pcpus? | |
| 14:33:15 | alex_xu | since that spec is only about the counting pcpu and vcpu... | |
| 14:33:55 | openstackgerrit | Matthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate https://review.openstack.org/578846 | |
| 14:35:06 | mriedem | pvc_: just fyi, today is a spec review sprint in nova so most people are busy with that. you could try asking your questions in the #openstack or #openstack-operators channels. for pci passthrough questions i'd normally direct you to sahid or cfriesen or moshele but none of them are online right now. | |
| 14:35:35 | mriedem | i'd also think that excluding the pci devices you don't want to expose from https://docs.openstack.org/nova/latest/configuration/config.html#pci.passthrough_whitelist would work, but i don't know a lot about that code | |
| 14:35:55 | mriedem | pvc_: you could also post a question to the openstack-dev mailing list | |
| 14:36:08 | mriedem | if this is a common problem and we don't have documentation for it, then we should have a docs bug | |
| 14:36:14 | pvc_ | thank you so much | |
| 14:36:17 | pvc_ | i will do that | |
| 14:36:20 | mriedem | yw | |
| 14:38:33 | jaypipes | alex_xu: yes | |
| 14:39:01 | jaypipes | alex_xu: since the hw:numa_xxx tags are currently the only way to trigger any pinning behaviour. | |
| 14:39:31 | sean-k-mooney | well not quite you can use hw:cpu_policy | |
| 14:39:33 | alex_xu | jaypipes: ok, i see now, then in the future, we want that case work correctly, right? | |
| 14:39:55 | jaypipes | alex_xu: and since my spec doesn't propose any changes to that, then the existing behaviour if an instance does not have the hw:numa_xxx specs means its vCPU threads float over whatever host processors are in CONF.cpu_shared_set. | |
| 14:40:10 | alex_xu | sean-k-mooney: yea, without hw_cpu_polciy also | |
| 14:40:20 | sean-k-mooney | alex_xu: you should assume that if you have resouce:PCPU=X the virt drive will pin those cores but how it does that is not really realted to jays spec | |
| 14:40:39 | jaypipes | sean-k-mooney: you should NOT assume that. | |
| 14:41:05 | sean-k-mooney | jaypipes: why that was the prerequeit for deprecating hw:cpu_policy | |
| 14:41:09 | jaypipes | sean-k-mooney: the only thing that guarantees assignment to a particular host CPU is the presence of hw:numa_xxx specs | |
| 14:41:45 | sean-k-mooney | the hw:numa_xxx specs today do not do that | |
| 14:41:55 | jaypipes | sean-k-mooney: that is a change that the virt driver will need to make, yes. but that change isn't part of my spec... | |