| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-25 | |||
| 14:25:44 | spatel | Last question my glance image showing raw image but when i deploy VM it convert image to QCOW why? | |
| 14:26:07 | spatel | is that normal ? | |
| 14:28:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send resource allocations in the port binding https://review.openstack.org/569459 | |
| 14:28:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transfer port.resource_request to the scheduler https://review.openstack.org/567268 | |
| 14:28:21 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317 | |
| 14:59:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove restart_scheduler_service() method https://review.openstack.org/613348 | |
| 14:59:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop legacy live migrate allocation compat code https://review.openstack.org/611975 | |
| 15:33:31 | melwitt | ||
| 15:36:36 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Migrate old style volume attachments on nova-compute startup https://review.openstack.org/549130 | |
| 15:56:45 | openstackgerrit | Merged openstack/nova-specs master: Support initial allocation ratios https://review.openstack.org/552105 | |
| 16:18:05 | stephenfin | cfriesen, bauzas, artom, sean-k-mooney: So, what way are we going on https://review.openstack.org/611088 ? | |
| 16:19:08 | sean-k-mooney | stephenfin: please have a config option and i would prefer it to not be enabled by defualt but i can live with it so long as i can turn it off | |
| 16:19:22 | stephenfin | Personally, I would rather do nothing because I consider the whole thing totally broken. However, that's likely not a realistic approach. As such, it's between a config option and not using the check for forced instances | |
| 16:19:39 | stephenfin | sean-k-mooney: So default to allowing live migrations w/ NUMA or not allowing them? | |
| 16:19:50 | sean-k-mooney | default to allowing it | |
| 16:19:54 | stephenfin | Why? | |
| 16:19:57 | cfriesen | stephenfin: default to safe, allow operators to opt in to being unsafe | |
| 16:19:57 | sean-k-mooney | mainly for ovs-dpdk | |
| 16:20:18 | sean-k-mooney | all ovs-dpdk deployment have numa toplogyes because of hugepages | |
| 16:20:22 | cfriesen | opposite of sean-k-mooney :) | |
| 16:20:27 | stephenfin | cfriesen: Yeah, that's what I was thinking | |
| 16:20:58 | stephenfin | sean-k-mooney: Relatively speaking though, I imagine users with NUMA topologies >>> users with OVS-DPDK | |
| 16:21:08 | sean-k-mooney | i know yahoo japan use livemigration in there production ovs-dpdk based deployment | |
| 16:21:28 | cfriesen | sean-k-mooney: wouldn't anyone using ovs-dpdk also use pinned CPUs? | |
| 16:21:36 | sean-k-mooney | cfriesen: no | |
| 16:21:44 | cfriesen | freaky | |
| 16:21:50 | sean-k-mooney | they may but that should not be assumed | |
| 16:22:22 | sean-k-mooney | cfriesen: if you entire cloud uses ovs-dpdk you still have instance that you want to over subscive so no pinning | |
| 16:22:50 | stephenfin | sean-k-mooney: I imagine it's still fairly common though | |
| 16:23:25 | sean-k-mooney | there are 2 main usecases for ovsdpdk hosting providres that want to run allot of instnace on a singel host where the aggreate throuput exceed what kernel ovs can do | |
| 16:23:46 | sean-k-mooney | or a small number of vnf that each neeed a lot of bandwith | |
| 16:23:58 | sean-k-mooney | the frist group dont use pinning the second do | |
| 16:24:21 | stephenfin | cfriesen: So, to be clear, workaround option is good enough, as opposed to allowing it for migrations with a requested_destination? | |
| 16:24:25 | stephenfin | For you, at least | |
| 16:25:29 | stephenfin | sean-k-mooney: The way I'm thinking is that this has a potentially devastating impact on workloads and their performance, so if operators are knowingly doing it, they will switch the toggle | |
| 16:25:53 | stephenfin | but in the meanwhile, we'll catch all the other folks who didn't know this wasn't working the way they thought | |
| 16:25:53 | sean-k-mooney | stephenfin: in the pinning case | |
| 16:26:18 | sean-k-mooney | in the hugepages case the livemigation will fail if it can allocat the right hugepages | |
| 16:26:27 | sean-k-mooney | and there is no performacne impact | |
| 16:26:46 | sean-k-mooney | modulo your numa aware vswithc stuff | |
| 16:26:47 | stephenfin | can or can't? | |
| 16:27:34 | sean-k-mooney | the migration will fail if it can not alloate hugepages | |
| 16:27:47 | cfriesen | stephenfin: yes, workaround option would be good enough for me. Let's keep it simple for now. | |
| 16:27:58 | sean-k-mooney | the dest node will emit a qemu error and we will abort the migration | |
| 16:28:09 | cfriesen | I'd rather see live migrations get fixed than get fancy with the workaround. | |
| 16:28:25 | sean-k-mooney | cfriesen: same | |
| 16:28:32 | stephenfin | cfriesen: Agreed. With any luck, this won't even survive the release | |
| 16:28:39 | stephenfin | But it'll be a useful backport, IMO | |
| 16:28:54 | stephenfin | sean-k-mooney: There's also the non-homogeneous hosts case to consider too | |
| 16:28:59 | sean-k-mooney | stephenfin: if you backport it it needs to be off by defualt | |
| 16:29:09 | sean-k-mooney | stephenfin: yes i know | |
| 16:29:10 | stephenfin | sean-k-mooney: Yup, in that case, agreed | |
| 16:29:57 | stephenfin | cfriesen: Cool. I'll wait to see if artom or bauzas want to pipe up. If not, I'll rework accordingly | |
| 16:30:09 | sean-k-mooney | basically want yahoo were doing is if the migration aported (that will happen in the non-homogeneous case too) then they did a manual migration specifying the host | |
| 16:30:33 | sean-k-mooney | e.g. if it did not jsut work the found a node that could fit it and forced the migration by setting it | |
| 16:31:31 | stephenfin | It wouldn't fail if the guest ended up split over multiple host NUMA nodes though | |
| 16:31:41 | sean-k-mooney | stephenfin: you can proably simply this a lot but doing the check at the api level and if the vm has a numa toplogy just reject the migration based on the config value | |
| 16:32:27 | stephenfin | I know that brings us back into performance issue territory but still, just shows how this is all sorts of broken rn | |
| 16:32:29 | sean-k-mooney | stephenfin: you are thinkg of pinning agin yes. and ture | |
| 16:33:17 | stephenfin | No, pinning isn't needed. We restrict an instance NUMA node to a range of host cores corresponding to host NUMA node | |
| 16:33:19 | cfriesen | sean-k-mooney: do you have any thoughts on whether we should support TIS vs CRB in the emulated TPM spec? and if so, should that be a separate thing from the TPM version? | |
| 16:33:24 | sean-k-mooney | stephenfin: yes it is but we know that you are technically in undefined behavior land if you migrate with a numa toplogy | |
| 16:33:52 | sean-k-mooney | cfriesen: i know thate they are a thing but not enough to comment on that | |
| 16:34:18 | stephenfin | Also, mriedem noted about doing it higher but it seemed no simpler. This doesn't affect anything but libvirt so that still seems like the place to do this. | |
| 16:34:21 | sean-k-mooney | cfriesen: prably use two traits one for the version and a second for the type | |
| 16:34:27 | cfriesen | sean-k-mooney: basically, with TPM 2.0 there are two different ways of talking to the TPM. | |
| 16:34:33 | cfriesen | and qemu supports both | |
| 16:34:44 | sean-k-mooney | ok | |
| 16:34:48 | cfriesen | so I was thinking it's not really a trait | |
| 16:34:56 | cfriesen | since it's not schedulable, at least for qemu | |
| 16:35:02 | sean-k-mooney | well then it s a flavor extra spec | |
| 16:35:09 | sean-k-mooney | and or image meta | |
| 16:36:00 | sean-k-mooney | so retuired trait for tpm_2.0 and hw:tpm_connection_mode=CRB | |
| 16:36:04 | cfriesen | if they specify CRB with TPM 1.2, should I ignore the CRB request or error out? | |
| 16:36:27 | sean-k-mooney | cfriesen: proabley error but you could leave that up to qemu | |
| 16:36:58 | sean-k-mooney | libvirt/qemu should be validteing there input and should reject it if invalide | |
| 16:37:35 | sean-k-mooney | you could optimes by checking in the driver but not sure that needs to be in the spec | |
| 16:37:49 | cfriesen | makes sense | |
| 16:39:20 | sean-k-mooney | stephenfin: if you have a config or provide a way to bypass the check im ok with it | |
| 16:40:17 | sean-k-mooney | stephenfin: if i cant turn it off you will break people and im not really ok with saying vpp ovs-dpdk, vrouter and snabb swtich deployments can nolonger live migrate | |
| 16:58:49 | kmalloc | cdent: out of curiosity, does Nova really support passing config data via paste-ini or is it mostly assumed you're passing through config via .conf files [even to middleware]? | |
| 16:59:11 | kmalloc | cdent: looking at how deep i need to support config passing into the new loader when allowing folks to drop paste. | |
| 16:59:28 | kmalloc | if you know. | |
| 16:59:43 | cdent | it's not so much the passing of config data (although that may be in there) but the idea that paste.ini can be edited to add other middlewares | |
| 17:00:06 | kmalloc | right. that is fine. | |
| 17:00:09 | kmalloc | i'm planning on supporting that | |
| 17:00:23 | cdent | and so it falls under the domain of stuff that we need to carry on dealing with incase people have their own changes | |
| 17:00:38 | kmalloc | the other question i have is: if i only allow middleware to be injected into a specific place in the pipeline, where would that be? | |
| 17:00:46 | kmalloc | or do we need it to really be very dynamic | |
| 17:00:47 | kmalloc | ? | |
| 17:00:56 | cdent | sean dague had more to say on the topic way back in the day, and probably new more about what was happening in the wild | |
| 17:01:23 | kmalloc | right. i can flip the coin either way, obviously i wont allow injecting things *after* nova's application | |
| 17:01:38 | cdent | I'm unable to answer that question with about "where" with regard to nova | |
| 17:01:56 | cdent | some of the nova-api oriented folks might be able to (gmann, alex_xu ) | |
| 17:02:04 | kmalloc | thnx :) | |
| 17:02:31 | kmalloc | i figure nova is the ideal target to aim for here, since much follows in it's lead | |
| 17:03:20 | cdent | yup | |
| 17:04:05 | kmalloc | keystone was the outlier but we drew the line and said "nope, no unwhitelisted middleware in our app space" but mostly for security/maintenance/behavior reasons | |
| 17:04:39 | kmalloc | it's super easy to add outer middleware to anything, but i don't want to block people who legitimately need things somewhere in the middle | |
| 17:04:47 | kmalloc | say post keystonemiddleware. | |