Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-25
16:20:22 cfriesen opposite of sean-k-mooney :)
16:20:27 stephenfin cfriesen: Yeah, that's what I was thinking
16:20:58 stephenfin sean-k-mooney: Relatively speaking though, I imagine users with NUMA topologies >>> users with OVS-DPDK
16:21:08 sean-k-mooney i know yahoo japan use livemigration in there production ovs-dpdk based deployment
16:21:28 cfriesen sean-k-mooney: wouldn't anyone using ovs-dpdk also use pinned CPUs?
16:21:36 sean-k-mooney cfriesen: no
16:21:44 cfriesen freaky
16:21:50 sean-k-mooney they may but that should not be assumed
16:22:22 sean-k-mooney cfriesen: if you entire cloud uses ovs-dpdk you still have instance that you want to over subscive so no pinning
16:22:50 stephenfin sean-k-mooney: I imagine it's still fairly common though
16:23:25 sean-k-mooney there are 2 main usecases for ovsdpdk hosting providres that want to run allot of instnace on a singel host where the aggreate throuput exceed what kernel ovs can do
16:23:46 sean-k-mooney or a small number of vnf that each neeed a lot of bandwith
16:23:58 sean-k-mooney the frist group dont use pinning the second do
16:24:21 stephenfin cfriesen: So, to be clear, workaround option is good enough, as opposed to allowing it for migrations with a requested_destination?
16:24:25 stephenfin For you, at least
16:25:29 stephenfin sean-k-mooney: The way I'm thinking is that this has a potentially devastating impact on workloads and their performance, so if operators are knowingly doing it, they will switch the toggle
16:25:53 stephenfin but in the meanwhile, we'll catch all the other folks who didn't know this wasn't working the way they thought
16:25:53 sean-k-mooney stephenfin: in the pinning case
16:26:18 sean-k-mooney in the hugepages case the livemigation will fail if it can allocat the right hugepages
16:26:27 sean-k-mooney and there is no performacne impact
16:26:46 sean-k-mooney modulo your numa aware vswithc stuff
16:26:47 stephenfin can or can't?
16:27:34 sean-k-mooney the migration will fail if it can not alloate hugepages
16:27:47 cfriesen stephenfin: yes, workaround option would be good enough for me. Let's keep it simple for now.
16:27:58 sean-k-mooney the dest node will emit a qemu error and we will abort the migration
16:28:09 cfriesen I'd rather see live migrations get fixed than get fancy with the workaround.
16:28:25 sean-k-mooney cfriesen: same
16:28:32 stephenfin cfriesen: Agreed. With any luck, this won't even survive the release
16:28:39 stephenfin But it'll be a useful backport, IMO
16:28:54 stephenfin sean-k-mooney: There's also the non-homogeneous hosts case to consider too
16:28:59 sean-k-mooney stephenfin: if you backport it it needs to be off by defualt
16:29:09 sean-k-mooney stephenfin: yes i know
16:29:10 stephenfin sean-k-mooney: Yup, in that case, agreed
16:29:57 stephenfin cfriesen: Cool. I'll wait to see if artom or bauzas want to pipe up. If not, I'll rework accordingly
16:30:09 sean-k-mooney basically want yahoo were doing is if the migration aported (that will happen in the non-homogeneous case too) then they did a manual migration specifying the host
16:30:33 sean-k-mooney e.g. if it did not jsut work the found a node that could fit it and forced the migration by setting it
16:31:31 stephenfin It wouldn't fail if the guest ended up split over multiple host NUMA nodes though
16:31:41 sean-k-mooney stephenfin: you can proably simply this a lot but doing the check at the api level and if the vm has a numa toplogy just reject the migration based on the config value
16:32:27 stephenfin I know that brings us back into performance issue territory but still, just shows how this is all sorts of broken rn
16:32:29 sean-k-mooney stephenfin: you are thinkg of pinning agin yes. and ture
16:33:17 stephenfin No, pinning isn't needed. We restrict an instance NUMA node to a range of host cores corresponding to host NUMA node
16:33:19 cfriesen sean-k-mooney: do you have any thoughts on whether we should support TIS vs CRB in the emulated TPM spec? and if so, should that be a separate thing from the TPM version?
16:33:24 sean-k-mooney stephenfin: yes it is but we know that you are technically in undefined behavior land if you migrate with a numa toplogy
16:33:52 sean-k-mooney cfriesen: i know thate they are a thing but not enough to comment on that
16:34:18 stephenfin Also, mriedem noted about doing it higher but it seemed no simpler. This doesn't affect anything but libvirt so that still seems like the place to do this.
16:34:21 sean-k-mooney cfriesen: prably use two traits one for the version and a second for the type
16:34:27 cfriesen sean-k-mooney: basically, with TPM 2.0 there are two different ways of talking to the TPM.
16:34:33 cfriesen and qemu supports both
16:34:44 sean-k-mooney ok
16:34:48 cfriesen so I was thinking it's not really a trait
16:34:56 cfriesen since it's not schedulable, at least for qemu
16:35:02 sean-k-mooney well then it s a flavor extra spec
16:35:09 sean-k-mooney and or image meta
16:36:00 sean-k-mooney so retuired trait for tpm_2.0 and hw:tpm_connection_mode=CRB
16:36:04 cfriesen if they specify CRB with TPM 1.2, should I ignore the CRB request or error out?
16:36:27 sean-k-mooney cfriesen: proabley error but you could leave that up to qemu
16:36:58 sean-k-mooney libvirt/qemu should be validteing there input and should reject it if invalide
16:37:35 sean-k-mooney you could optimes by checking in the driver but not sure that needs to be in the spec
16:37:49 cfriesen makes sense
16:39:20 sean-k-mooney stephenfin: if you have a config or provide a way to bypass the check im ok with it
16:40:17 sean-k-mooney stephenfin: if i cant turn it off you will break people and im not really ok with saying vpp ovs-dpdk, vrouter and snabb swtich deployments can nolonger live migrate
16:58:49 kmalloc cdent: out of curiosity, does Nova really support passing config data via paste-ini or is it mostly assumed you're passing through config via .conf files [even to middleware]?
16:59:11 kmalloc cdent: looking at how deep i need to support config passing into the new loader when allowing folks to drop paste.
16:59:28 kmalloc if you know.
16:59:43 cdent it's not so much the passing of config data (although that may be in there) but the idea that paste.ini can be edited to add other middlewares
17:00:06 kmalloc right. that is fine.
17:00:09 kmalloc i'm planning on supporting that
17:00:23 cdent and so it falls under the domain of stuff that we need to carry on dealing with incase people have their own changes
17:00:38 kmalloc the other question i have is: if i only allow middleware to be injected into a specific place in the pipeline, where would that be?
17:00:46 kmalloc or do we need it to really be very dynamic
17:00:47 kmalloc ?
17:00:56 cdent sean dague had more to say on the topic way back in the day, and probably new more about what was happening in the wild
17:01:23 kmalloc right. i can flip the coin either way, obviously i wont allow injecting things *after* nova's application
17:01:38 cdent I'm unable to answer that question with about "where" with regard to nova
17:01:56 cdent some of the nova-api oriented folks might be able to (gmann, alex_xu )
17:02:04 kmalloc thnx :)
17:02:31 kmalloc i figure nova is the ideal target to aim for here, since much follows in it's lead
17:03:20 cdent yup
17:04:05 kmalloc keystone was the outlier but we drew the line and said "nope, no unwhitelisted middleware in our app space" but mostly for security/maintenance/behavior reasons
17:04:39 kmalloc it's super easy to add outer middleware to anything, but i don't want to block people who legitimately need things somewhere in the middle
17:04:47 kmalloc say post keystonemiddleware.
17:07:32 melwitt they're APAC, so you might consider the dev ML with [nova][api]
17:15:00 kmalloc melwitt: ah thnx. yeah Pacific has poor overlap with <insert most currently relevant timezones for OpenStack here>
17:15:18 kmalloc melwitt: yeah i'll get an email spun up as well.
17:16:01 melwitt sounds cool
17:32:04 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Integration tests using gabbi-tempest https://review.openstack.org/613386
17:46:28 cfriesen is there a current recommendation on whether to enable the "send_service_user_token" config option? It still defaults to "False", just wondering what the downsides would be...is there an additional keystone round-trip?
17:51:13 artom stephenfin, sorry, missed your ping, I think you settled on config option? I'm cool with that
17:51:35 melwitt cfriesen: found this in the spec, maybe that is the main downside https://specs.openstack.org/openstack/nova-specs/specs/ocata/implemented/use-service-tokens.html#performance-impact
18:26:02 openstackgerrit Merged openstack/nova master: api-ref: Add descriptions of error cases https://review.openstack.org/610429
18:32:58 Sundar melwitt: Do specs need to merged this week (for Stein milestone 1) to be considered for implementation in Stein?
18:33:23 melwitt Sundar: no. our spec freeze is milestone 2, Jan 10
18:35:01 Sundar melwitt: Thanks. Is there any way to accelerate the coverage for the spec https://review.openstack.org/#/c/603955/ ?
18:35:26 Sundar I understand there are lots of other stuff in the Nova plate
18:37:13 melwitt Sundar: looks like there was some review last week, and you've made some updates
18:39:13 melwitt I can mention it in the nova meeting today that it's been updated and ready for review again
18:39:36 Sundar Great. Thanks!
18:47:58 melwitt sean-k-mooney: could you please reply on this os-vif release review about whether we need a major version bump? https://review.openstack.org/612844
19:07:11 openstack bug 1764556 in OpenStack Compute (nova) ""nova list" fails with exception.ServiceNotFound if service is deleted and has no UUID" [Medium,In progress] https://launchpad.net/bugs/1764556 - Assigned to melanie witt (melwitt)
19:07:11 openstackgerrit melanie witt proposed openstack/nova master: Add functional recreate test for bug 1764556 https://review.openstack.org/562041

Earlier   Later