Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-11-09
15:50:47 spatel sean-k-mooney: Howdy!!!
15:50:50 spatel Morning
15:50:52 spatel https://bugs.launchpad.net/nova/+bug/1792763
15:50:52 openstack Launchpad bug 1792763 in OpenStack Compute (nova) "tap TX packet drops during high cpu load " [Undecided,Invalid]
15:51:34 spatel Yes this could be resolved or close.. because its design question ( i won't say its BUG )
15:53:39 sean-k-mooney ya the drops you were seeing were a limitation of linux bridge as you said so this is not something nova can fix
15:54:41 sean-k-mooney spatel: the kernel can only handel about 1.4mpps on a 3.4GHz cpu and generally its less then that
15:54:50 sean-k-mooney once you exceed that level you get drops.
15:55:51 spatel In my test after 50kpps i was seeing TX drop on tap interface
15:56:33 spatel I think this is issue of tap interface design, it run on kernel space which is overhead on kernel
15:56:42 spatel virtio i meant
15:56:48 sean-k-mooney to get to 1.4 you need kernel ovs with kernel vhost module to acclerate it
15:57:22 sean-k-mooney spatel: yes so its not something openstack can remedy
15:57:33 spatel ++
15:57:47 spatel In my case i am using linuxbridge (not ovs)
15:58:05 spatel do you think OVS is better in performance compare to linuxbridge ?
15:58:13 sean-k-mooney yes it is
15:58:28 sean-k-mooney bar multicast tunnelling
15:58:50 sean-k-mooney if you have a multicast hevey workload use linux bridge as ovs falls back to unicast
15:59:26 sean-k-mooney but in gereral ovs out performes linux bridge in vm based workloads
15:59:32 spatel when you say multicast what is the relation here?
16:00:31 sean-k-mooney linux bridge support using multicast endpoint for teant networks meaning it can more efficetly handel tenatn traffic with a high proportion of broadcase or multicast traffic
16:01:03 sean-k-mooney ovs does not and has to fall back to a unicast mesh toplogy for vxlan
16:01:22 sean-k-mooney but for typical workloads ovs will out perfrom linux bridge
16:04:21 dansmith mriedem: I threw a comment in there about using sysmeta to let virt drivers declare some ops as invalid for an instance. is there some reason that's not reasonable?
16:04:53 dansmith presumably 403 is allowed for pretty much any operation on any microversion, so I would think it'd be not a huge deal, and immediately applies to existing operations in certain situations
16:09:42 mriedem dansmith: yeah, replied
16:09:51 mriedem it really goes back to the capabilities thing we've discussed several times before
16:11:49 mriedem i'm mostly concerned about snapshot, because if you can't move the instance, users are at least going to want to be able to snapshot it i'd think before it has to be destroyed and recreated elsewhere because the compute it's on is going away
16:12:08 mriedem of course this is where someone says, "just attach a data volume and rewrite the application to use that"
16:12:15 spatel sean-k-mooney: thanks for clear that point.. :) i have all unicast workload
16:15:24 dansmith mriedem: did he say snapshot wasn't supported? I would think it would be
16:16:06 dansmith the airplane wifi is sucking too hard for me to even open it again
16:16:12 mriedem it wasn't mentioned
16:16:25 mriedem that's why i asked, because it sure seems like a lot can't be supported
16:18:24 mriedem we got a bug b/c of the limit of tenant ids for the aggregate multitenancy isolation filter, that's resolved with the placement request filter, but doesn't look like the docs for the placement filter mention you can namespace the metadata so you can add as many tenants as you want
16:18:26 dansmith mriedem: the suspend/resume and live migration are about in-memory state, which is why they're hard to support I think
16:18:35 dansmith snapshot, reboot, cold migrate should all be fine I would think
16:18:44 dansmith based on my reading and assumptions about how this works
16:19:07 dansmith mriedem: hmm, I was sure I put that in there
16:19:58 mriedem don't see it, i can push up something for that
16:20:09 dansmith okay
16:20:15 mriedem and i'll probably update the docs for the old filter to mention the limitation (and link to the bug) and say the placement one is a better replacement
16:21:31 dansmith ack
16:21:36 dansmith did I have it in the commit message or something?
16:21:40 dansmith I was sure I wrote words about this
16:22:49 mriedem https://review.openstack.org/#/c/545002/27 "This also allows making this filter advisory but not required, and supports multiple tenants per aggregate, unlike the original filter."
16:22:51 mriedem maybe that
16:24:52 dansmith nova with the ``filter_tenant_id`` key (optionally suffixed with any string for
16:24:52 dansmith multiple tenants,
16:24:55 dansmith https://review.openstack.org/#/c/557490/8/releasenotes/notes/tenant_aggregate_placement_filter-c2fed8889f43b6e3.yaml
16:24:58 dansmith in the reno not hte docs
16:25:04 dansmith tha's mah bad
16:25:22 mriedem k, i'll copy that
16:28:58 openstackgerrit Balazs Gibizer proposed openstack/nova master: Calculate port_id rp_uuid mapping for binding https://review.openstack.org/616239
16:28:59 openstackgerrit Balazs Gibizer proposed openstack/nova master: Pass allocations and traits to neturonv2 api https://review.openstack.org/616240
16:28:59 openstackgerrit Balazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding https://review.openstack.org/569459
16:29:00 openstackgerrit Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317
16:33:39 openstackgerrit Matt Riedemann proposed openstack/nova master: Mention meta key suffix in tenant isolation with placement docs https://review.openstack.org/616991
16:38:36 mriedem sean-k-mooney: is it just me or is NeutronLinuxBridgeInterfaceDriver completely replaced with os-vif now?
16:41:50 sean-k-mooney ill need to check but probably
16:42:17 mriedem i think it would only be used via the linuxnet_interface_driver config option, but i don't see anything with neutron in nova using the code path that hits that option
16:42:21 mriedem only the nova-network l3 stuff
16:42:46 sean-k-mooney we can likely kill it when we kill nova networks
16:42:59 mriedem sure but this is a neutron-specific driver
16:43:45 sean-k-mooney mriedem: ill look into it next week while and see what its actully used for but your right
16:44:12 sean-k-mooney * while ye are at the summit
16:45:17 cfriesen stephenfin: question about your commit https://review.openstack.org/#/c/526329 One of our guys says he ran into a scenario in pike where "image_chunks" itself was None due to things like firewall breakage or server-side problems. Does that get handled properly currently?
16:46:17 openstackgerrit Merged openstack/nova stable/pike: Fix the request context in ServiceFixture https://review.openstack.org/599839
16:46:23 openstackgerrit Merged openstack/nova stable/pike: Add functional test for affinity with multiple cells https://review.openstack.org/599840
16:54:27 openstackgerrit Matt Riedemann proposed openstack/nova master: Delete NeutronLinuxBridgeInterfaceDriver https://review.openstack.org/616995
17:03:23 cfriesen mriedem: regarding the disk sector size issue...are you aware of any 8K sector disks or did you suggest it for future expansion?
17:03:35 stephenfin cfriesen: Based on that as-is, it would not
17:04:02 stephenfin cfriesen: Though I'd have expected to see an exception raised by the client, more so than anything else
17:06:04 mriedem cfriesen: was just suggesting based on what was noted in the bug report
17:07:15 sean-k-mooney im not aware of any 8k sector discs but i belive we can also diskcover the sector size by querying the disk via sysfs so we proably dont need to hardcode it
17:07:30 sean-k-mooney that said 4k and 512 are the most common
17:07:56 cfriesen I thought that 4K was still the highest supported physical sector size (since you'd want to be able to read a whole disk sector into a memory page)
17:08:07 dansmith not everyone uses 4k pages :)
17:08:26 sean-k-mooney power pc i think is 16k
17:08:30 dansmith I thought there were some SAN types that used larger sector sizes just because of the network optimization,
17:08:36 dansmith even if not backed by actual 8k
17:09:01 cfriesen as far as I know the block size can be different from the sector size
17:09:04 dansmith also, netapp I think uses some super odd sizes, even to the point of having weirdly low-level-formatted drives for them
17:09:38 sean-k-mooney i know some raid controls can be configured to exposed larger sector sizes but i dont know how common that is anymore
17:09:53 dansmith yep
17:10:41 sean-k-mooney mriedem: i think one of the things you suggested was just making a config option correct
17:11:34 sean-k-mooney something like directio_sector_sizes=512,4096
17:13:32 dansmith ah, but it's hidden to the LUN: https://kb.netapp.com/app/answers/answer_view/a_id/1001353/~/how-can-the-bytes%2Fsector-be-changed-in-a-luns-geometry%3F-
17:14:46 cfriesen the goal here is to figure out if the filesystem supports O_DIRECT. according to the man page, this should be set to the logical block size of the underlying storage, which can be determined using the ioctl() BLKSSZGET operation or by calling "blockdev --getss"
17:16:20 sean-k-mooney cfriesen: i think part of the issue is that on older kernel < 2.4 O_DIRECT required alinged access
17:16:42 sean-k-mooney but on bsd and newer linux kernel O_DIRECT did not reuqire alinged acess
17:17:13 sean-k-mooney cfriesen: im not actully sure of we need to do the alignment check we are doing anymore
17:17:30 openstackgerrit Merged openstack/nova stable/queens: fixtures: Track volume attachments within CinderFixtureNewAttachFlow https://review.openstack.org/612494
17:17:34 cfriesen the man page says: Under Linux 2.4, transfer sizes, and the alignment of the user buffer
17:17:34 cfriesen and the file offset must all be multiples of the logical block size
17:17:34 cfriesen of the filesystem. Since Linux 2.6.0, alignment to the logical block
17:17:34 cfriesen size of the underlying storage (typically 512 bytes) suffices.
17:17:40 cfriesen oh, ick. sorry
17:17:42 openstackgerrit Merged openstack/nova stable/queens: Add regression test for bug#1784353 https://review.openstack.org/612495

Earlier   Later