Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-03-09
19:57:00 openstackgerrit sean mooney proposed openstack/nova master: Support per port numa policies with SR-IOV https://review.opendev.org/c/openstack/nova/+/773792
20:05:02 lyarwood sean-k-mooney: sorry had to go afk, so devstack is still running somehow. Is your host pretty loaded at the moment?
20:06:59 sean-k-mooney no load is at 4.38 on the host
20:07:10 sean-k-mooney it has 24cores/48threads
20:07:13 lyarwood weird
20:07:18 sean-k-mooney it might be ipv6 being slow
20:07:25 lyarwood I'm using the medium flavor with 12 vcpus
20:07:48 sean-k-mooney i dont have native ipv6 and the tunnel is sometimes slow
20:07:53 lyarwood I'm not using the async stuff but still, it's been running for over an hour now
20:08:00 lyarwood kk
20:08:17 sean-k-mooney ya that weird what does the load look like in the vm
20:09:01 sean-k-mooney io wait is also not that high in the host %0.90
20:10:45 lyarwood kk let me try again with DEVSTACK_PARALLEL=true
20:11:08 sean-k-mooney it should only take about 20 mings without it
20:12:45 lyarwood takes about 6 on a local f32 vm
20:12:54 lyarwood kk, it's looking quicker tbh
20:19:03 sean-k-mooney ya something is wrong with my ipv6 routing http://paste.openstack.org/show/803402/
20:19:15 sean-k-mooney dns is working but i cant ping
20:19:28 sean-k-mooney thats form the server that is hosting the vm
20:30:08 openstackgerrit Merged openstack/nova master: pci manager: replace node_id parameter with compute_node https://review.opendev.org/c/openstack/nova/+/778747
20:30:42 openstackgerrit Merged openstack/nova master: apidb: Compact Queens database migrations https://review.opendev.org/c/openstack/nova/+/759404
20:31:08 sean-k-mooney lyarwood: problems on my routeer the tunnel is not working properly. for now if you do
20:31:11 sean-k-mooney sysctl -w net.ipv6.conf.all.disable_ipv6=1
20:31:16 sean-k-mooney sysctl -w net.ipv6.conf.default.disable_ipv6=1
20:31:20 sean-k-mooney it will work around the issue
20:31:33 sean-k-mooney with sudo of course
20:31:50 sean-k-mooney it shoudl be falling back to ipv4 anyway but that can slow down installs
20:32:20 sean-k-mooney this is one of the reason im going to be reinstallin my cloud in a month or too
20:32:29 openstackgerrit Merged openstack/nova master: pci: track host NUMA topology in stats https://review.opendev.org/c/openstack/nova/+/774149
20:32:40 sean-k-mooney ipv6 is nice but somethiem it has issue so untill i have it nativly im going to remove it
20:32:59 sean-k-mooney am what? ^
20:34:54 sean-k-mooney stephenfin: gibi the numa node is already in the pci_stats in the compute node table
20:35:11 sean-k-mooney as is the host numa toplogy blob
20:36:34 sean-k-mooney {"product_id": "101e", "vendor_id": "15b3", "numa_node": 0, "tags": {"dev_type": "vdpa", "physical_network": null, "parent_ifname": "enp6s0f0_0"}, "count": 3}
20:39:53 sean-k-mooney artom: i really wish you asked me to review that
20:40:14 openstackgerrit Artom Lifshitz proposed openstack/nova master: fakelibvirt: make kB_mem default not laughable https://review.opendev.org/c/openstack/nova/+/779559
20:40:16 sean-k-mooney these were class method because we did not want to store this data in an object
20:40:34 artom sean-k-mooney, the NUMA node yes, but not the socket
20:41:06 sean-k-mooney right but the socket you were goint to store/lookup seperately
20:41:27 artom sean-k-mooney, I save the host numa_topology in the pci stats object
20:41:34 artom (I forget what it's called exactly)
20:41:37 sean-k-mooney im reviewing https://review.opendev.org/c/openstack/nova/+/774149 now to fiture out what you cahgned but this is going to conflcit with most of my patches
20:42:24 openstackgerrit Merged openstack/nova master: conf: Clean up docs for scheduler options https://review.opendev.org/c/openstack/nova/+/773639
20:42:57 artom Storing it (the numa_topology) in the object was cleaner than passing it through
20:43:24 artom Only the socket policy filter() method needs it, but it'd have been passed through like 42 other methods to get there
20:43:27 sean-k-mooney but those function were ment to be pur fucntion of there input
20:43:45 artom They still are
20:43:50 artom They're just in the object now
20:44:01 sean-k-mooney but do they use any data via self
20:44:04 sean-k-mooney if so they are not
20:44:08 artom Nope
20:44:17 artom Only the new filter for the socket policy
20:44:18 sean-k-mooney then they can stay class methods
20:44:26 sean-k-mooney you can call class methods via self
20:44:36 artom No, because they *call* the new socket filter method
20:44:42 artom Which needs to be on self, not cls
20:44:56 artom Because of the aforementioned requirement for numa_topology
20:45:10 artom You can call cls from self, but not the other way around
20:46:01 sean-k-mooney sure but you could pass in the numa object
20:46:42 artom Yeah, it's what I was saying
20:46:43 sean-k-mooney where are you usin ghtis the next patch
20:46:50 artom Yeah, next patch
20:47:04 artom It's what I was saying - storing self.numa_topology was much cleaner than passing it rhough
20:47:06 artom *through
20:47:06 sean-k-mooney i would not have done what you have in the one that just merged
20:47:22 sean-k-mooney ya but it break the design of the class
20:47:31 sean-k-mooney and that make me uncofrotable
20:47:46 sean-k-mooney we intentionally did not do what you are not doing
20:48:40 sean-k-mooney so ya you are now using self here https://review.opendev.org/c/openstack/nova/+/772779/17/nova/pci/stats.py#336
20:50:05 artom I figured there was a reason, on the other hand, there was also a thing where node_id was optional which looks like it was only for testing
20:50:23 artom So it was hard to tell what was legit reason and what was programmer laziness ;)
20:50:27 melwitt sean-k-mooney: curious how you get ~5 devstack times? do you disable certain services or something?
20:51:05 sean-k-mooney i get time of about 15 mins
20:51:24 sean-k-mooney artom: the numa node is optional
20:51:27 sean-k-mooney not all devices have one
20:51:50 artom sean-k-mooney, not the numa node, the compute node_id
20:52:03 sean-k-mooney oh thats required
20:52:12 artom It didn't use to be
20:52:13 sean-k-mooney where is that optional
20:52:24 melwitt hm ok. when I do it with DEVSTACK_PARALLEL=1 it takes 25-30 minutes but that is with default things enabled, nearly empty local.conf
20:52:36 melwitt I was wondering how people are getting these really fast times
20:52:43 artom sean-k-mooney, https://review.opendev.org/c/openstack/nova/+/778747/2/nova/pci/manager.py
20:52:58 sean-k-mooney melwitt: it has to be DEVSTACK_PARALLEL=True
20:53:21 sean-k-mooney unless they fixed that 1 will not work
20:53:23 melwitt sean-k-mooney: I used it, it gave me the async time report at the end
20:53:31 melwitt maybe I used True, I don't know for sure
20:53:36 sean-k-mooney this is also my default contoler http://paste.openstack.org/show/803405/
20:53:52 sean-k-mooney i have some service disabled yes
20:53:59 melwitt thanks
20:54:21 sean-k-mooney you could disable tempest and horizon form that list if you anted
20:54:31 sean-k-mooney or cinder i guess
20:54:43 sean-k-mooney but that a faily simple compute cloud
20:55:02 sean-k-mooney baisclaly no heat or swift form the default set
20:56:23 sean-k-mooney artom: i think that was needed
20:56:36 artom sean-k-mooney, only for testing as far as I could find
20:56:38 sean-k-mooney artom: we initalise the pci tracker before the comptue service is registered
20:56:52 artom That's the only place where it was ever not passed in
20:56:53 sean-k-mooney artom: no i think its need for a fresh install
20:57:02 artom sean-k-mooney, wouldn't that then explode in testing?
20:57:04 sean-k-mooney its not that is passed i think its set

Earlier   Later