Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-03-09
19:28:36 sean-k-mooney if you dont add locked
19:28:48 artom Yeah, I agree there's precedent
19:29:11 artom (In terms for breaking unless the user does something they strictly-speaking should not have to do)
19:29:23 artom But... doesn't mean we shouldn't strive to improve on that :)
19:30:01 sean-k-mooney well again we did not wnat reqouce usage to change basked on port type
19:30:15 sean-k-mooney that is why you were required to enable hugepages in the flavor or image
19:30:43 sean-k-mooney there are othere issue this create for attach too
19:31:15 artom Well you clearly shouldn't be allowed to attach ports that require locked memory to a running instance :)
19:31:37 artom (Unless it already has locked memory)
19:32:23 sean-k-mooney yep
19:32:30 artom Which we would need to track, and it could come from not only the extra spec, but also other source that caused libvirt to do it for us, etc etc
19:32:33 sean-k-mooney but not just running
19:32:34 artom Hence: spec discussion :)
19:32:43 sean-k-mooney it would be incorrect to change it on hard reboot
19:32:52 sean-k-mooney or iff it was off
19:33:06 sean-k-mooney e.g. you cant add to any instance already on the host
19:33:13 sean-k-mooney as it change the memory usage of the guest
19:33:30 sean-k-mooney same way addign a vhost-user port shoudl not make the guest suddenly use hugepages
19:57:00 openstackgerrit sean mooney proposed openstack/nova master: Support per port numa policies with SR-IOV https://review.opendev.org/c/openstack/nova/+/773792
20:05:02 lyarwood sean-k-mooney: sorry had to go afk, so devstack is still running somehow. Is your host pretty loaded at the moment?
20:06:59 sean-k-mooney no load is at 4.38 on the host
20:07:10 sean-k-mooney it has 24cores/48threads
20:07:13 lyarwood weird
20:07:18 sean-k-mooney it might be ipv6 being slow
20:07:25 lyarwood I'm using the medium flavor with 12 vcpus
20:07:48 sean-k-mooney i dont have native ipv6 and the tunnel is sometimes slow
20:07:53 lyarwood I'm not using the async stuff but still, it's been running for over an hour now
20:08:00 lyarwood kk
20:08:17 sean-k-mooney ya that weird what does the load look like in the vm
20:09:01 sean-k-mooney io wait is also not that high in the host %0.90
20:10:45 lyarwood kk let me try again with DEVSTACK_PARALLEL=true
20:11:08 sean-k-mooney it should only take about 20 mings without it
20:12:45 lyarwood takes about 6 on a local f32 vm
20:12:54 lyarwood kk, it's looking quicker tbh
20:19:03 sean-k-mooney ya something is wrong with my ipv6 routing http://paste.openstack.org/show/803402/
20:19:15 sean-k-mooney dns is working but i cant ping
20:19:28 sean-k-mooney thats form the server that is hosting the vm
20:30:08 openstackgerrit Merged openstack/nova master: pci manager: replace node_id parameter with compute_node https://review.opendev.org/c/openstack/nova/+/778747
20:30:42 openstackgerrit Merged openstack/nova master: apidb: Compact Queens database migrations https://review.opendev.org/c/openstack/nova/+/759404
20:31:08 sean-k-mooney lyarwood: problems on my routeer the tunnel is not working properly. for now if you do
20:31:11 sean-k-mooney sysctl -w net.ipv6.conf.all.disable_ipv6=1
20:31:16 sean-k-mooney sysctl -w net.ipv6.conf.default.disable_ipv6=1
20:31:20 sean-k-mooney it will work around the issue
20:31:33 sean-k-mooney with sudo of course
20:31:50 sean-k-mooney it shoudl be falling back to ipv4 anyway but that can slow down installs
20:32:20 sean-k-mooney this is one of the reason im going to be reinstallin my cloud in a month or too
20:32:29 openstackgerrit Merged openstack/nova master: pci: track host NUMA topology in stats https://review.opendev.org/c/openstack/nova/+/774149
20:32:40 sean-k-mooney ipv6 is nice but somethiem it has issue so untill i have it nativly im going to remove it
20:32:59 sean-k-mooney am what? ^
20:34:54 sean-k-mooney stephenfin: gibi the numa node is already in the pci_stats in the compute node table
20:35:11 sean-k-mooney as is the host numa toplogy blob
20:36:34 sean-k-mooney {"product_id": "101e", "vendor_id": "15b3", "numa_node": 0, "tags": {"dev_type": "vdpa", "physical_network": null, "parent_ifname": "enp6s0f0_0"}, "count": 3}
20:39:53 sean-k-mooney artom: i really wish you asked me to review that
20:40:14 openstackgerrit Artom Lifshitz proposed openstack/nova master: fakelibvirt: make kB_mem default not laughable https://review.opendev.org/c/openstack/nova/+/779559
20:40:16 sean-k-mooney these were class method because we did not want to store this data in an object
20:40:34 artom sean-k-mooney, the NUMA node yes, but not the socket
20:41:06 sean-k-mooney right but the socket you were goint to store/lookup seperately
20:41:27 artom sean-k-mooney, I save the host numa_topology in the pci stats object
20:41:34 artom (I forget what it's called exactly)
20:41:37 sean-k-mooney im reviewing https://review.opendev.org/c/openstack/nova/+/774149 now to fiture out what you cahgned but this is going to conflcit with most of my patches
20:42:24 openstackgerrit Merged openstack/nova master: conf: Clean up docs for scheduler options https://review.opendev.org/c/openstack/nova/+/773639
20:42:57 artom Storing it (the numa_topology) in the object was cleaner than passing it through
20:43:24 artom Only the socket policy filter() method needs it, but it'd have been passed through like 42 other methods to get there
20:43:27 sean-k-mooney but those function were ment to be pur fucntion of there input
20:43:45 artom They still are
20:43:50 artom They're just in the object now
20:44:01 sean-k-mooney but do they use any data via self
20:44:04 sean-k-mooney if so they are not
20:44:08 artom Nope
20:44:17 artom Only the new filter for the socket policy
20:44:18 sean-k-mooney then they can stay class methods
20:44:26 sean-k-mooney you can call class methods via self
20:44:36 artom No, because they *call* the new socket filter method
20:44:42 artom Which needs to be on self, not cls
20:44:56 artom Because of the aforementioned requirement for numa_topology
20:45:10 artom You can call cls from self, but not the other way around
20:46:01 sean-k-mooney sure but you could pass in the numa object
20:46:42 artom Yeah, it's what I was saying
20:46:43 sean-k-mooney where are you usin ghtis the next patch
20:46:50 artom Yeah, next patch
20:47:04 artom It's what I was saying - storing self.numa_topology was much cleaner than passing it rhough
20:47:06 sean-k-mooney i would not have done what you have in the one that just merged
20:47:06 artom *through
20:47:22 sean-k-mooney ya but it break the design of the class
20:47:31 sean-k-mooney and that make me uncofrotable
20:47:46 sean-k-mooney we intentionally did not do what you are not doing
20:48:40 sean-k-mooney so ya you are now using self here https://review.opendev.org/c/openstack/nova/+/772779/17/nova/pci/stats.py#336
20:50:05 artom I figured there was a reason, on the other hand, there was also a thing where node_id was optional which looks like it was only for testing
20:50:23 artom So it was hard to tell what was legit reason and what was programmer laziness ;)
20:50:27 melwitt sean-k-mooney: curious how you get ~5 devstack times? do you disable certain services or something?
20:51:05 sean-k-mooney i get time of about 15 mins
20:51:24 sean-k-mooney artom: the numa node is optional
20:51:27 sean-k-mooney not all devices have one
20:51:50 artom sean-k-mooney, not the numa node, the compute node_id
20:52:03 sean-k-mooney oh thats required
20:52:12 artom It didn't use to be
20:52:13 sean-k-mooney where is that optional
20:52:24 melwitt hm ok. when I do it with DEVSTACK_PARALLEL=1 it takes 25-30 minutes but that is with default things enabled, nearly empty local.conf
20:52:36 melwitt I was wondering how people are getting these really fast times
20:52:43 artom sean-k-mooney, https://review.opendev.org/c/openstack/nova/+/778747/2/nova/pci/manager.py

Earlier   Later