Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-08-06
12:22:30 sean-k-mooney im talking about the directory
12:22:44 auniyal install -d --> that mean mkdir -p
12:22:44 sean-k-mooney not the files
12:23:05 sean-k-mooney right which uses the default umask for the current process for the permsions
12:23:12 sean-k-mooney im not sur eyou shoud be relying on that
12:23:25 sean-k-mooney you likel yshoudl follwo the other exampels where teh mode is alwasy explcitly set
12:24:21 sean-k-mooney auniyal: all other usage of `install` in that file has an explcity -m to set the mode
12:24:28 sean-k-mooney so you are deviating form the conventiosn in the file
12:24:57 sean-k-mooney you proably shoudl be doing install -d -m 755
12:25:06 sean-k-mooney here https://review.rdoproject.org/r/c/openstack/nova-distgit/+/58076/8/openstack-nova.spec#580
12:26:46 auniyal yeah above all setting permission to dir,
12:26:46 auniyal I only thought of creating it, then setting chmod to file
12:30:34 auniyal thanks Sean
12:31:33 mohsen__ sean-k-mooney: So how is it possible to have pCPUs with better performance than vCPUs, along with using networking and cinder-volume to create instances?
12:31:33 mohsen__ Note that once I changed the cpu_dedicated_set from one numa node to another, and the benchmarking result stayed the same.
12:31:52 sean-k-mooney mohsen__: by the way on an otherwise idel system you are unlikely to see many of the beniftis of pinning. pinning helps performance for 2 reasons., 1 it enfoces a explict numa toplogy so the guest memroy and cpus are allcoated form teh same numa ndoe, (that why you shoudl alwasy set hw:mem_page_size=<some valid value> i.e. small that give about 70% of the performace boost,
12:31:54 sean-k-mooney the rest comes form reducign cache misses and prevent other vms form consdume cpu time.
12:33:13 sean-k-mooney mohsen__: tha tis very stragne because in the perfomnance testing we did in the past it did impove performance. the main reasoin for CPU pinning is not raw trhoughput however, it benifit is in determinisum
12:33:37 sean-k-mooney it shoudl not reduce the performce vs vcpus becasue the emulation of the vcpu is identical
12:34:01 sean-k-mooney all cpu pinnign does is use cgroups/taskset to prevent the kernel form movinv the cpus
12:34:23 sean-k-mooney and on the nova side we prevent other vms form beign schduled to the same cores
12:34:38 sean-k-mooney its why we call it cpu_policy=dedicated
12:35:01 sean-k-mooney you care giveing an entire host cpu to the vm provide you are isolating other host process form those cores
12:35:45 sean-k-mooney regarding newtwokrign and cidner , if your using sriov you can allcoat vfs form the local numa node or you can use a cross numa bond for the stroage network
12:36:33 sean-k-mooney mohsen__: when you swap the numa nodes did you delete and recreate the vms by the way
12:48:43 opendevreview Rajesh Tailor proposed openstack/nova master: Add support for using cell-name in cell_v2 commands https://review.opendev.org/c/openstack/nova/+/954460
12:50:33 mohsen__ sean-k-mooney: Yes. I removed the old instances and after reconfiguration, created new ones. So you mean cpu performance improvement isn't necessarily the benefit of cpu pinning?
12:50:33 mohsen__ having such an expectation isn't always valid?
13:25:02 sean-k-mooney cpu pinning was create to provide predicable perfroamce under load
13:25:39 sean-k-mooney it often provides mroe performce as a side effect of enfoceign numa affintiy between guest cpus ram and pci devices
13:29:50 sean-k-mooney but most of the uplift comes enableing numa affintiy as cpu pinning implictly enables hw:numa_nodes=1
13:30:23 sean-k-mooney adding hw:numa_nodes=1 hw:mem_page_size=small
13:30:46 sean-k-mooney you will get most of the performace uplift that is expect form cpu pinnin for floating guests
14:09:46 opendevreview Ian Watson proposed openstack/nova master: Add migration conditional for resource tracker https://review.opendev.org/c/openstack/nova/+/955690
14:24:25 sean-k-mooney stephenfin: also interesting error in the logs https://paste.opendev.org/show/bxMqpRogUlo3jHCe2trb/ that likely related to your volume bug
14:24:46 sean-k-mooney we shoudl be able to catch that and abort starting the agent and let systemd or whatever restart it
14:24:53 stephenfin ooohh, good spot
14:25:01 sean-k-mooney https://0908e5c3034094a51a1a-ddb50bb0d8d4f5453b9288f160ef0a30.ssl.cf5.rackcdn.com/openstack/70be4003a0ae40b4bfaae667688f694c/controller/logs/screen-n-cpu.txt
14:25:14 sean-k-mooney its right in the startup just after the config is printed
14:25:18 sean-k-mooney so it pretyy early
14:25:31 sean-k-mooney but my guess is it never recovers form that if it happens
14:25:39 sean-k-mooney so either we need to abort or retry
14:26:10 sean-k-mooney well or something more indelgent then the 30s tought i have given it
14:26:32 sean-k-mooney can you see if you have the same error in your logs if you have time
14:28:49 sean-k-mooney stephenfin: https://tinyurl.com/yw5ynd7d
14:29:00 sean-k-mooney its happening alot in the usptream jobs
14:29:18 stephenfin yes, I see it in the flamingo VM. I don't see it in the bobcat VM
14:29:29 sean-k-mooney across c9s ubuntu and debian
14:29:31 stephenfin and I've since deleted all the other VMs
14:29:39 sean-k-mooney ack
14:29:43 sean-k-mooney so that likely the problem
14:29:59 sean-k-mooney i dont know what the cause iss but thats a smoking gun
14:30:49 sean-k-mooney i.e. i dont knwo why it does not happen after restarting nova-comptue but apprently it does not always fail
14:33:06 stephenfin sean-k-mooney: dumped it into the bug https://bugs.launchpad.net/nova/+bug/2119114/comments/3
14:33:24 sean-k-mooney cool i didnt have the link to that
15:12:55 noonedeadpunk ++
15:24:08 cardoe sean-k-mooney: so not sure if you saw my follow up on the vlan spec. The issue you pointed out in chat about the id field and that dansmith brought up on gerrit itself isn't something that spec can really fix right now. It's just defining the behavior of nova code today (and the behavior that its been since June 2015).
15:25:04 cardoe So I'm really not sure what to fix cause that spec isn't about changing the behavior of existing "phy" interfaces. It's about including "vlan" interfaces. The "vlan" interface references to "phy" interface its running on.
15:26:22 sean-k-mooney the spec si free to extend or modify the schema and change the data that is stored in specific fileds
15:26:40 sean-k-mooney it is not possibel for nova ot know what the name of the Phy interface will be
15:26:50 sean-k-mooney so the value that are provdie are and alwasy were informational
15:27:03 sean-k-mooney you are ment ot match on things like the mac adress
15:27:17 sean-k-mooney when you are consumign this to identify the phy not the id or name
15:27:29 sean-k-mooney its just an identifyer for the parsing program
15:29:26 abhishekk o/ can I get a review on https://review.opendev.org/c/openstack/nova/+/950623, it has one +2, this will be important change to get rid of show_multiple_locations config option
15:38:16 noonedeadpunk fwiw, I've addressed comments in https://review.opendev.org/c/openstack/nova-specs/+/955783 and I think implementation is also available for preliminary review
15:45:56 melwitt thanks sean-k-mooney
15:47:29 sean-k-mooney abhishekk: does it work this time? :) if i recall we reverted it because there was soem issue on the glance side that needed to be adressed first
15:47:57 sean-k-mooney abhishekk: ah i see dan is already +2 that reverting the revert
15:48:03 sean-k-mooney ill take a look shortly
15:48:46 sean-k-mooney we can always have a revert^3 patch if needed i guess.
16:48:49 abhishekk sean-k-mooney: thank you!!! I have added rechecks 7-8 times on revert patch to ensure that failure does not occur again
17:08:23 sean-k-mooney noonedeadpunk: oh the parallel mgtions
17:08:36 sean-k-mooney we really need to create teh 2026.1 folder in the specs repo
17:08:45 sean-k-mooney so you can propose that to the right pplace
17:09:59 sean-k-mooney noonedeadpunk: ill try and loop back to that but it likely will be next week at this rate
17:15:39 opendevreview Stephen Finucane proposed openstack/nova master: api: Add response body schemas for servers APIs (1/6) https://review.opendev.org/c/openstack/nova/+/956234
17:15:39 opendevreview Stephen Finucane proposed openstack/nova master: api: Add response body schemas for servers APIs (2/6) https://review.opendev.org/c/openstack/nova/+/956236
17:15:40 opendevreview Stephen Finucane proposed openstack/nova master: api: Add response body schemas for servers APIs (3/6) https://review.opendev.org/c/openstack/nova/+/956237
17:15:40 opendevreview Stephen Finucane proposed openstack/nova master: api: Add response body schemas for servers APIs (4/6) https://review.opendev.org/c/openstack/nova/+/956238
17:15:41 opendevreview Stephen Finucane proposed openstack/nova master: api: Add response body schemas for servers APIs (5/6) https://review.opendev.org/c/openstack/nova/+/956239
17:15:41 opendevreview Stephen Finucane proposed openstack/nova master: api: Add response body schemas for servers APIs (6/6) https://review.opendev.org/c/openstack/nova/+/956240
17:15:42 opendevreview Stephen Finucane proposed openstack/nova master: api: Add response body schemas for server shares APIs https://review.opendev.org/c/openstack/nova/+/956266
17:15:42 opendevreview Stephen Finucane proposed openstack/nova master: tests: Invert validation check https://review.opendev.org/c/openstack/nova/+/956241
#openstack-nova - 2025-08-07
05:51:06 noonedeadpunk sean-k-mooney: but it's not yet feature freeze for 2025.2 if I read the schedule right? and this is is quite tiny, easy, so maybe it's the right place? ;)
09:58:33 sean-k-mooney noonedeadpunk: we are well pass the spec approval freeze so this is something for next cycle
09:58:51 sean-k-mooney noonedeadpunk: the spec freeze was milestone 2
10:02:10 noonedeadpunk ah, right, true
10:02:17 noonedeadpunk I was looking for feature freeze
10:03:22 sean-k-mooney that does not mean it cant merge early in 2026.1 as in late september/october
10:03:26 noonedeadpunk it's kinda sad though, as this is pretty much the only way so far to get live migrations working reliable until gnutls is patched and with memory being encrypted
10:03:59 noonedeadpunk or well... use tunnels...
10:03:59 sean-k-mooney well its not actully fixing the issue
10:04:10 noonedeadpunk yeah, true
10:04:24 sean-k-mooney it just make it less likely as each coonenction gets it own rekey timer
10:04:39 sean-k-mooney the workaroudn for now is to just disable tls3 with aes
10:04:44 sean-k-mooney via the crypto polices
10:04:56 noonedeadpunk which you can;'t do outside of EL
10:04:58 sean-k-mooney i.e. disable the specific policy that has the issue
10:05:05 sean-k-mooney EL?

Earlier   Later