Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-04-29
14:35:19 sean-k-mooney infra are going to send a staus update for the same shortly
14:52:11 kashyap sean-k-mooney: Since you've reviewed an older version (PS-5) and a newer one (PS-7), I'll just address the PS-7 bits here: https://review.opendev.org/#/c/631154/
14:52:15 kashyap sean-k-mooney: That okay?
14:55:54 sean-k-mooney kashyap: am sure
14:56:25 sean-k-mooney i think i coppied most of the relevent bits although if you read both and just resond on 7 that is fine with me
14:56:42 sean-k-mooney or well update it in version 8
14:56:54 kashyap sean-k-mooney: While I comment in the spec, on the "increased memory usage" bit -- I knew that thing, but haven't explicitly mentioned it because it requires precise tests, in what scenarios, etc
14:57:06 kashyap You can't just put a generic: "in all cases memory is increased"
14:57:32 sean-k-mooney kashyap: not really even with 1 pci root port dan found it used more memeory the pc
14:57:34 kashyap It requires more testing; so somebody ought to do the "performance testing guy's job"...
14:57:45 sean-k-mooney so i think in all configurtion it has more memory overhead
14:58:06 kashyap sean-k-mooney: Right; I'll mention that, but need to carefully write it in context and with a config example
14:58:42 sean-k-mooney or we can jsut say we expect that q35 will use more memroy as we have never seen a case where it uses less
14:58:44 kashyap sean-k-mooney: Also I don't want us to get sucked into that black-hole and get derailed...
14:58:56 kashyap But that begs the question: "how much more memory than before"
14:59:13 sean-k-mooney sure an that we can leave to the operator
14:59:14 kashyap Which requires clear a example benchmark
14:59:18 sean-k-mooney or perfomance guy
14:59:42 kashyap Okido; I actually first mentioned it locally and then removed it, as I was still thinking of it
15:00:37 sean-k-mooney i mainly just want them to be aware that they should consider it when upgrading so they can factor it in to there host memory reservation and capastity planning
15:01:55 kashyap Yeah, definitely. Thx for the taking time respond.
15:35:28 spatel sean-k-mooney: do you know how much CPU would be enough to reserve for hypervisor? using isolcpus option?
15:45:11 sean-k-mooney i dont adviase using isolcpus
15:46:11 sean-k-mooney generally 1 phsycial core is more then enough on a compute node or 1 per numa nodes if you want to do affintiy of interupts
15:47:59 spatel sean-k-mooney: but mostly for NFV they suggest using isolcpus for isolation
15:48:15 sean-k-mooney spatel: i generally recommend you use the vcpu_pin_set or in train+ cpu_dedicate_set and cpu_shared_set to do the reservation
15:48:34 spatel you are saying 2 cpu core would be more than enough for hypervisor (per NUMA)?
15:48:40 sean-k-mooney spatel: you should only use isolcpus on realtime hosts and only on the cpus used for pinned vms
15:49:32 spatel I am doing all cpu pinning (dedicated) option for all my workload
15:49:55 spatel we need performance not quantity.
15:51:14 spatel I am planning to use isolcpus + vcpu_pin_set (both option to allocate dedicated CPU)
15:55:21 openstackgerrit Jiri Suchomel proposed openstack/nova master: Add ability to download Glance images into the libvirt image cache via RBD https://review.opendev.org/574301
15:55:24 spatel sean-k-mooney: ^^
15:56:13 sean-k-mooney 1 phsyical core(2 hyperthreads) is normally enouch for a compute node if you are not using heavy telemetry
15:56:40 sean-k-mooney spatel: you can use isolcpus + vcpu_pin_set but only if the vm is pinned
15:57:52 sean-k-mooney when you use isolcpus it disables the linux kernel shcduler for those cores
15:58:05 sean-k-mooney so if you have floating vms then they wont float
15:58:30 spatel sean-k-mooney: sounds good, yes we do pinned VM (currently i have assigned 8 cores but wanted to see what people mostly recommend )
15:58:32 sean-k-mooney in generally isolcpus is only a good idea if you are running realtime wrokloads
15:58:59 sean-k-mooney spatel: you might want to look into tuned by the way
15:59:11 sean-k-mooney it supports configuring this via userspace/sysfs
15:59:25 spatel tuned profile?
15:59:39 sean-k-mooney https://github.com/redhat-performance/tuned/tree/master/profiles/cpu-partitioning
16:00:02 sean-k-mooney isolcpus is a deprecated kernel argument
16:00:04 sean-k-mooney https://github.com/redhat-performance/tuned/blob/master/profiles/cpu-partitioning/cpu-partitioning-variables.conf
16:00:25 melwitt gmann: I'm trying to understand a bit about how/why two grenade jobs would run on stable/ussuri (we don't have an example yet). and I looked at the tempest change and realized I don't understand why it ran two grenade jobs on the name change patch https://review.opendev.org/722551 could you please explain why two jobs run on openstack/tempest? I thought it would have been only one
16:01:01 sean-k-mooney tuned uses the the sysfs/cgroups interface to achive the same effect without the drawbacks
16:01:31 spatel sean-k-mooney: ohh good to know :)
16:01:37 spatel will look into that
16:01:54 sean-k-mooney normally i would jsut set isolated_cores=2,4-7 and not set no_balance_cores=5-10
16:02:09 sean-k-mooney although no_balance_cores=5-10 would be useful for realtime hosts or ovs-dpdk
16:02:52 spatel Do i need to restart machine to set this values ?
16:03:54 gmann melwitt: sure. for nova stable/ussuri, it will be both job running if you recheck any ussuri backport (or testing patch) until 724189 is merged. this is because compute template in Tempest switched to new job (https://review.opendev.org/#/c/722551/3/.zuul.yaml@543) and nova stable/ussuri ./.zuul.yaml have old job also listed for irrelevant file
16:04:01 sean-k-mooney spatel: am i dont think so
16:04:16 sean-k-mooney you would on teh kernel command line but not with tuned
16:04:23 gmann melwitt: https://github.com/openstack/nova/blob/stable/ussuri/.zuul.yaml#L400
16:04:39 spatel sean-k-mooney: yes kernel does require reboot but lets me test in tuned
16:05:14 melwitt gmann: sorry I mean as an aside, why did two jobs run on https://review.opendev.org/722551 ? I realized I didn't understand that
16:05:43 gmann melwitt: on Tempest side it is still running because Tempest gate run 'integrated-gate-py3' template which is running all service tests. and that template is on openstack-zuul-jobs side so once i update that template then Tempest also will have single new job
16:06:00 melwitt thanks
16:06:54 gmann compute and service specific template are taken care by 722551 but integrated-gate-py3 template is not yet.
16:07:59 gmann best things we did in grenade side is we alias the grenade-py3 to new zuulv3 native job to avoid running legacy + new jobs during this migration. It is same zuulv3 jobs running twice with different name so will not cause issue.
17:09:53 openstackgerrit Stephen Finucane proposed openstack/nova master: WIP: objects: Add migrate-on-load behavior for legacy NUMA objects https://review.opendev.org/724381
17:10:18 stephenfin dansmith: That's not complete, but when you've a chance can you sanity check to see if that's what you're after? ^
17:10:57 openstackgerrit sean mooney proposed openstack/nova master: silence amqp heartbeat warning https://review.opendev.org/724188
17:12:54 artom That just looks like stephenfin harpooned a dude coming at him
17:14:53 sean-k-mooney hehe you dont know what stephenfin gets up to on his runs
17:15:01 sean-k-mooney its one way to keep otheres away
17:15:27 sean-k-mooney i mean would you approch a person running with a harpoon
17:16:40 artom Depends
17:16:45 artom Am I a masochistic whale?
17:42:08 openstackgerrit sean mooney proposed openstack/nova master: [WIP] add workaround to disable multiple port bindings https://review.opendev.org/724386
17:42:08 openstackgerrit sean mooney proposed openstack/nova master: [DNM] testing with force_legacy_port_binding workaround https://review.opendev.org/724387
17:49:50 sean-k-mooney by the way how do people feel about ^ as a temproy workaround for this long standing issue while we figure out how to fully fix this
17:51:28 sean-k-mooney i think my other patches are still the wright approch https://review.opendev.org/#/c/602432/ and https://review.opendev.org/#/c/640258 but i have not looked at this in a year and was not planning to but i guess i can try to find time to look at this again
20:43:15 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add nova-status upgrade check and reno for policy new defaults https://review.opendev.org/723645
20:55:47 gmann gibi: i updated the upgrade check with more safer way to cover all cases how policy is initialized or not. I have added test also for that and waiting for grenade job result which run upgrade check. also I am preparing new devstack but some issue, please let me know if it run fine now on your env- https://review.opendev.org/723645
20:56:26 gmann i mean with the policy file you showed in http://paste.openstack.org/show/792881/
21:03:43 JamesBenson All: Does anyone know if Nova supports the Tesla M2050 or M2070 for vGPU?
21:08:16 melwitt JamesBenson: I don't know but here's the doc we have if you didn't already find https://docs.openstack.org/nova/latest/admin/virtual-gpu.html
21:09:41 JamesBenson melwitt: Yes, thanks, I saw that resource too. I have some older m1000e's that I got and just not sure if I should bother installing openstack or just have them as one off's for testing code.
21:10:22 melwitt ok. bauzas would know but he's off by now today. sean-k-mooney might know ^
21:48:26 openstackgerrit melanie witt proposed openstack/nova-specs master: Re-propose nova-audit spec for Victoria https://review.opendev.org/724430
22:37:51 melwitt gmann: heya, would you mind looking over this review? it looks sane to me but could use your api validation expertise https://review.opendev.org/407514
22:38:49 melwitt I'm not clear on whether it could possibly cause any backward compat issues. it means to only target the 500 error casd
22:38:51 melwitt *case
#openstack-nova - 2020-04-30
00:11:51 gmann melwitt: ack. I will check after finishing the testing of policy upgrade check.
00:17:57 melwitt awesome thanks
00:49:30 gmann gibi: bauzas : uploaded the upgrade check result of fresh devstack env on patch and those are as expected. tested 3 cases 1. policy file in json format with new defaults only bug scenario http://paste.openstack.org/show/792909/ 2. policy file with both defaults present http://paste.openstack.org/show/792910/ 3. policy file generated in yaml format - http://paste.openstack.org/show/792911/
01:05:42 gmann case4: no policy file - http://paste.openstack.org/raw/792913/
02:02:41 openstackgerrit xuyuanhao proposed openstack/nova master: the vms can not be force deleted when vm_status is soft-delete and task-state=deleting https://review.opendev.org/724260
03:12:28 gmann melwitt: tested locally and it lgtm from backword compatibility point of view and without microversion. but it is fixing more APIs than listed in commit msg so need test coverage for those.
03:52:18 melwitt gotcha thanks gmann
07:12:16 gibi good morning
07:13:44 PrinzElvis hi all
07:15:34 PrinzElvis https://www.irccloud.com/pastebin/KYjd5FUo/Rabbitmq_error_from_rocky_to_stein
07:29:55 gibi PrinzElvis: the only thing seems a bit strange to me is "{% endfor %}/{{ om_rpc_vhost }}" as that will be expanded to "//" as om_rpc_vhost is "/"
07:30:43 gibi PrinzElvis: but I'm no way expert on the rabbit config
07:44:53 PrinzElvis gibi: Ok I removed this "//" in my "transport_url = rabbit://openstack:2fe0525ec5db4f1b97d4d055527501ec@10.10.1.11:5672,openstack:2fe0525ec5db4f1b97d4d055527501ec@10.10.1.12:5672,openstack:2fe0525ec5db4f1b97d4d055527501ec@10.10.1.13:5672// "
07:45:33 nightmare_unreal artom: can you have another look at : https://review.opendev.org/#/c/715395/ . Thanks . Also i don't understand what can cause nova-live-migration zuul job to fail ? what does it even check ?

Earlier   Later