| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-03-16 | |||
| 11:48:33 | brinzhang_ | stephenfin: thanks | |
| 11:49:22 | brinzhang_ | thess feature all done to review, api change and novaclient change all done :) | |
| 12:03:59 | openstackgerrit | waleed mousa proposed openstack/os-vif master: [Follow Up] OVS DPDK port representors support https://review.opendev.org/705018 | |
| 12:07:32 | openstackgerrit | waleed mousa proposed openstack/os-vif master: [Follow Up] OVS DPDK port representors support https://review.opendev.org/705018 | |
| 12:09:32 | brinzhang_ | dansmith: I have added the nova and non-nova exception functional tests for instance action events fault details, pls see https://review.opendev.org/#/c/694430/7/nova/tests/functional/test_instance_actions.py | |
| 12:13:45 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/rocky: Clean up allocation if unshelve fails due to neutron https://review.opendev.org/713196 | |
| 12:13:45 | openstack | bug 1862633 in OpenStack Compute (nova) "unshelve leak allocation if update port fails" [Medium,Fix released] https://launchpad.net/bugs/1862633 - Assigned to Balazs Gibizer (balazs-gibizer) | |
| 12:13:45 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/rocky: Reproduce bug 1862633 https://review.opendev.org/713187 | |
| 12:38:52 | openstackgerrit | Lee Yarwood proposed openstack/nova master: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713035 | |
| 13:17:34 | openstackgerrit | John Garbutt proposed openstack/nova master: Make unified limits APIs return reserved of 0 https://review.opendev.org/712141 | |
| 13:17:34 | openstackgerrit | John Garbutt proposed openstack/nova master: Assert API behavior for noop quota driver https://review.opendev.org/712140 | |
| 13:17:35 | openstackgerrit | John Garbutt proposed openstack/nova master: Enforce api and db limits https://review.opendev.org/712142 | |
| 13:17:35 | openstackgerrit | John Garbutt proposed openstack/nova master: Add logic to enforce local api and db limits https://review.opendev.org/712139 | |
| 13:26:41 | dansmith | brinzhang_: okay, after coffee | |
| 13:27:08 | brinzhang_ | dansmith: good morning ^^ | |
| 14:09:26 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fix intermittently failing regression case https://review.opendev.org/713243 | |
| 14:15:22 | openstackgerrit | John Garbutt proposed openstack/nova master: Update quota_class APIs for db and api limits https://review.opendev.org/712143 | |
| 14:17:10 | sean-k-mooney | bauzas: would you have time to look at https://review.opendev.org/#/c/666914/21 and the follow up patches. it would be nice to be able to close that out. gibi maybe you could take a look to since efried_gone is nolonger here to review. since stephen an i are the author we need a non redhat person to review unless we are moving away form that requirement. alex_xu or johnthetubaguy would also work if | |
| 14:17:12 | sean-k-mooney | they are around. | |
| 14:20:29 | openstackgerrit | Lee Yarwood proposed openstack/nova master: DNM - Test TEMPEST_EXTEND_ATTACHED_ENCRYPTED_VOLUME https://review.opendev.org/707593 | |
| 14:26:36 | gibi | sean-k-mooney: ack. I cannot promise too much (have a long review queue atm) but added it to my queue | |
| 14:28:22 | gibi | sean-k-mooney: regarding the non-RH core requirement. I see this requirement as something that will be very problematic due to less diversity in the core team | |
| 14:29:59 | sean-k-mooney | gibi: ya. dont worry if you cant review but the trifect rule is likely to become a problem unless the diversity fo the core team can be restored. as someone who like that rule i would be sad to see it go but i guess we will see how it goes | |
| 14:32:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fix intermittently failing regression case https://review.opendev.org/713243 | |
| 14:33:16 | brinzhang_ | gmann: would like check os-instance-action policy again? https://review.opendev.org/#/c/706470/ | |
| 14:33:55 | gmann | brinzhang_: yeah, i will check after my internal meeting | |
| 14:35:22 | brinzhang_ | gmann: yeah, if it is ok, I think I can continue to do the os-instance-action (bp/action-event-fault-details) feature, there is a new policy besed on the default policy change | |
| 14:36:09 | brinzhang_ | gmann: thanks | |
| 14:37:52 | gmann | brinzhang_: yeah, you can continue on that, make dependency. | |
| 14:40:09 | brinzhang_ | gmann: Adding a default policy is only the remaining part, everything else is ready(TODO in it). So I want to do it after this done. | |
| 14:40:45 | openstackgerrit | Merged openstack/nova master: Cleanup test for system reader and reader_or_owner rules https://review.opendev.org/712515 | |
| 14:57:33 | openstackgerrit | Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494 | |
| 14:58:48 | bauzas | sean-k-mooney: I'll try but tbh, those next weeks will be crazy | |
| 14:59:01 | bauzas | but adding it to my queue | |
| 14:59:53 | sean-k-mooney | bauzas: yep i understand just trying to see if we can complete some blueprints that are close to being done. | |
| 15:00:00 | bauzas | sure | |
| 15:01:32 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add new default roles in os-instance-actions policies https://review.opendev.org/706470 | |
| 15:01:51 | kashyap | lyarwood: Hiya, hope the renewed comments make sense: forgot that we also need MIN_QEMU_BLOCKDEV | |
| 15:02:09 | kashyap | lyarwood: And a different, newer version for MIN_LIBVIRT_BLOCKDEV. (Notes in the review :)) | |
| 15:05:07 | artom | Remind me again what are the criteria for reporting a VM status as UNKNOWN? Host being down is one of them, right? | |
| 15:05:19 | artom | melwitt, ^^ if you're awake | |
| 15:05:21 | dansmith | yeah or cell down | |
| 15:05:29 | artom | dansmith, aha, ack, thanks! | |
| 15:05:42 | dansmith | it started at cell down only, but I think melwitt changed that recently | |
| 15:06:51 | artom | dansmith, that's the impression I got - not super important, just clearing out old (downstream) BZs | |
| 15:12:40 | spatel | sean-k-mooney: Good morning, if i want to disable Hyper Threading without BIOS setting how do i do that? I was reading AWS article here using offline cpu threads, do you think this is valid way to do that - https://aws.amazon.com/blogs/compute/disabling-intel-hyper-threading-technology-on-amazon-linux/ | |
| 15:14:22 | sean-k-mooney | spatel: yes you can do it via the sys virtual file system | |
| 15:14:54 | sean-k-mooney | the bios is the best way to do it as ofline cores will not unpartion the l1 cache | |
| 15:15:01 | spatel | sean-k-mooney: does that perform as good as BIOS setting ? | |
| 15:15:10 | sean-k-mooney | not quite | |
| 15:15:27 | sean-k-mooney | if you disable it in the bios it dobles the l1 cache avaiable to the core | |
| 15:15:58 | sean-k-mooney | when ht is enabled the l1 cache is partioned so the each ht has its onw region of the l1 cache | |
| 15:16:14 | spatel | We found erlang doing pretty good job when disable HT but again i don't want to do that setting from BIOS (its painful) i want to give that control to end users | |
| 15:16:41 | sean-k-mooney | but with it disable at the bios level all the l1 cache is avaiable to the physical core. at least on older intel architecutres | |
| 15:17:12 | sean-k-mooney | spatel: are you using cpu pinning | |
| 15:17:21 | spatel | Yes CPU pinning | |
| 15:17:26 | sean-k-mooney | if so you can use the cpu_thread_policy | |
| 15:17:47 | sean-k-mooney | create multiple flaovrs for the earlang instnace and let them choose | |
| 15:18:04 | spatel | sean-k-mooney: cpu_thread_policy=isolate ? | |
| 15:18:16 | sean-k-mooney | e.g. hw:cpu_thread_policy=prefer vs hw:cpu_threads_policy=isolate | |
| 15:18:20 | sean-k-mooney | yes | |
| 15:18:51 | spatel | sean-k-mooney: I have tried all kind of combination but erlang doesn't like it. | |
| 15:19:03 | sean-k-mooney | the sysfs performace delta is pretty small since normally your app/data wont fit in l1 anyway | |
| 15:19:03 | spatel | when i run VM on single NUMA perfrmance is really good | |
| 15:21:31 | spatel | sean-k-mooney: look at this - https://imgur.com/a/8zapZ8x | |
| 15:22:13 | spatel | To understand better CPU topology i am comparing them with AWS and here what i found | |
| 15:22:40 | spatel | On my openstack VM CPU topo looks very strange | |
| 15:23:01 | sean-k-mooney | the lower image looks like what i would expect | |
| 15:23:08 | spatel | l1d & l1i cache is shared | |
| 15:23:36 | spatel | why aws has l1d and l1i outside | |
| 15:23:53 | sean-k-mooney | i dont know | |
| 15:24:06 | sean-k-mooney | but you can alter this in the libvirt xml i think | |
| 15:24:27 | spatel | That is the problem, I have check with Alicloud and aws and both has perfect CPU topo but my openstack has very odd output | |
| 15:24:36 | sean-k-mooney | this is not something we would expose however | |
| 15:24:41 | spatel | I think it could be QEMU version or bug | |
| 15:25:07 | sean-k-mooney | well what do you mean by odd | |
| 15:25:08 | spatel | I am planning to upgrade my qemu to 4.2 (currently running 2.12) | |
| 15:25:19 | sean-k-mooney | the imgae you provide showing the kvm instance | |
| 15:25:25 | sean-k-mooney | look like real hardware would | |
| 15:25:28 | spatel | qemu-kvm | |
| 15:25:43 | spatel | both are virtual machine | |
| 15:26:22 | sean-k-mooney | sure but looking at https://imgur.com/a/8zapZ8x the bottom image looks corect the top look incorrect | |
| 15:27:06 | spatel | you are saying AWS instance looks incorrect? | |
| 15:27:13 | sean-k-mooney | yes | |
| 15:27:35 | sean-k-mooney | that is the toplogy that we should see if and only if you had HT disabled | |
| 15:27:36 | spatel | If i run same command on my host compute it looks exactly like AWS one | |
| 15:27:48 | spatel | sean-k-mooney: no | |
| 15:28:08 | spatel | Let me show you my two physical compute topo (HT vs non-HT) | |
| 15:28:10 | spatel | hold on.. | |
| 15:30:04 | sean-k-mooney | ok so looking locally they have changed how this work in later versions | |
| 15:30:37 | sean-k-mooney | the view that you see in openstack i how it used to work in nehalem and i belive up to sandybridge or ivybridge | |
| 15:32:05 | sean-k-mooney | spatel: as i said the bios seting used to change the toplogy between the aws one and the openstack one at the hardware level | |
| 15:32:20 | spatel | sean-k-mooney: https://imgur.com/a/at3WBBf | |
| 15:32:44 | spatel | This is my two compute host (one has HT enable and second has HT enable) | |
| 15:33:13 | sean-k-mooney | spatel: yep as i said this has changed with different hardware micorarchitecutres | |
| 15:33:30 | spatel | If you look bottom picture (its very similar to AWS virtual instance, that means AWS virtual machine correctly exposing physical topology including cache) | |
| 15:33:36 | sean-k-mooney | spatel: openstack/nova is not currently setting the cpu cache toplogy its decied by libvirt | |
| 15:34:02 | sean-k-mooney | spatel: openstack is not ment to expose the host toplogy by defualt | |
| 15:34:19 | spatel | even in host-passthrough ? | |
| 15:34:23 | sean-k-mooney | correct | |