| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2024-02-22 | |||
| 13:14:18 | sean-k-mooney | since it sa set since the a range | |
| 13:16:09 | sean-k-mooney | hum no the patch is correct which means its not populated? | |
| 13:16:47 | sean-k-mooney | https://github.com/openstack/nova/blob/3209f6551652cff7bef0b9d9719ab940dd05a0f8/nova/virt/libvirt/migration.py#L113-L117 | |
| 13:17:35 | sean-k-mooney | @artom i mostly aggre with this comment but i belive we dont alwasy set emulatorpin in the xml | |
| 13:17:52 | sean-k-mooney | https://github.com/openstack/nova/blob/3209f6551652cff7bef0b9d9719ab940dd05a0f8/nova/virt/libvirt/migration.py#L113-L126 | |
| 13:18:05 | artom | Yeah, I think I need an if there | |
| 13:18:15 | artom | It won't be set if there's no emulator thread policy | |
| 13:18:36 | sean-k-mooney | if emulatorpin is not found you need to default it to vcpupin | |
| 13:19:08 | sean-k-mooney | so ya you need an if here https://github.com/openstack/nova/blob/3209f6551652cff7bef0b9d9719ab940dd05a0f8/nova/virt/libvirt/migration.py#L127 | |
| 13:19:27 | sean-k-mooney | well | |
| 13:19:33 | sean-k-mooney | actully proably not there | |
| 13:19:45 | sean-k-mooney | but where we buidl teh dst_numa_info in the first place | |
| 13:19:59 | sean-k-mooney | well either works i guess | |
| 13:20:39 | sean-k-mooney | for backport reason we proably want to have _update_numa_xml have the fallback if its not set in dst_numa_info | |
| 13:20:52 | sean-k-mooney | but we should fix where we are creating the numa info to also set it properly | |
| 13:22:02 | sean-k-mooney | so add somethign like | |
| 13:23:18 | sean-k-mooney | emulator_pin = info.emulator_pins if info.emulator_pins else info.cpu_pins.values() | |
| 13:23:38 | sean-k-mooney | emulatorpin.set('cpuset',hardware.format_cpu_spec(emulator_pin)) | |
| 13:42:44 | opendevreview | Amit Uniyal proposed openstack/nova master: enforce remote console shutdown https://review.opendev.org/c/openstack/nova/+/901824 | |
| 14:53:39 | opendevreview | Pavlo Shchelokovskyy proposed openstack/nova master: Auto set heartbeat_in_pthread for wsgi services https://review.opendev.org/c/openstack/nova/+/909880 | |
| 15:31:48 | mnaser | It seems `nova-grenade-multinode` is broken in `stable/zed`. Does anyone know of anything that sticks out | |
| 15:32:13 | mnaser | https://review.opendev.org/c/openstack/nova/+/909098 we're eating up resources with folks rechecking it, we're at 3x, i think the job is deffo broken | |
| 15:33:13 | mnaser | ok, that's neutron being broken | |
| 15:33:19 | mnaser | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_39b/909098/2/check/nova-grenade-multinode/39bc664/controller/logs/screen-q-svc.txt | |
| 15:34:05 | frickler | iiuc grenade jobs should be dropped from zed now that yoga is unmaintained? | |
| 15:43:15 | dansmith | idk that we actually said that (removing grenade if the source is unmaintained) but that make sense to me | |
| 15:43:58 | clarkb | historically we've dropped grenade jobs for N+1 when N is no longer maintained | |
| 15:44:10 | clarkb | beacuse the grenade jobs start at N and upgrade to N+1 and you very quickly bitrot | |
| 15:46:01 | dansmith | you mean historically...removed right? I meant in response to the recent policy to have branches that are specifically _unmaintained_ but still around | |
| 15:46:11 | clarkb | correct historically | |
| 15:47:05 | dansmith | in the past, IIRC, we'd remove a grenade when it upgrades from a branch that gets deleted because we can't land a fix to make it installable, but it's slightly less clear when the branch exists but is in the newly-minted unmaintained state | |
| 15:47:25 | opendevreview | Artom Lifshitz proposed openstack/nova master: POC: power up dedicated cores during pre_live_migration https://review.opendev.org/c/openstack/nova/+/909806 | |
| 15:48:05 | artom | Let's see if the emulator_pins fix is enough | |
| 15:53:15 | frickler | those grenade runs seem also have been in a weird situation where the stable/yoga branches were already removed in the repos, but still present in our in-image-git-cache. but I also don't understand why n-cond seems to get started, but doesn't produce any log | |
| 15:57:34 | melwitt | I dunno if yall have seen but it looks like nova-next is at a 100% fail rate, POST_FAILURE | |
| 15:57:41 | melwitt | ForbiddenException: 403: Client Error for url: https://213.32.74.123:9696/networking/v2.0/ports/f5b273ab-1c98-44f6-9664-8c1a16d5abda, (rule:update_port and rule:update_port:binding:profile) is disallowed by policy | |
| 15:57:41 | melwitt | + openstack port unset --binding-profile allocation port-normal-qos | |
| 15:57:41 | melwitt | Deleting allocation key from the binding:profile of the bandwidth aware port | |
| 15:58:36 | melwitt | not sure what changed that could have affected policy. I'll look around | |
| 16:02:18 | sean-k-mooney | melwitt: i have a fix for it | |
| 16:02:40 | melwitt | ok thank goodness | |
| 16:02:43 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/909859 | |
| 16:02:50 | sean-k-mooney | i will readd the test coverage later | |
| 16:03:17 | sean-k-mooney | tldr modifying binding_profile now needd a service token | |
| 16:03:20 | sean-k-mooney | hence the 403 | |
| 16:03:25 | melwitt | ahh | |
| 16:14:39 | melwitt | sean-k-mooney: looks like there's at least one more thing that should be removed https://review.opendev.org/c/openstack/nova/+/909859/1/gate/post_test_hook.sh#174 | |
| 16:15:29 | sean-k-mooney | oh right | |
| 16:15:41 | sean-k-mooney | yep unset woudl also not work | |
| 16:16:11 | sean-k-mooney | im conflicted if we shoudl also remove that coverage | |
| 16:16:27 | sean-k-mooney | or if i shoudl bite the bullet and try and repalce it with a python script | |
| 16:18:08 | melwitt | yeah, I wondered that too. not sure how hard it would be to get something with service token working | |
| 16:18:27 | sean-k-mooney | the my-key stuff is not really that imporant | |
| 16:18:36 | sean-k-mooney | but the unset there is a key part of that test | |
| 16:18:52 | sean-k-mooney | melwitt: well we have nova avialabe in this env | |
| 16:19:06 | sean-k-mooney | so i was hoping we could just import some nova code for hat | |
| 16:19:16 | sean-k-mooney | this is running on the contoler | |
| 16:19:27 | sean-k-mooney | so the nova.conf should have all the setting set correctly | |
| 16:19:38 | sean-k-mooney | so in theory its improt the nvoa.conf module | |
| 16:19:40 | melwitt | yeah, it would | |
| 16:19:48 | sean-k-mooney | create the neutron clinet form our clinet module | |
| 16:19:52 | sean-k-mooney | and do port update | |
| 16:19:58 | melwitt | I should know bc I added it but 😛 | |
| 16:20:09 | melwitt | hm k | |
| 16:20:16 | bauzas | JayF: are you around ? | |
| 16:20:59 | JayF | What's up? | |
| 16:21:21 | bauzas | JayF: about https://review.opendev.org/c/openstack/nova/+/903915/ I wonder if you could provide a new revision due to sean-k-mooney's -1 ? | |
| 16:21:34 | bauzas | also, do we now have a Tempest job testing it ? | |
| 16:22:24 | JayF | Sean and I talked in depth about a testing plan earlier this week, and today and tomorrow is the time I set aside to implement it. At which point I'll re-stack all the sharding changes, make sure they pass CI, and do some manual testing | |
| 16:23:08 | JayF | We have implemented significant sharding testing in ironic for ensuring the API works properly, the second step I'm working on today is the scenario tests | |
| 16:23:09 | sean-k-mooney | cool just an fyi tomorrow is a "rechage day" at redhat meaning none of us will be here tomorow | |
| 16:23:14 | sean-k-mooney | but we can take a look on monday | |
| 16:23:35 | JayF | Yeah I think I have all the information I need based on our conversation earlier, I just need to not have my day stolen by a thousand tiny conversations | |
| 16:32:37 | melwitt | sean-k-mooney: the new_websocket_client method is what gets called when a new request to the proxy comes in, so I also dunno why that would need to be called to close a connection ... ? | |
| 16:37:17 | sean-k-mooney | auniyal: can you try https://review.opendev.org/c/openstack/nova/+/901824/18/nova/console/websocketproxy.py#154 | |
| 16:46:38 | auniyal | sean-k-mooney, ack thanks | |
| 16:48:51 | auniyal | sean-k-mooney, can we raise an Exception as invalid target token, inside | |
| 16:49:00 | auniyal | reason I am saying is https://github.com/openstack/nova/blob/master/nova/console/websocketproxy.py#L265 | |
| 16:50:59 | sean-k-mooney | that is coming from self.do_proxy | |
| 16:51:01 | sean-k-mooney | right | |
| 16:51:28 | auniyal | yes, so there some exception will be raised, like something happend in network or at client | |
| 16:52:00 | sean-k-mooney | so if we think about the reason why that might happen | |
| 16:52:22 | sean-k-mooney | the only reason i can think of is the client already disconnect before the time expired | |
| 16:52:43 | sean-k-mooney | in which case i dont think we should be raising an excption as that is an expected scenairo | |
| 16:53:06 | sean-k-mooney | so im not convinced raising an excption is correct | |
| 16:53:14 | auniyal | ack, | |
| 16:53:20 | sean-k-mooney | a debug level log woudl be fine | |
| 16:53:32 | sean-k-mooney | but do you have any other edge case in mind | |
| 16:53:51 | auniyal | okay, also if something really happend at client side https://review.opendev.org/c/openstack/nova/+/901824/18/nova/console/websocketproxy.py#294 | |
| 16:54:35 | auniyal | no, so as you said, everytime we come to close connection we will have valid tsock | |
| 16:55:03 | auniyal | ack, will remove if-else then, thanks | |
| 16:56:22 | sean-k-mooney | you have already protected agasint the socket being cloased with the OSError and the fileno being -1 | |
| 16:56:37 | sean-k-mooney | so i htink you have handled all the edgecase that are required there | |
| 16:59:06 | auniyal | ack, I'll just run tests locally and respin | |
| 16:59:55 | opendevreview | sean mooney proposed openstack/nova master: [S-RBAC] adapt nova-next for port's binding:profile field change https://review.opendev.org/c/openstack/nova/+/909859 | |
| 17:06:43 | sean-k-mooney | melwitt: by the way using blockcopy instead of blockrebase is a nice find | |
| 17:07:04 | sean-k-mooney | melwitt: it remiened me of something for future us | |
| 17:07:31 | sean-k-mooney | we currenly have some legacy code in hwo we do live snapshots to workaround some bugs with older version fo qemu/libvirt | |
| 17:07:42 | sean-k-mooney | my understanding is those issues have been fixed | |
| 17:07:57 | sean-k-mooney | so we may be able to simply that code alot in the future | |