| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-22 | |||
| 14:40:28 | openstackgerrit | sean mooney proposed openstack/os-vif master: move os-vif-ovs to be a non legacy job. https://review.opendev.org/701601 | |
| 14:40:29 | openstackgerrit | sean mooney proposed openstack/os-vif master: Revert "[Follow Up] OVS DPDK port representors support" https://review.opendev.org/703672 | |
| 15:04:27 | efried | sean-k-mooney, gmann: So is the PyYAML thing in n-l-m sorted yet or still busted? | |
| 15:06:26 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: Block rebuild when NUMA topology changed https://review.opendev.org/702972 | |
| 15:06:26 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: Remove 'test_cold_migrate_with_physnet_fails' test https://review.opendev.org/702971 | |
| 15:06:27 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: FUP for in-place numa rebuild https://review.opendev.org/702974 | |
| 15:06:27 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: Disable NUMATopologyFilter on rebuild https://review.opendev.org/702973 | |
| 15:06:56 | frickler | efried: busted, need a fix in devstack and ds-gate each | |
| 15:07:00 | sean-k-mooney | efried: its breaking os-vifs 1 remaining legacy job too | |
| 15:07:25 | sean-k-mooney | frickler: i assume ds-gate is installing os-testr or something like that | |
| 15:08:33 | sean-k-mooney | the non legacy versions seem to be fine but its the installation of os-testr that is trigering PyYAML to be reinstalled | |
| 15:09:48 | efried | I can't tell if https://review.opendev.org/#/c/561597/ will fix it temporarily until https://review.opendev.org/#/c/703735/ can be landed. | |
| 15:10:20 | efried | frickler: ^ ? | |
| 15:10:48 | frickler | efried: the former isn't the fix, but the cause of the new failures | |
| 15:11:06 | efried | oh? I was seeing them before that patch landed | |
| 15:11:21 | efried | I think | |
| 15:11:28 | frickler | fixes are https://review.opendev.org/703792 and https://review.opendev.org/#/c/703271/4 | |
| 15:12:39 | frickler | oh https://review.opendev.org/#/c/703735/ is the fix, 703271 is just the bottom of that stack | |
| 15:13:40 | sean-k-mooney | frickler: i was wondering if a better fix might be to start running devstack with USE_VENV=True | |
| 15:13:56 | sean-k-mooney | that shoudl prevent conflict with site packages right | |
| 15:14:12 | efried | frickler: thanks. Who can approve these? | |
| 15:14:44 | sean-k-mooney | however if we wanted to keep the cross project compatblity checking we might want a USE_SHARED_VENV option too | |
| 15:15:42 | frickler | sean-k-mooney: clarkb made some tests with USE_VENV, I don't think he got it into a working state | |
| 15:16:25 | sean-k-mooney | it tried it last night and it got pretty far but it failed to start glance | |
| 15:16:33 | frickler | efried: I pinged some folks in #-qa already, but devstack cores are rare these days. | |
| 15:16:48 | sean-k-mooney | but yes i have not used it beforfe because its not tested in the gate | |
| 15:17:10 | efried | frickler: is there any value to me opening a bug? Or would that just be unnecessary red tape cause now we have to mark commit messages etc? | |
| 15:17:54 | frickler | efried: the patches are in place and seem to work, so I don't see any value in a bug | |
| 15:17:58 | efried | ack | |
| 15:18:46 | efried | sean-k-mooney: are you tracking the effort to migrate n-l-m to zv3? Or is that gmann? Clearly we're going to keep running into this kind of nonsense, and we've been suffering with the gzip thing. | |
| 15:19:11 | frickler | unless someone wants to pick up mriedem's task of feeding elastic-recheck | |
| 15:21:03 | sean-k-mooney | i dont have a bug for that currently. but i can file one | |
| 15:22:34 | sean-k-mooney | i have been tied up with a backport to queens of the in place numab rebuild that i need to get done by friday to do a backport of a feature downstream so i have not had time to work on the live migration job yet | |
| 15:29:50 | openstack | Launchpad bug 1860573 in OpenStack Compute (nova) "Nova legacy jobs should be ported to zuul v3 native jobs" [High,Triaged] | |
| 15:29:50 | sean-k-mooney | efried: https://bugs.launchpad.net/nova/+bug/1860573 | |
| 15:34:52 | efried | thanks sean-k-mooney | |
| 15:35:08 | efried | gmann: ^ for your subscribing enjoyment | |
| 16:16:15 | gmann | efried: ACK. | |
| 16:16:24 | gmann | devstack change is +A too | |
| 16:53:18 | kashyap | sean-k-mooney: Know where is the actual failure here? - https://zuul.opendev.org/t/openstack/build/ce5c1b08a48348bcad0d27dd05b0d076 | |
| 16:53:25 | kashyap | sean-k-mooney: It says "FAILED with status: 2" | |
| 16:54:02 | kashyap | Maybe this - https://0aa0d36168d68dde4230-9fa499072a9f8bf63e024cc09284603e.ssl.cf5.rackcdn.com/616603/15/check/nova-live-migration/ce5c1b0/job-output.txt | |
| 16:54:23 | kashyap | Yep | |
| 16:56:56 | efried | kashyap: n-l-m jobs are failing with PyYAML bs. | |
| 16:57:09 | efried | Fixes in devstack and devstack-gate are merging | |
| 16:57:28 | efried | https://review.opendev.org/#/c/703735/ and its dep | |
| 16:57:32 | kashyap | efried: Ah, thank you, good sir | |
| 16:58:29 | efried | if you want to see the real failure, you have to look in devstacklog.txt.gz | |
| 16:58:32 | efried | which you have to unzip | |
| 16:58:36 | efried | which is a pita | |
| 16:58:49 | efried | and will go away once we migrate n-l-m to zuulv3 | |
| 16:59:00 | efried | which is nontrivial | |
| 16:59:05 | efried | but has been for a while | |
| 16:59:05 | efried | but in the works | |
| 16:59:19 | kashyap | (Okay, I quoted you on the Nova change.) | |
| 16:59:35 | kashyap | efried: Yeah, that's the "main script" the job-output.txt moans about. Thank you | |
| 16:59:37 | efried | but we've been essentially broken by it almost continuously for a couple of weeks. | |
| 17:00:00 | kashyap | Hmm | |
| 17:01:07 | efried | kashyap: If you have any ability/time to help with that migration, it would be much appreciated. | |
| 17:01:46 | efried | I've been avoiding digging in because I would be starting off so far behind the curve that by the time I figured out how to get started someone else would (hopefully) already have it figured out. | |
| 17:02:11 | kashyap | (Afraid, not this week, preparing for a work conf on Fri) | |
| 17:02:14 | efried | not that that would be time wasted for me, but it terms of priorities, I need to be doing other things. | |
| 17:02:20 | efried | okay, it was worth a shot :P | |
| 17:02:39 | kashyap | efried: Yeah, completely understand | |
| 17:02:47 | kashyap | Can't dig into every mom-n-pop failure :D | |
| 17:03:05 | melwitt | kashyap: I'm just reminded, we'd appreciate your review on this proposed revert https://review.opendev.org/703596 | |
| 17:03:05 | kashyap | This channel is logged, /me should mind his language | |
| 17:03:11 | efried | unfortunately the mom-n-pop failures have been what's killing us, and consuming a bunch of my time. | |
| 17:03:26 | kashyap | melwitt: Hi, /me clicks | |
| 17:03:29 | efried | mostly me floundering around trying to find someone who can help with them. | |
| 17:03:34 | kashyap | efried: :-( | |
| 17:05:07 | kashyap | melwitt: To start with, that commit doesn't even tell why on god's green earth it is trying to revert | |
| 17:05:21 | melwitt | kashyap: take that up with sean-k-mooney lol | |
| 17:05:34 | kashyap | I despise "naked commit messages | |
| 17:05:41 | kashyap | s/commit/commit"/ | |
| 17:05:43 | kashyap | :D | |
| 17:05:44 | melwitt | me too | |
| 17:06:06 | sean-k-mooney | kashyap: it is what you get form the ui by default | |
| 17:06:16 | kashyap | Okay, Sean comments on PS1 | |
| 17:06:20 | sean-k-mooney | but i left some comment on the backport then fielded the reviert | |
| 17:06:36 | kashyap | sean-k-mooney: It just doesn't make sense at all to do CPU comparison check on AArch64 | |
| 17:06:45 | sean-k-mooney | that is incorrect | |
| 17:06:52 | kashyap | melwitt: I documented "why" on the main change | |
| 17:06:53 | sean-k-mooney | we have the info in /proc/cpuinfo | |
| 17:07:07 | sean-k-mooney | so we can check the cpu flags and cpu model | |
| 17:07:10 | kashyap | sean-k-mooney: I relied on the expertise on the libvirt dev who wrote that code. Want to argue with him? | |
| 17:07:19 | sean-k-mooney | sure | |
| 17:07:40 | kashyap | And hell, even AArch64 folks *themselves* told that KVM guests are to be run via 'host-passthrough' | |
| 17:07:49 | sean-k-mooney | yes | |
| 17:08:01 | kashyap | s/told/tell/ | |
| 17:08:02 | sean-k-mooney | but host-passthough means that the guest will see the host cpu flags and model | |
| 17:08:09 | kashyap | Yes, so what? | |
| 17:08:14 | sean-k-mooney | im not arguing that we should not use host-passthough | |
| 17:08:20 | kashyap | That's the whole point in this case | |
| 17:08:34 | sean-k-mooney | im am saying when you do its more important to ensur the flags and modle do not change | |
| 17:08:52 | kashyap | melwitt: sean-k-mooney: Let me quote my message from the main change, since I don't know what people have read or haven't | |
| 17:08:55 | kashyap | [quote] | |
| 17:08:56 | kashyap | So it doesn't make sense to do CPU compatability check on AArch64. And the AArch64 folks themselves recommend that the way to run KVM guests on AArch64 is via 'host-passthrough'. | |
| 17:09:00 | kashyap | It is true that libvirt does not know how to detect host CPU model on AArch64, but even if it _wants_ to know, it cannot, because even the `/proc/cpuinfo` on AArch64 doesn't show anything interesting. There are lots of vendors making different AArch64 CPUs, and they are not easily comparable. They all differ in various ways. (This is also confirmed by Jiri Denemark of libvirt.) | |
| 17:09:06 | kashyap | [/quote] | |