| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-22 | |||
| 06:02:39 | frickler | so I started to fix the live-migration job at https://review.opendev.org/703735 , but a better solution would be to make os-testr work with system pyyaml. the latted is depended on by e.g. cloud-init so it's very hard to remove. or set up using os-testr from within a venv only | |
| 06:03:03 | frickler | also, has anyone started work to move that job to zuul v3? | |
| 10:47:15 | stephenfin | frickler: I assume "install everything devstack'y in a virtualenv" is much harder than it seems | |
| 10:47:35 | stephenfin | frickler: and yes, mriedem has started work moving to zuul v3 and I think sean-k-mooney is continuing that | |
| 10:51:40 | bauzas | stephenfin: gibi: worth looking at an already-approved spec that I could work on in Ussuri ? https://review.opendev.org/#/c/702943/ | |
| 10:58:14 | stephenfin | bauzas: Sure. Got some downstream release'y stuff to take care of this morning first though | |
| 10:58:30 | bauzas | hah | |
| 10:58:32 | bauzas | your turn | |
| 11:36:04 | openstackgerrit | John Garbutt proposed openstack/nova-specs master: Small fixes to unified limits spec https://review.opendev.org/703773 | |
| 12:30:06 | gibi | it seems I need to spend my time today on downstream issues | |
| 12:47:05 | openstackgerrit | Lee Yarwood proposed openstack/nova master: virt: Provide block_device_info during rescue https://review.opendev.org/700811 | |
| 12:47:05 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Add support for stable device rescue https://review.opendev.org/700812 | |
| 12:47:06 | openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Add stable device rescue docs https://review.opendev.org/700837 | |
| 13:54:41 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'nova.image.api' module https://review.opendev.org/702451 | |
| 13:54:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: nova-net: Remove unused nova-network objects https://review.opendev.org/697156 | |
| 13:54:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Update API reference guide https://review.opendev.org/703796 | |
| 14:40:28 | openstackgerrit | sean mooney proposed openstack/os-vif master: move os-vif-ovs to be a non legacy job. https://review.opendev.org/701601 | |
| 14:40:29 | openstackgerrit | sean mooney proposed openstack/os-vif master: Revert "[Follow Up] OVS DPDK port representors support" https://review.opendev.org/703672 | |
| 15:04:27 | efried | sean-k-mooney, gmann: So is the PyYAML thing in n-l-m sorted yet or still busted? | |
| 15:06:26 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: Remove 'test_cold_migrate_with_physnet_fails' test https://review.opendev.org/702971 | |
| 15:06:26 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: Block rebuild when NUMA topology changed https://review.opendev.org/702972 | |
| 15:06:27 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: Disable NUMATopologyFilter on rebuild https://review.opendev.org/702973 | |
| 15:06:27 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: FUP for in-place numa rebuild https://review.opendev.org/702974 | |
| 15:06:56 | frickler | efried: busted, need a fix in devstack and ds-gate each | |
| 15:07:00 | sean-k-mooney | efried: its breaking os-vifs 1 remaining legacy job too | |
| 15:07:25 | sean-k-mooney | frickler: i assume ds-gate is installing os-testr or something like that | |
| 15:08:33 | sean-k-mooney | the non legacy versions seem to be fine but its the installation of os-testr that is trigering PyYAML to be reinstalled | |
| 15:09:48 | efried | I can't tell if https://review.opendev.org/#/c/561597/ will fix it temporarily until https://review.opendev.org/#/c/703735/ can be landed. | |
| 15:10:20 | efried | frickler: ^ ? | |
| 15:10:48 | frickler | efried: the former isn't the fix, but the cause of the new failures | |
| 15:11:06 | efried | oh? I was seeing them before that patch landed | |
| 15:11:21 | efried | I think | |
| 15:11:28 | frickler | fixes are https://review.opendev.org/703792 and https://review.opendev.org/#/c/703271/4 | |
| 15:12:39 | frickler | oh https://review.opendev.org/#/c/703735/ is the fix, 703271 is just the bottom of that stack | |
| 15:13:40 | sean-k-mooney | frickler: i was wondering if a better fix might be to start running devstack with USE_VENV=True | |
| 15:13:56 | sean-k-mooney | that shoudl prevent conflict with site packages right | |
| 15:14:12 | efried | frickler: thanks. Who can approve these? | |
| 15:14:44 | sean-k-mooney | however if we wanted to keep the cross project compatblity checking we might want a USE_SHARED_VENV option too | |
| 15:15:42 | frickler | sean-k-mooney: clarkb made some tests with USE_VENV, I don't think he got it into a working state | |
| 15:16:25 | sean-k-mooney | it tried it last night and it got pretty far but it failed to start glance | |
| 15:16:33 | frickler | efried: I pinged some folks in #-qa already, but devstack cores are rare these days. | |
| 15:16:48 | sean-k-mooney | but yes i have not used it beforfe because its not tested in the gate | |
| 15:17:10 | efried | frickler: is there any value to me opening a bug? Or would that just be unnecessary red tape cause now we have to mark commit messages etc? | |
| 15:17:54 | frickler | efried: the patches are in place and seem to work, so I don't see any value in a bug | |
| 15:17:58 | efried | ack | |
| 15:18:46 | efried | sean-k-mooney: are you tracking the effort to migrate n-l-m to zv3? Or is that gmann? Clearly we're going to keep running into this kind of nonsense, and we've been suffering with the gzip thing. | |
| 15:19:11 | frickler | unless someone wants to pick up mriedem's task of feeding elastic-recheck | |
| 15:21:03 | sean-k-mooney | i dont have a bug for that currently. but i can file one | |
| 15:22:34 | sean-k-mooney | i have been tied up with a backport to queens of the in place numab rebuild that i need to get done by friday to do a backport of a feature downstream so i have not had time to work on the live migration job yet | |
| 15:29:50 | sean-k-mooney | efried: https://bugs.launchpad.net/nova/+bug/1860573 | |
| 15:29:50 | openstack | Launchpad bug 1860573 in OpenStack Compute (nova) "Nova legacy jobs should be ported to zuul v3 native jobs" [High,Triaged] | |
| 15:34:52 | efried | thanks sean-k-mooney | |
| 15:35:08 | efried | gmann: ^ for your subscribing enjoyment | |
| 16:16:15 | gmann | efried: ACK. | |
| 16:16:24 | gmann | devstack change is +A too | |
| 16:53:18 | kashyap | sean-k-mooney: Know where is the actual failure here? - https://zuul.opendev.org/t/openstack/build/ce5c1b08a48348bcad0d27dd05b0d076 | |
| 16:53:25 | kashyap | sean-k-mooney: It says "FAILED with status: 2" | |
| 16:54:02 | kashyap | Maybe this - https://0aa0d36168d68dde4230-9fa499072a9f8bf63e024cc09284603e.ssl.cf5.rackcdn.com/616603/15/check/nova-live-migration/ce5c1b0/job-output.txt | |
| 16:54:23 | kashyap | Yep | |
| 16:56:56 | efried | kashyap: n-l-m jobs are failing with PyYAML bs. | |
| 16:57:09 | efried | Fixes in devstack and devstack-gate are merging | |
| 16:57:28 | efried | https://review.opendev.org/#/c/703735/ and its dep | |
| 16:57:32 | kashyap | efried: Ah, thank you, good sir | |
| 16:58:29 | efried | if you want to see the real failure, you have to look in devstacklog.txt.gz | |
| 16:58:32 | efried | which you have to unzip | |
| 16:58:36 | efried | which is a pita | |
| 16:58:49 | efried | and will go away once we migrate n-l-m to zuulv3 | |
| 16:59:00 | efried | which is nontrivial | |
| 16:59:05 | efried | but in the works | |
| 16:59:05 | efried | but has been for a while | |
| 16:59:19 | kashyap | (Okay, I quoted you on the Nova change.) | |
| 16:59:35 | kashyap | efried: Yeah, that's the "main script" the job-output.txt moans about. Thank you | |
| 16:59:37 | efried | but we've been essentially broken by it almost continuously for a couple of weeks. | |
| 17:00:00 | kashyap | Hmm | |
| 17:01:07 | efried | kashyap: If you have any ability/time to help with that migration, it would be much appreciated. | |
| 17:01:46 | efried | I've been avoiding digging in because I would be starting off so far behind the curve that by the time I figured out how to get started someone else would (hopefully) already have it figured out. | |
| 17:02:11 | kashyap | (Afraid, not this week, preparing for a work conf on Fri) | |
| 17:02:14 | efried | not that that would be time wasted for me, but it terms of priorities, I need to be doing other things. | |
| 17:02:20 | efried | okay, it was worth a shot :P | |
| 17:02:39 | kashyap | efried: Yeah, completely understand | |
| 17:02:47 | kashyap | Can't dig into every mom-n-pop failure :D | |
| 17:03:05 | kashyap | This channel is logged, /me should mind his language | |
| 17:03:05 | melwitt | kashyap: I'm just reminded, we'd appreciate your review on this proposed revert https://review.opendev.org/703596 | |
| 17:03:11 | efried | unfortunately the mom-n-pop failures have been what's killing us, and consuming a bunch of my time. | |
| 17:03:26 | kashyap | melwitt: Hi, /me clicks | |
| 17:03:29 | efried | mostly me floundering around trying to find someone who can help with them. | |
| 17:03:34 | kashyap | efried: :-( | |
| 17:05:07 | kashyap | melwitt: To start with, that commit doesn't even tell why on god's green earth it is trying to revert | |
| 17:05:21 | melwitt | kashyap: take that up with sean-k-mooney lol | |
| 17:05:34 | kashyap | I despise "naked commit messages | |
| 17:05:41 | kashyap | s/commit/commit"/ | |
| 17:05:43 | kashyap | :D | |
| 17:05:44 | melwitt | me too | |
| 17:06:06 | sean-k-mooney | kashyap: it is what you get form the ui by default | |
| 17:06:16 | kashyap | Okay, Sean comments on PS1 | |
| 17:06:20 | sean-k-mooney | but i left some comment on the backport then fielded the reviert | |
| 17:06:36 | kashyap | sean-k-mooney: It just doesn't make sense at all to do CPU comparison check on AArch64 | |
| 17:06:45 | sean-k-mooney | that is incorrect | |
| 17:06:52 | kashyap | melwitt: I documented "why" on the main change | |
| 17:06:53 | sean-k-mooney | we have the info in /proc/cpuinfo | |