| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-01 | |||
| 09:13:14 | openstackgerrit | Theodoros Tsioutsias proposed openstack/nova-specs master: Add PENDING vm state https://review.openstack.org/554212 | |
| 09:13:47 | ttsiouts | johnthetubaguy: I tried to refresh the spec ^ | |
| 10:03:21 | sean-k-mooney | o/ | |
| 10:04:17 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Update the contributor guide https://review.openstack.org/606926 | |
| 10:05:40 | sean-k-mooney | stephenfin: when you get a chance can you run your eyes over https://review.openstack.org/#/c/604992/3. looks like an easy addtion to me. | |
| 10:05:48 | stephenfin | ack | |
| 10:06:23 | sean-k-mooney | stephenfin: i think we need to port other vif types from kuyr-kubernetes too but they can be in different patches | |
| 10:35:08 | sean-k-mooney | ralonsoh: hi are you around today? | |
| 10:35:47 | ralonsoh | sean-k-mooney: very well, fighting with DVR and FIP (and SRIOV) | |
| 10:36:52 | stephenfin | sean-k-mooney: Also, "i prefer explcit else clauses" | |
| 10:37:00 | stephenfin | sean-k-mooney: Just wanted you to know you're wrong | |
| 10:37:04 | stephenfin | sean-k-mooney: That is all. | |
| 10:37:05 | stephenfin | :) | |
| 10:37:17 | sean-k-mooney | ralonsoh: if you bussy i can respin https://review.openstack.org/#/c/605422/2 | |
| 10:38:05 | ralonsoh | sean-k-mooney: sure, I'll do this now | |
| 10:38:40 | sean-k-mooney | stephenfin: hehe well we can agree to disagree on that. it can help prevent some types of errors when you are maintaing the fucntion in the future but also consumes indentation space so is not allways desireable | |
| 10:48:35 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Allow 'nova-xvpvncproxy' to be called with CLI args https://review.openstack.org/606929 | |
| 10:48:35 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Rewrite the console doc https://review.openstack.org/606148 | |
| 10:51:27 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Remove IPTools deprecated implementation https://review.openstack.org/605422 | |
| 10:51:45 | ralonsoh | sean-k-mooney: done | |
| 10:55:27 | sean-k-mooney | ralonsoh: thanks :) | |
| 11:11:38 | openstackgerrit | Theodoros Tsioutsias proposed openstack/nova-specs master: Enable rebuild for instances in cell0 https://review.openstack.org/554218 | |
| 11:15:21 | openstackgerrit | Theodoros Tsioutsias proposed openstack/nova-specs master: Add PENDING vm state https://review.openstack.org/554212 | |
| 11:28:36 | openstackgerrit | Merged openstack/nova stable/rocky: Use INFO for logging no allocation candidates https://review.openstack.org/605771 | |
| 11:48:27 | openstackgerrit | Merged openstack/os-vif master: Add support for Windows network commands https://review.openstack.org/487405 | |
| 12:05:59 | kashyap | stephenfin: On that version bump thing for libvirt, I just looked at "what's involved" to clean up | |
| 12:06:15 | kashyap | stephenfin: I need to address at least 10 unit tests. Will get to it this week. Currently buried in something a bit more urgent | |
| 12:30:43 | stephenfin | kashyap: Yup, it's low priority and can be done anytime this cycle. No rush! | |
| 12:31:56 | kashyap | Yep. | |
| 12:32:40 | kashyap | Then, I might even wait until I'm through all the CPU modelling stuff that I'm looking into (until 27 Oct) | |
| 12:40:36 | mdbooth | bauzas: LM failure on https://review.openstack.org/#/c/604400/ | |
| 12:41:05 | mdbooth | So, the exact failure is unlikely to be relevant. Chances are that this is just the first thing which touched that host, and the host was broken by the evacuate test we just ran. | |
| 12:41:35 | bauzas | mdbooth: so, you got a failure for req-dd88c142-4710-49d0-b6e9-594c6a1e89f6 (live-migrate action) because of http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/screen-n-sch.txt.gz#_Sep_28_20_39_55_712058 | |
| 12:41:52 | mdbooth | bauzas: Yes | |
| 12:42:01 | bauzas | rather http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/screen-n-sch.txt.gz#_Sep_28_20_39_55_711740 | |
| 12:42:07 | mdbooth | So, the question is: why does the computefilter think the host is down? | |
| 12:42:26 | bauzas | mdbooth: because the db record is old | |
| 12:42:45 | bauzas | mdbooth: I mean the updated_at filed | |
| 12:42:47 | bauzas | field* | |
| 12:42:52 | mdbooth | bauzas: Right, but in the compute logs I see the host joining the service group within the last 60 secs | |
| 12:42:58 | bauzas | https://github.com/openstack/nova/blob/master/nova/scheduler/filters/compute_filter.py#L44 | |
| 12:43:12 | mdbooth | However, I note that it doesn't update the record at that point, just schedules it to be updated in 5 seconds | |
| 12:43:40 | mdbooth | So I then got to wondering why it doesn't seem to have been updated 5 seconds later. | |
| 12:44:00 | mdbooth | I was wondering if perhaps a long running init_host might be related? | |
| 12:44:16 | mdbooth | Because that's feasible following the evacuate test | |
| 12:45:01 | bauzas | https://github.com/openstack/nova/blob/master/nova/servicegroup/drivers/db.py#L73 | |
| 12:45:09 | mdbooth | However, threadgroup.add_timer seems to be unconditional | |
| 12:45:21 | bauzas | and https://github.com/openstack/nova/blob/master/nova/servicegroup/drivers/db.py#L60 | |
| 12:45:59 | mdbooth | bauzas: Right, so it hasn't updated its record within the last 60 seconds | |
| 12:46:20 | bauzas | just checking, it's 60sec http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/screen-n-sch.txt.gz#_Sep_28_20_25_02_439642 | |
| 12:46:27 | bauzas | yup | |
| 12:46:32 | mdbooth | But given that it joined the servicegroup http://logs.openstack.org/00/604400/6/check/nova-live-migration/4ba65e4/logs/subnode-2/screen-n-cpu.txt.gz?#_Sep_28_20_39_12_277355 | |
| 12:46:38 | bauzas | now, I'm checking the compute logs | |
| 12:46:53 | mdbooth | I can't see why not. | |
| 12:47:08 | bauzas | it's a periodic task IIRC | |
| 12:47:41 | mdbooth | Looks like it's a threadgroup timer | |
| 12:47:57 | mdbooth | Which would be eventlet-fu | |
| 12:48:18 | mdbooth | But I wonder if its execution could be blocked by, eg, a long-running periodic task | |
| 12:49:51 | sean-k-mooney | mdbooth: we had a downstream bugs(3) related to osp7 upgrades that saw similar behavior. compute filter elimated host becase host was still down after upgrade. in the downstream case it was causeing tempset test to fail as the heartbeat had not been recived yet and the host was down. this seams similar. | |
| 12:50:28 | mdbooth | sean-k-mooney: The immediate cause would certainly be the same. I wonder if the underlying cause is, though. | |
| 12:50:33 | mdbooth | Did you get to the bottom of it? | |
| 12:51:40 | sean-k-mooney | mdbooth: no in the down stream case we saw that subsequent schduling attempts suceeded as the heatbeat had been processed so we closed as not a but as the cloud had not stablised after upgrade before running the tests | |
| 12:52:05 | sean-k-mooney | e.g. it corrected it self a few seconds after teh tempest test was started | |
| 12:57:49 | bauzas | mdbooth: honestly, I don't see the problem with the service group .join() call | |
| 12:58:44 | bauzas | mdbooth: is this only a transient issue ? | |
| 12:59:10 | bauzas | mdbooth: if that's more than that, try to have a DNM patch adding some log like herehttps://github.com/openstack/nova/blob/master/nova/servicegroup/drivers/db.py#L91 | |
| 13:11:01 | gryf | is anyone know what is happening with this blueprint: https://blueprints.launchpad.net/nova/+spec/numa-aware-live-migration ? | |
| 13:12:16 | mnaser | a little idle, simple clean backport if someone has a second or two: https://review.openstack.org/#/q/I811e84af46d678c3fdbf94ee400eabe659fc3d4e | |
| 13:14:19 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: WIP: Fix test_instance_action functional test failure https://review.openstack.org/606947 | |
| 13:17:30 | mdbooth | Folks, anybody else hitting the bug in parallel evacuate in CI I have a fix up for it here: https://review.openstack.org/#/c/605436/ | |
| 13:18:08 | mdbooth | It's not a one line fix, though, so you'll need to block out some time to review it. However, probably not as much time as we're losing to this test failing continuously. | |
| 13:29:30 | mdbooth | Hmm, I just had to recheck all 3 commits from this morning for that bug :/ | |
| 13:53:22 | efried | n-sch/placement meeting in 7 minutes in #openstack-meeting-alt | |
| 14:45:33 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Fix test_instance_action functional test failure https://review.openstack.org/606947 | |
| 14:57:57 | openstackgerrit | Chris Dent proposed openstack/nova master: Do not encode server create location reponse header https://review.openstack.org/606967 | |
| 15:25:04 | mriedem | bauzas: let's not run the linuxbridge job on all nova changes https://review.openstack.org/#/c/606616/ | |
| 15:25:08 | mriedem | ^ fixes a regression | |
| 15:25:23 | bauzas | mriedem: /me looks | |
| 15:26:13 | bauzas | mriedem: oh shit, reviewing the py3-first changes was horribly difficult, my bad if I made a regression :( | |
| 15:26:31 | mriedem | it's ok, one line within hundreds, hard to spot | |
| 15:27:42 | bauzas | mriedem: https://review.openstack.org/#/c/606616/1/.zuul.yaml@186 why this line couldn't be only needed ? | |
| 15:28:17 | bauzas | ^(?!nova/network/.*)(?!nova/virt/libvirt/vif.py).*$ means 'all but nova/network or nova/virt/libvirt/vif.py AFAIU | |
| 15:28:38 | bauzas | but anyway, a nit | |
| 15:29:19 | bauzas | +Wd | |
| 15:30:21 | mriedem | yeah could do that in a follow up | |
| 15:39:06 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop nova-multiattach job https://review.openstack.org/606981 | |
| 15:41:38 | openstackgerrit | Chris Dent proposed openstack/nova master: Do not encode server create location response header https://review.openstack.org/606967 | |
| 15:47:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop nova-multiattach job https://review.openstack.org/606981 | |
| 15:52:07 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop nova-multiattach job https://review.openstack.org/606981 | |
| 15:55:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove redundant irrelevant-files from neutron-tempest-linuxbridge https://review.openstack.org/606989 | |
| 15:55:30 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Consider nested allocations during allocation cleanup https://review.openstack.org/606050 | |
| 15:55:30 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ignore forcing of live migration for nested instance https://review.openstack.org/605785 | |
| 15:55:31 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Run negative server moving tests with nested RPs https://review.openstack.org/604125 | |
| 15:55:31 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ignore forcing of evacuation for nested instance https://review.openstack.org/606111 | |
| 16:07:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Allow 'nova-xvpvncproxy' to be called with CLI args https://review.openstack.org/606929 | |
| 16:07:15 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Rewrite the console doc https://review.openstack.org/606148 | |
| 16:07:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Add minimal documentation for MKS consoles https://review.openstack.org/606993 | |
| 16:07:16 | openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Add minimal documentation for RDP consoles https://review.openstack.org/606992 | |
| 16:10:34 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Update --max-rows parameter description for archive_deleted_rows https://review.openstack.org/606995 | |
| 16:21:33 | stephenfin | This console documentation stuff is a damn rabbit hole :/ | |