| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-03-19 | |||
| 22:04:41 | zzzeek | melwitt: your only option is to try the NullPool and turn off pooling entirely | |
| 22:04:59 | zzzeek | melwitt: unfortunately oslo.db has no option for that so if you wanted to test it you'd need to monkeypatch oslo.db | |
| 22:05:44 | melwitt | oh. good to know. I wouldn't want to use it then. I was just curious if there was any good/recommended way to address the performance problem with rollbacks if there were one | |
| 22:08:10 | melwitt | it's weird. in the scheduler, there's this constant flow of service select/update/commit happening for the service heartbeats, and when things fail, all of the other selects (for services, compute_nodes) just disappear entirely | |
| 22:08:44 | melwitt | and all I see are the steady select/update/commit for the heartbeats | |
| 22:10:58 | openstackgerrit | Merged openstack/nova master: libvirt: Fix unit test error block info on non x86 architecture https://review.opendev.org/713905 | |
| 22:15:19 | melwitt | ohhh dang I actually see now that the debug logging I added in my DNM patch is _not_ showing up, meaning we're not even trying to query for compute_nodes. that's bad. /me keeps digging | |
| 22:31:06 | zzzeek | melwitt: sorry I dont know that code very well, keep me in the loop :) | |
| 22:31:16 | zzzeek | (dont know it at all really ... :) ) | |
| 22:37:06 | melwitt | oh, derp, false alarm I think. I only put the debug statements in the master branch and this is a grenade job so if it failed on the old side, that would be stable/train which doesn't have the debug logging on it. I need to cherry pick this DNM to train again and run with Depends-On | |
| 22:42:19 | openstackgerrit | melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478 | |
| 22:42:19 | openstack | bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929 | |
| 22:42:29 | openstackgerrit | melanie witt proposed openstack/nova stable/train: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/713116 | |
| 22:43:13 | openstackgerrit | melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478 | |
| #openstack-nova - 2020-03-20 | |||
| 00:08:09 | openstackgerrit | Merged openstack/nova stable/train: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713836 | |
| 01:55:57 | openstackgerrit | melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478 | |
| 01:55:57 | openstack | bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929 | |
| 05:00:33 | openstackgerrit | Arthur Dayne proposed openstack/nova master: libvirt:driver:Disallow AIO=native when 'O_DIRECT' is not available https://review.opendev.org/682772 | |
| 06:49:52 | ygk_12345 | can someone look into this please https://bugs.launchpad.net/nova/+bug/1868203 | |
| 06:49:52 | openstack | Launchpad bug 1868203 in OpenStack Compute (nova) "nova-compute error" [Undecided,New] | |
| 06:58:05 | openstackgerrit | Arthur Dayne proposed openstack/nova master: libvirt:driver:Disallow AIO=native when 'O_DIRECT' is not available https://review.opendev.org/682772 | |
| 08:01:39 | openstackgerrit | Brin Zhang proposed openstack/nova master: libvirt:driver:Disallow AIO=native when 'O_DIRECT' is not available https://review.opendev.org/682772 | |
| 08:07:15 | openstackgerrit | Brin Zhang proposed openstack/nova master: libvirt:driver:Disallow AIO=native when 'O_DIRECT' is not available https://review.opendev.org/682772 | |
| 08:14:52 | brinzhang_ | lyarwood:https://review.opendev.org/#/c/682772/ fixed your comment, and add the test case you asked, pls review again if you are free ^^ | |
| 09:06:56 | openstackgerrit | Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494 | |
| 09:20:15 | nightmare_unreal | hey guys, how can I run tests for python-openstackclient | |
| 09:20:25 | nightmare_unreal | i wanted to test it locally before submitting a patch | |
| 09:24:01 | openstackgerrit | Lee Yarwood proposed openstack/nova master: nova-live-migration: Only stop n-cpu and q-agt during evacuation testing https://review.opendev.org/714057 | |
| 09:24:14 | lyarwood | nightmare_unreal: tox -e py36 ; tox -e pep8 | |
| 09:24:33 | lyarwood | sean-k-mooney: ^ btw, I missed that we are running c-vol and g-api on the subnodes for some reason | |
| 09:24:54 | lyarwood | I was sure we removed these recently | |
| 09:29:19 | openstackgerrit | Merged openstack/nova master: Support unshelve with qos ports https://review.opendev.org/704759 | |
| 09:30:13 | nightmare_unreal | thanks lyarwood but in which directory to run ? inside devstack or inside openstackclient | |
| 09:31:02 | ygk_12345 | hi all | |
| 09:31:36 | ygk_12345 | can someone look into this once ? https://bugs.launchpad.net/nova/+bug/1868203 | |
| 09:31:36 | openstack | ygk_12345: Error: Could not gather data from Launchpad for bug #1868203 (https://launchpad.net/bugs/1868203). The error has been logged | |
| 09:35:10 | lyarwood | nightmare_unreal: openstackclient sorry | |
| 09:35:40 | nightmare_unreal | thanks | |
| 09:41:43 | nightmare_unreal | thanks lyarwood | |
| 09:57:53 | openstackgerrit | melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478 | |
| 09:57:53 | openstack | bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929 | |
| 10:02:40 | openstackgerrit | melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478 | |
| 10:02:40 | openstack | bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929 | |
| 10:06:02 | openstackgerrit | melanie witt proposed openstack/nova master: DNM: try to get some debug info for bug 1844929 https://review.opendev.org/701478 | |
| 10:06:02 | openstack | bug 1844929 in OpenStack Compute (nova) "grenade jobs failing due to "Timed out waiting for response from cell" in scheduler" [High,Confirmed] https://launchpad.net/bugs/1844929 | |
| 10:06:42 | ygk_12345 | can someone look into this https://bugs.launchpad.net/nova/+bug/1868203 | |
| 10:06:42 | openstack | Launchpad bug 1868203 in OpenStack Compute (nova) "nova-compute error" [Undecided,New] | |
| 10:07:09 | openstackgerrit | Merged openstack/nova master: Add test coverage of existing flavor_access policies https://review.opendev.org/713556 | |
| 10:08:54 | nightmare_unreal | how to add my changes to doc or how to build doc for openstackclient ? | |
| 10:09:12 | lyarwood | ygk_12345: can you attach the n-cpu log to the bug? Looks like you're trying to launch qemu-system-ppc64 instances and failing? | |
| 10:09:46 | lyarwood | nightmare_unreal: tox -e docs | |
| 10:10:00 | ygk_12345 | lyarwood it is openstack ansible rocky. where can I find the n-cpu log ? | |
| 10:10:20 | lyarwood | nightmare_unreal: review tox.ini and https://docs.openstack.org/python-openstackclient/latest/contributor/developing.html | |
| 10:10:27 | nightmare_unreal | thanks lyarwood , but how to add my doc code. for eg. i made a new flag | |
| 10:10:57 | lyarwood | nightmare_unreal: look for changes under docs (using tig and/or tig blame) for examples. | |
| 10:11:09 | jkulik | Can someone close https://bugs.launchpad.net/nova/+bug/1855776 it should have been fixed by https://review.opendev.org/#/c/710214/ as it seems to be a duplicate of https://bugs.launchpad.net/nova/+bug/1865040 | |
| 10:11:09 | openstack | Launchpad bug 1855776 in OpenStack Compute (nova) "Aggregate ID validation" [Undecided,In progress] - Assigned to Mykola Yakovliev (vegasq) | |
| 10:11:10 | openstack | Launchpad bug 1865040 in OpenStack Compute (nova) "Able to show update and delete aggregate with invalid id" [Undecided,Fix released] - Assigned to Johannes Kulik (jkulik) | |
| 10:11:11 | nightmare_unreal | thanks | |
| 10:11:28 | openstackgerrit | Luyao Zhong proposed openstack/nova stable/train: bug-fix: Reject live migration with vpmem https://review.opendev.org/714064 | |
| 10:11:28 | lyarwood | ygk_12345: sorry no idea | |
| 10:14:38 | lyarwood | jkulik: if you can still reproduce against master feel free to reuse https://bugs.launchpad.net/nova/+bug/1855776 and submit a fix. | |
| 10:14:38 | openstack | Launchpad bug 1855776 in OpenStack Compute (nova) "Aggregate ID validation" [Undecided,In progress] - Assigned to Mykola Yakovliev (vegasq) | |
| 10:15:40 | jkulik | lyarwood, it looks exactly like the problem in the second bug, which got closed by merging. I just want that bug closed, too :) | |
| 10:17:46 | gibi | ygk_12345: regarding bug #1868203 I feels like a QEMU problem on your compute. If QEMU fails to start then nova will not know the available resources on that compute | |
| 10:17:46 | openstack | bug 1868203 in OpenStack Compute (nova) "nova-compute error" [Undecided,New] https://launchpad.net/bugs/1868203 | |
| 10:18:04 | gibi | ygk_12345: you should reach out the QEMU devs | |
| 10:20:33 | gibi | jkulik: marked the bug as duplicate of 1865040 | |
| 10:20:42 | gibi | jkulik: thanks for noticing | |
| 10:20:46 | jkulik | gibi, thanks | |
| 10:25:04 | ygk_12345 | @gibi the libvirtd service is in error state | |
| 10:25:59 | lyarwood | ygk_12345: if libvirtd isn't starting then that's a bug for the libvirt folks | |
| 10:26:19 | gibi | ygk_12345: based on the libvird service logs you added to the bug report the libvirtd is in error most probably due to QEMU cannot be started | |
| 10:27:12 | ygk_12345 | @gibi and what is that placement error I mentioned ? is it related to this libvirtd ? | |
| 10:27:47 | gibi | ygk_12345: could be related yes | |
| 10:28:12 | gibi | ygk_12345: is reported missing RP has the same uuid as the faulty compute node uuid? | |
| 10:28:27 | gibi | sorry I have to jump on a call very soon | |
| 10:39:09 | kashyap | ygk_12345: How is libvirtd failing to start? /me looks at the scroll | |
| 10:39:52 | ygk_12345 | kashyap http://paste.openstack.org/show/790930/ | |
| 10:41:37 | ygk_12345 | @las | |
| 10:41:45 | ygk_12345 | kashyap it is ubuntu 18 | |
| 10:42:05 | kashyap | ygk_12345: What libvirt and QEMU versions? | |
| 10:42:28 | kashyap | This "relocation error" looks like some dependency problem | |
| 10:42:46 | ygk_12345 | kashyap how to check ? | |
| 10:44:01 | kashyap | ygk_12345: To get versions? Please use 'apt' command: `apt list libvirt` (or whatever Ubuntu calls it) | |
| 10:44:02 | ygk_12345 | kashyap libvirtd (libvirt) 5.4.0 | |
| 10:44:08 | kashyap | And also QEMU version | |
| 10:44:38 | ygk_12345 | root@b2a-1:~# apt list qemu | |
| 10:44:38 | ygk_12345 | Listing... Done | |
| 10:44:39 | ygk_12345 | qemu/bionic-updates,now 1:4.0+dfsg-0ubuntu9.3~cloud0 amd64 [installed] | |
| 10:47:59 | stephenfin | artom: When you're about, I've replied on https://review.opendev.org/#/c/631053/ | |
| 10:48:20 | stephenfin | artom: let me know if you want me to rework the commit message to make my point clearer | |
| 10:50:00 | nightmare_unreal | hello if i want to add something to doc for e.g. in quota docs here : https://docs.openstack.org/python-openstackclient/pike/cli/command-objects/quota.html | |
| 10:50:03 | nightmare_unreal | how can i do it | |
| 10:50:10 | nightmare_unreal | I want to add doc for --force flag | |
| 10:50:21 | nightmare_unreal | i have just submitted a patch for it | |
| 10:50:30 | nightmare_unreal | but don't know how to add doc for it | |
| 10:53:22 | lyarwood | nightmare_unreal: edit doc/source/cli/command-objects/quota.rst | |
| 10:53:35 | kashyap | ygk_12345: Your installation on Ubuntu of QEMU/libvirt is kaput with broken libraries. How did you even install it? | |
| 10:53:42 | lyarwood | nightmare_unreal: FWIW this channel isn't for python-openstackclient btw, just being polite and trying to help. | |