| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-07-24 | |||
| 18:38:14 | melwitt | I know he's allowed to be there lol | |
| 18:38:24 | artom | WELL WHAT THE HELL ARE WE ARGUING ABOUT | |
| 18:38:27 | melwitt | I just thought if we swapped cause he was out, I was surprised when he was there | |
| 18:38:37 | melwitt | I DONT KNOW | |
| 18:38:41 | artom | WHY ARE WE YELLING | |
| 18:39:02 | melwitt | WE ARE HAVING TROUBLE CONTROLLING THE VOLUME OF OUR VOICE | |
| 18:39:06 | artom | OH RIGHT I STARTED IT IM SO SORRY | |
| 18:40:08 | melwitt | s/TROUBLE/DIFFICULTY/ | |
| 18:40:15 | artom | Umm, how about we do real work for a bit? Is there a way to see stats for a particular job? nova-ceph-multistore just failed twice on me | |
| 18:40:26 | dansmith | lol | |
| 18:40:26 | dansmith | dude | |
| 18:40:34 | artom | mriedem would have hacked up a logtash query in seconds | |
| 18:40:37 | dansmith | have you like paid attention to the last three hours in here at all? | |
| 18:40:38 | melwitt | omg | |
| 18:40:49 | dansmith | and also, logstash is still fubar I think | |
| 18:40:55 | melwitt | no u di'nt | |
| 18:41:21 | artom | dansmith, me? Pay attention? lol u cray cray | |
| 18:41:27 | dansmith | apparently ;) | |
| 19:22:53 | mriedem | i only do logdna queries these days now anyway | |
| 19:43:25 | melwitt | dansmith: I opened https://bugs.launchpad.net/nova/+bug/1888895 for the gate failure. going to mail the ML now | |
| 19:43:25 | openstack | Launchpad bug 1888895 in devstack-plugin-ceph "nova-ceph-multistore job fails often with 'No valid host was found. There are not enough hosts available.'" [Undecided,In progress] | |
| 19:43:44 | dansmith | cool | |
| 19:44:15 | melwitt | great, the WIP patch just failed | |
| 19:44:25 | melwitt | [errno 110] error connecting to the cluster wtf | |
| 19:46:27 | dansmith | maybe that config made it fail to start? | |
| 19:47:05 | dansmith | no ceph logs | |
| 19:47:54 | dansmith | hrm, don't even see it set the ini, | |
| 19:47:59 | dansmith | so maybe it didn't even get that far | |
| 19:50:44 | melwitt | yeah must be unless it's a fluke coincidence that ceph totally bombed this time | |
| 19:51:41 | dansmith | I think it must be a fluke bombing because it didn't run that line | |
| 19:53:01 | dansmith | ah here we go: 2020-07-24 18:32:36.045 | /opt/stack/devstack-plugin-ceph/devstack/lib/ceph: line 429: (24G: value too great for base (error token is "24G") | |
| 19:55:46 | melwitt | ah | |
| 19:55:59 | dansmith | hacky fix | |
| 20:05:20 | dansmith | melwitt: oops, should have put the bug on that, sorry | |
| 20:05:24 | dansmith | but it'll need cleanup | |
| 20:05:35 | melwitt | ah yeah | |
| 20:05:54 | melwitt | I was just thinking, I think all ceph jobs in openstack are broken over this, not just ours | |
| 20:06:50 | dansmith | potentially, but like I said, my job may be running slower or have different behaviors | |
| 20:07:26 | dansmith | actually | |
| 20:07:40 | melwitt | yeah. I was thinking of replying to mention other ceph jobs may be affected too. I see the older version 14.2.2 being pulled in openstack/tempest, for example | |
| 20:07:44 | dansmith | heh, I just saw a glance failure on the plain ceph job which is the same novalidhost | |
| 20:07:48 | dansmith | so I think yeah. | |
| 20:07:57 | dansmith | yeah | |
| 20:08:14 | dansmith | this is from one of my glance patches: https://e81e6b81331830d4903c-5acdef5dc10478cee5291df1596ec66a.ssl.cf1.rackcdn.com/742065/9/check/devstack-plugin-ceph-tempest-py3/292fd21/testr_results.html | |
| 20:09:17 | melwitt | ah yeah. and this is the tempest job failure I was looking at https://1ca2ee7583d21788b1d8-42b9b3ca9891e58d539431fcfb5b799d.ssl.cf2.rackcdn.com/742836/2/check/devstack-plugin-ceph-tempest-py3/547fab7/testr_results.html | |
| 20:09:27 | melwitt | (I picked a recent patch proposed to the tempest repo) | |
| 20:09:37 | dansmith | cool | |
| 20:09:56 | dansmith | that's awesome because of two things: | |
| 20:10:02 | dansmith | 1. I didn't break stuff | |
| 20:10:13 | dansmith | 2. I get to steal the credit from sean-k-mooney for fixing more people! | |
| 20:10:31 | melwitt | lol awww | |
| 20:10:45 | dansmith | dang, now I better give him honorable mention in the commit message ;P | |
| 20:11:08 | melwitt | hey, I found the default config value too | |
| 20:11:42 | melwitt | after that he blew past me finding how/where it was used | |
| 20:12:11 | dansmith | heh | |
| 20:12:51 | dansmith | he gave me a line to copy/paste, so... | |
| 20:13:44 | melwitt | yeah, that's what it's all about | |
| #openstack-nova - 2020-07-25 | |||
| 03:08:32 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add test coverage of hosts policies https://review.opendev.org/743013 | |
| 03:20:06 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Introduce scope_types in hosts policy https://review.opendev.org/743014 | |
| 04:10:41 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add new default roles in hosts policies https://review.opendev.org/743015 | |
| 05:02:45 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.PY2 and six.PY3 https://review.opendev.org/726512 | |
| 05:03:06 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.add_metaclass https://review.opendev.org/726565 | |
| 05:03:18 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.reraise https://review.opendev.org/726898 | |
| 05:03:34 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.moves https://review.opendev.org/727224 | |
| 05:03:48 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.iteritems/itervalues/iterkeys https://review.opendev.org/727757 | |
| 05:12:37 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.byte2int/int2byte https://review.opendev.org/727777 | |
| 05:13:03 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.binary_type/integer_types/string_types https://review.opendev.org/728094 | |
| 05:13:24 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (1/2) https://review.opendev.org/728109 | |
| 05:13:41 | openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (2/2) https://review.opendev.org/728117 | |
| 21:20:04 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add new default roles in hosts policies https://review.opendev.org/743015 | |
| 21:55:51 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add test coverage of extensions policies https://review.opendev.org/743045 | |
| 22:09:22 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add scope and new default roles in extensions policies https://review.opendev.org/743046 | |
| #openstack-nova - 2020-07-26 | |||
| 18:39:27 | openstackgerrit | Lee Yarwood proposed openstack/nova master: nova-live-migration: Add encrypted volume to BFV evacuation test https://review.opendev.org/743067 | |
| #openstack-nova - 2020-07-27 | |||
| 01:53:15 | openstackgerrit | Keigo Noha proposed openstack/nova master: Change default num_retries for glance to 3 https://review.opendev.org/740389 | |
| 07:52:32 | bauzas | good morning Nova | |
| 08:16:55 | stephenfin | o/ | |
| 08:33:25 | kashyap | (Not that this kind of info be mentioned openly on a publicly logged channel. Where is my good sense...) | |
| 08:49:26 | openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: YAML file loading and schema validation https://review.opendev.org/673341 | |
| 08:49:45 | openstackgerrit | Luyao Zhong proposed openstack/nova master: vpmem enhancement for resource cleanup https://review.opendev.org/714653 | |
| 08:54:22 | tony_su | alex_xu: stephenfin: I updated the first patch for Provider Config File accordingly and answered all comments.. It would be grateful for you to spare some time to make a new round of review. Thanks. | |
| 08:54:55 | tony_su | I am going on the remaining patches ... | |
| 09:33:12 | stephenfin | tony_su: Hey o/ I've left a new review on that. Much better, but it still needs some work. Let me know if anything is unclear :) | |
| 09:57:16 | stephenfin | sean-k-mooney: did ye get to the bottom of the failing ceph-multistore job last week? | |
| 09:59:15 | stephenfin | lyarwood: Care to look at this again and possibly fast approve in gibi's absence (a trivial merge conflict resulted in loss of votes) https://review.opendev.org/#/c/742651/ | |
| 10:03:30 | brinzhang | stephenfin: there is a patch about nova-cyborg internaction, hope you can review https://review.opendev.org/#/c/716186/ | |
| 10:04:28 | brinzhang | stephenfin: gibi has +2 for that, and sean-k-mooney is one of the modifiers. | |
| 10:11:18 | stephenfin | brinzhang: I can try but realistically I know very little about Cyborg, so I don't know how much help I'll be. bauzas or similar might be a better shout | |
| 10:11:24 | lyarwood | stephenfin: ./me looks | |
| 10:12:55 | brinzhang | stephenfin: thanks, I will ask bauzas and dansmith give some help too. | |
| 10:14:06 | brinzhang | bauzas, dansmith: hope you can review the nova-cyborg interaction patch, and there is the cyborg operatcion support patch (evacuate, shelve/unshelve patch), thanks | |
| 10:16:21 | lyarwood | stephenfin: https://review.opendev.org/#/c/742961/ - re ceph-multistore, that's the fix. | |
| 10:25:34 | sean-k-mooney | stephenfin: yes we swapped to the bluestore backend and created a 10G backing file inside the normally backing file and were just running out of space | |
| 10:25:40 | stephenfin | lyarwood: thanks | |
| 10:26:22 | sean-k-mooney | stephenfin: so now we are setting the undocumented test only config option in ceph to set it to loopback device -4g | |
| 10:26:25 | sean-k-mooney | so 20G | |
| 10:26:32 | sean-k-mooney | that seam to solve the issue | |
| 10:27:24 | stephenfin | makes sense | |
| 10:27:25 | sean-k-mooney | brinzhang: ill review the cyborg patch again now | |
| 10:27:40 | lyarwood | sean-k-mooney: https://github.com/openstack/devstack/blob/e799770f59dfa81c78599bc4313503815b210b52/.zuul.yaml#L343 - any idea if we can move that into the ceph plugin .zuul.yaml? | |
| 10:28:26 | lyarwood | I'd like to add /etc/ceph/ to the list but adding it in devstack feels wrong | |