| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-01-21 | |||
| 14:40:46 | dansmith | lyarwood: ^ this tries to hit them too | |
| 14:43:04 | sean-k-mooney | hehe i like the title | |
| 14:43:25 | dansmith | I dunno why that has started failing all the sudden, | |
| 14:43:32 | dansmith | like if a name changed and we started running those or something | |
| 14:43:46 | dansmith | but they end up putting an image int he wrong glance repo, which doesn't work | |
| 14:44:46 | dansmith | oh, heh | |
| 14:44:57 | sean-k-mooney | https://github.com/openstack/cinder-tempest-plugin/commit/5ab3fef76283ca91b92fa130ba2685886bd8af68 | |
| 14:44:58 | dansmith | looks like there's already something in gate | |
| 14:45:00 | sean-k-mooney | maybe that | |
| 14:45:10 | dansmith | sean-k-mooney: yuuup | |
| 14:46:08 | sean-k-mooney | so ya just new test added | |
| 14:47:38 | dansmith | lyarwood: sorry, just saw yours from 1am.. I had half figured this out last night but ran out of time, thanks for grabbing that | |
| 14:47:52 | dansmith | that makes three distinct and legit non-nova, non-glance issues with that job in a week :) | |
| 14:53:12 | sean-k-mooney | well at least there is now testing for this so its still progress | |
| 14:53:26 | sean-k-mooney | even if there are teething issues | |
| 14:59:12 | dansmith | lyarwood: I assume that snuck in because we run the whole tempest-cinder-plugin on that job, but they don't gate on it.. so my first thought was just to not load their tempest plugin in there to prevent this from happening in the future.. thoughts on that? | |
| 14:59:44 | dansmith | sean-k-mooney: yes, especially good when they're actual problems and not just flaky or unreliable tests | |
| 15:03:19 | lyarwood | dansmith: morning, sorry was afk | |
| 15:03:51 | lyarwood | dansmith: yeah correct, I was going to see if we could add this job into their gate tbh | |
| 15:03:56 | dansmith | lyarwood: np, it took a night of sleep to realize that test was from a tempest plugin, thanks for nailing that whilst I slept | |
| 15:04:02 | dansmith | lyarwood: that also works | |
| 15:04:50 | lyarwood | so what's the deal with these project specific tempest plugins | |
| 15:05:01 | lyarwood | from what I've seen most of this should live in core tempest anyway | |
| 15:05:03 | dansmith | I dunno, asking for trouble :) | |
| 15:05:21 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Replace blind retry with libvirt event waiting in detach https://review.opendev.org/c/openstack/nova/+/770246 | |
| 15:05:29 | lyarwood | hehe | |
| 15:05:52 | lyarwood | I just don't get the need for loads of different test repos all doing the same time tbh, I'll go away and read up | |
| 15:05:54 | dansmith | without co-gating it's just an attack vector | |
| 15:12:56 | lyarwood | dansmith: multistore attempts to cover some edge usecases right? | |
| 15:21:06 | dansmith | lyarwood: the whole arrangement in devstack is edgy yeah | |
| 15:21:49 | lyarwood | dansmith: cool, so looking at the actual test that borked here this might be something we can fix on their side | |
| 15:22:16 | lyarwood | dansmith: for whatever reason the test is creating a unique image for the instance to use | |
| 15:22:41 | lyarwood | dansmith: when it really isn't required as the test is about cloning an encrypted volume attached to the instance | |
| 15:23:06 | dansmith | yeah, and it creates it in the wrong glance store, which is the problem | |
| 15:23:06 | lyarwood | I'll submit a few changes to the cinder plugin before adding our job to their gate somehow | |
| 15:23:18 | lyarwood | we might be able to drop the blacklist around encrypted volumes after all | |
| 15:23:26 | lyarwood | skiplist* | |
| 15:23:36 | dansmith | brownlist | |
| 15:23:40 | dansmith | like, turds | |
| 15:25:31 | lyarwood | ^_^ | |
| 15:26:02 | spatel | noonedeadpunk: In your designate example where is pool_id? http://paste.openstack.org/show/801787/ | |
| 15:26:55 | noonedeadpunk | o_O | |
| 15:27:01 | spatel | i had issue that issue and when i put pool id then it works | |
| 15:27:08 | spatel | what is the workaround for that? | |
| 15:27:47 | spatel | does OSA generate random pool id ? (because default isn't valid) | |
| 15:28:43 | spatel | sorry folks wrong channel | |
| 15:34:57 | sean-k-mooney | i had issue with the pool id in kolla ansible too | |
| 15:35:37 | sean-k-mooney | for me i had to both mess with the config and the db to eventulaly get it to work | |
| 15:36:00 | sean-k-mooney | but i remmeber id did not jsut work for me when it was deployed | |
| 15:36:38 | sean-k-mooney | i added | |
| 15:36:40 | sean-k-mooney | [service:central] | |
| 15:36:43 | sean-k-mooney | default_pool_id = 6a67bd3a8e654ddeb3675b8534899f16 | |
| 15:36:45 | sean-k-mooney | workers = 5 | |
| 15:36:54 | sean-k-mooney | and then i think i needed to edit the db to make it match | |
| 15:37:03 | sean-k-mooney | it was strange | |
| 15:46:04 | openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Allow encryption tests in nova-ceph-multistore again https://review.opendev.org/c/openstack/nova/+/771837 | |
| 15:46:09 | lyarwood | dansmith: ^ hopefully this work | |
| 15:46:33 | lyarwood | that links through to a fix and another change adding the job to gate of the tempest plugin | |
| 15:47:18 | lyarwood | ah I think I also need to fix some core tempest tests | |
| 15:47:21 | lyarwood | my bad | |
| 15:47:41 | dansmith | yeah, there are core tests too, and that's why last night I was like "wtf, how is this just now breaking when nothing has changed here" | |
| 15:48:53 | lyarwood | ack kk, I'll let this fail and then fix them up, it's likely the same bogus image creation copy pasted between tempest and the plugin tbh | |
| 15:49:21 | dansmith | wouldn't be surprised, yeah | |
| 15:49:28 | dansmith | which is further support for "why is this separate?" | |
| 15:49:44 | lyarwood | yup | |
| 15:50:48 | gibi | nova weekly meeting starts in 10 minutes in #openstack-meeting-3 | |
| 15:50:55 | bauzas | gibi: given the curfew we have at 6pm, I'll stay in the meeting until 5.30pm our time | |
| 15:54:52 | gibi | bauzas: O | |
| 15:54:53 | gibi | K | |
| 16:06:39 | dansmith | hey sean-k-mooney, you like devstacking.. would you run this locally and compare before/after numbers? https://review.opendev.org/c/openstack/devstack/+/771505 | |
| 16:07:57 | sean-k-mooney | " know this is going to be controversial, but hear me out." L) | |
| 16:08:03 | sean-k-mooney | * :) | |
| 16:08:08 | sean-k-mooney | sure i can try that | |
| 16:08:20 | dansmith | I actually maybe should remove that, | |
| 16:08:30 | dansmith | because so far it has been super stable for me, and easy to debug things | |
| 16:08:45 | dansmith | initial attempt interleaved all the parallel logging, which I knew would be hard to swallow for people | |
| 16:09:01 | sean-k-mooney | we still have some native calls to osc by the way that dont use osc wrap | |
| 16:09:05 | dansmith | but now it looks very clean, just potentially different orderings | |
| 16:09:14 | sean-k-mooney | if we fixed up those it would also help | |
| 16:09:19 | dansmith | sean-k-mooney: oscwrap is just for time accounting | |
| 16:09:23 | dansmith | it doesn't make it any faster | |
| 16:09:38 | dansmith | and it sets up an alias, so everything should be using it, AFAI | |
| 16:09:39 | sean-k-mooney | waith i tought it used osc in itneractive mode via a socket | |
| 16:09:44 | dansmith | no | |
| 16:09:50 | sean-k-mooney | did we not merge danpbs prototype of that | |
| 16:09:53 | sean-k-mooney | oh ok | |
| 16:10:32 | dansmith | I tried doing that myself this week, but getting the return code is going to be non-trivial and we need that in a *lot* of places in devstack | |
| 16:10:55 | sean-k-mooney | well ya ill pull that into one of my vms and let it run | |
| 16:11:01 | dansmith | we do lots of single-threaded things in devstack, waiting on osc is just the dumbest one :) | |
| 16:11:12 | dansmith | sean-k-mooney: ooh, thanks I'd definitely appreciate run in a vm | |
| 16:11:19 | dansmith | I'm on baremetal | |
| 16:12:00 | sean-k-mooney | well i ment run it in one of my openstack vms so that will be similar to the upstream ci runs | |
| 16:12:08 | dansmith | that's cool | |
| 16:12:43 | dansmith | performance in the actual nodepool workers varies so much from minute to minute it's hard to prove with linkable numbers that it's better | |
| 16:12:48 | dansmith | so more anecdotal evidence would be good | |
| 16:14:03 | dansmith | I'm also on spinning rust currently, and I feel like I could be getting better parallel performance by overlapping the db syncs if I wasn't | |
| 16:14:23 | sean-k-mooney | i rebooted my cloud at the weekend so need to start up my third paryt ci again but i can pproably have it do a few full tempest runs too but ill take a look at that spereatly | |
| 16:14:46 | sean-k-mooney | since my thridparty ci just runs on one host its pretty consitnet | |
| 16:14:55 | dansmith | so far it hasn't failed a tempest run, so I'm pretty confident that it's still building a consistent devstack | |
| 16:16:33 | artom | Do we... do we have a config option to log in JSON somehow? | |