| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-10-04 | |||
| 14:03:41 | mriedem | and yeah when you're admin listing 1K all_tenants instances we're going to ask neutron for the combined set of ports and security groups for all of them | |
| 14:03:53 | dansmith | I don't see what a note would hurt, given not everyone would piece together what is happening where, but... whatever, we can talk about it in the patch | |
| 14:04:26 | mriedem | i just have no idea what the limits would be, so wouldn't want to say "you'll likely timeout" | |
| 14:04:51 | dansmith | sure, it'd just be a warning about "too many" | |
| 14:05:10 | dansmith | whatever, I'll stop trying to address your concerns :) | |
| 14:05:12 | mriedem | "Since the number of images to cache is not rate limited use caution when sending a large set of images." | |
| 14:05:13 | mriedem | something like that | |
| 14:05:36 | mriedem | anyway | |
| 14:05:52 | dansmith | or I could push validation to the conductor | |
| 14:06:21 | mriedem | failing fast in the API is nice though... | |
| 14:06:23 | mriedem | for this kind of thing | |
| 14:06:30 | dansmith | so basically, | |
| 14:06:40 | dansmith | you will not be pleased with any outcome, is that right mr. midwest? | |
| 14:06:50 | mriedem | that's my general stance with anything in my life yes | |
| 14:06:56 | dansmith | right, got it :) | |
| 14:07:38 | mriedem | laura was all riled up about this impeachment stuff last night and i had to remind her to keep expectations very low and assume the worst in whatever outcome she wants - like i do! | |
| 14:07:40 | mriedem | it's very healthy | |
| 14:07:51 | dansmith | heh | |
| 14:09:12 | mriedem | so here is a thing to compare to in the api, | |
| 14:09:25 | mriedem | during server create i don't think we limit the number of bdms you can specify, | |
| 14:09:44 | mriedem | you can't include the same pre-existing volume more than once since we don't support multiattach that way, | |
| 14:10:04 | mriedem | but you could say, boot me 10 servers with 255 bdms each and they are all using the same image for the volume's disk, | |
| 14:10:23 | mriedem | the api will GET that image 2550 times at least | |
| 14:10:28 | mriedem | since we're too dumb to cache the results | |
| 14:11:08 | mriedem | that's in _validate_bdm called from a for loop in _provision_instances | |
| 14:13:06 | dansmith | yeah, you're saying we do that today and nobody is complaining? | |
| 14:13:23 | dansmith | obviously less likely to run afoul of that with 255 BDMs vs. images to pre-cache | |
| 14:13:29 | dansmith | but same deal | |
| 14:13:47 | dansmith | actually, that all happens in the conductor right? | |
| 14:13:49 | dansmith | used to be in the api | |
| 14:14:12 | mriedem | no it's in the api | |
| 14:14:22 | mriedem | and yeah no one is complaining about that right now | |
| 14:14:23 | dansmith | okay | |
| 14:14:46 | mriedem | i remember the power boyz really wanting a couple hundred volumes attached per server | |
| 14:14:58 | mriedem | so do that + multicreate | |
| 14:15:42 | dansmith | yeah | |
| 14:18:58 | dansmith | mriedem: well, we have the new imagecache group, maybe we just chuck a configurable limit in there.. no real reason not to I guess | |
| 14:20:10 | mriedem | config-driven api is the only reason not to... | |
| 14:20:21 | dansmith | yup | |
| 14:20:24 | mriedem | which is why i was thinking of quota config etc | |
| 14:26:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add boot from volume functional test with a huge request https://review.opendev.org/686734 | |
| 14:32:34 | openstackgerrit | Eric Fried proposed openstack/nova-specs master: Add 'Core Liaison' spec process https://review.opendev.org/685857 | |
| 14:33:27 | efried | updated to make dansmith his own liaison on his precaching spec, and took out the idioms gibi called me on ^ | |
| 14:33:49 | efried | also clarified self-liaison wording can just be "None" as bauzas suggested ^ | |
| 14:49:00 | efried | aspiers: update on SEV fix? | |
| 14:49:55 | stephenfin | mriedem: I'm reworking the NeutronFixture to mock out the security group-related APIs instead. It's going to be much larger but seems more correct | |
| 14:50:21 | efried | aspiers: I'm gonna say by... Tuesday we will merge the "document it's busted" patch if the fix isn't ready. | |
| 14:50:38 | efried | so that we can be sure it's through the gate by Thursday when we have to produce our final RC. | |
| 14:52:00 | mriedem | stephenfin: ugh....can we just do that in a follow up? | |
| 14:52:07 | mriedem | b/c that's going to slow things down quite a bit | |
| 14:52:43 | stephenfin | mriedem: We can. Want me to just fix that commit message comment and merge as-is so? | |
| 14:53:10 | bauzas | efried: honestly, I'm not opinionated, I just wanted to explain it would look a bit weirdo if the same name would be for both the owner and the liaison | |
| 14:53:28 | bauzas | efried: something either "None" or "Not needed" is cool with me | |
| 14:53:33 | efried | bauzas: I agree, I tried to explain it in the FAQ | |
| 14:53:40 | mriedem | stephenfin: if you're going to rev the commit message, i think you could also the thing i asked for in the stub which is only return a default security group for an instance that has a non-empty info_cache | |
| 14:53:44 | mriedem | stephenfin: i think that's as easy as: | |
| 14:53:48 | efried | bauzas: and yes, latest PS has "or None" in the template | |
| 14:53:57 | bauzas | ok, will look at it then | |
| 14:54:02 | efried | thx | |
| 14:54:16 | mriedem | if instance.get_network_info() | |
| 14:55:46 | stephenfin | I'm not sure if can rely on that since I think we mock a lot of it. See https://github.com/stephenfin/nova/commit/afb4698e8f7e39154e90dfe775d2d0e3e1f20dbc#diff-c731e6017d325067e73f0124f4ade91c | |
| 14:55:58 | stephenfin | Is that something else though? | |
| 14:56:13 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add boot from volume functional test with a huge request https://review.opendev.org/686734 | |
| 14:56:18 | mriedem | dansmith: so ^ proves my point about the server create case, mayhap i should open a bug for that | |
| 14:56:28 | dansmith | mriedem: ack | |
| 15:00:50 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add boot from volume functional test with a huge request https://review.opendev.org/686734 | |
| 15:02:54 | mriedem | stephenfin: ok so i guess for the current patch, how about just (1) fix the commit message and (2) leave a TODO in the new stub method that it should be smarter about instances that don't have networks/ports and thus won't have security groups | |
| 15:12:57 | stephenfin | mriedem: Yup, I can do that | |
| 15:13:00 | stephenfin | Will respin now | |
| 15:22:28 | openstackgerrit | Matt Riedemann proposed openstack/nova master: nova-net: Migrate 'test_quota_sets' functional tests https://review.opendev.org/684334 | |
| 15:27:09 | mriedem | gibi: could i ask you to hit the bottom 2 FUPs in the cross-cell resize series https://review.opendev.org/#/c/676228/ since i think you're familiar with the changes they are...FUPing (aleady have a +2 on them) | |
| 15:27:58 | gibi | mriedem: sure, looking... | |
| 15:29:42 | mriedem | dansmith: what do you think about getting this CellTimeout logging traceback change in to debug the gate failure? https://review.opendev.org/#/c/684118/ - i'm unable to recreate on that change itself after many rechecks but it's one of the top gate failures: http://status.openstack.org/elastic-recheck/#1844929 | |
| 15:30:59 | dansmith | mriedem: sure | |
| 15:33:09 | gibi | efried: so https://blueprints.launchpad.net/nova/+spec/support-move-ops-with-qos-ports is in Implemented state. So I think I need to open a new bp for U about the rest of the move operations (live migrate, evacuate, unshelve) and then propose a spec that is a slimmed down version of the original spec https://review.opendev.org/#/c/652608/ Do you agree? | |
| 15:33:25 | mriedem | i did get the mysqld logs published for these failing grenade jobs and it looks like we do start mysqld at least 3 times during the grenade run which i didn't expect | |
| 15:33:37 | mriedem | we can see things really start to go south about the time that we hit the cell timeout failures | |
| 15:33:38 | mriedem | https://zuul.opendev.org/t/openstack/build/4085120e390f4f1e971c6ff61304a596/log/logs/mysql/error.txt.gz#213 | |
| 15:33:54 | dansmith | mriedem: yeah that seems weird | |
| 15:34:02 | efried | gibi: That would be fine. If you prefer I can un-implement the train bp and you can reuse it. I don't really care too much. But I think the old process hounds have a preference for creating a new bp (-ussuri suffix). | |
| 15:34:03 | dansmith | mriedem: are we actually upgrading mysql packages? | |
| 15:34:08 | mriedem | so i guess 2 starts would mean...old devstack and new devstack | |
| 15:34:11 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Correctly mock out security groups in NeutronFixture https://review.opendev.org/685927 | |
| 15:34:15 | stephenfin | mriedem: ^ | |
| 15:34:25 | stephenfin | think that's what you were looking for | |
| 15:36:18 | gibi | efried: OK, I will go with a new one. thanks | |
| 15:37:50 | mriedem | dansmith: we're not upgrading packages, same version for all 3 starts: | |
| 15:37:51 | mriedem | mysqld (mysqld 5.7.27-0ubuntu0.18.04.1) starting as process | |
| 15:38:06 | dansmith | mriedem: so why are we restarting it at all? | |
| 15:38:41 | mriedem | for one old and new devstack will just blindly try to restart the service i think | |
| 15:38:47 | mriedem | i need to dig into the grenade log | |
| 15:38:49 | dansmith | odd | |
| 15:39:15 | sean-k-mooney | dansmith: we also could be tweaking the mysql config between release | |
| 15:39:25 | dansmith | mriedem: unrelated, remind me.. are we not supposed to 404 for things like specifying a missing image id in my request? | |
| 15:39:35 | sean-k-mooney | although i dont think we actully do set much in grenade | |
| 15:39:36 | dansmith | sean-k-mooney: possible, but not sure what that would be | |
| 15:39:51 | mriedem | dansmith: 400 | |
| 15:39:51 | dansmith | sean-k-mooney: but maybe some implicit restart in case it has changed or something | |
| 15:39:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Correctly mock out security groups in NeutronFixture https://review.opendev.org/685927 | |
| 15:39:55 | dansmith | mriedem: okay | |