| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-04-13 | |||
| 19:00:17 | jgwentworth | hm, okay. so that means they didn't run nova-manage online_data_migrations? | |
| 19:00:28 | jgwentworth | no, not deleted records | |
| 19:00:38 | cfriesen | jgwentworth: pretty sure we ran it but it would have skipped the deleted records | |
| 19:00:41 | superdan | [11:54:35] <cfriesen>jgwentworth: do you know why we didn't just apply uuids to deleted services for consistency? | |
| 19:00:44 | jgwentworth | existing non-deleted service records that lacked a service UUID | |
| 19:00:50 | jgwentworth | oh, I missed that | |
| 19:01:09 | cfriesen | jgwentworth: we'll try that fix you pointed me at...hopefully that solves the on-demand issue | |
| 19:01:16 | superdan | cfriesen: for instance-related things, skipping deleted saves us a ton of work of course | |
| 19:01:31 | superdan | maybe less on services, but .. if there's no way to undelete a service there's not much point | |
| 19:01:42 | jgwentworth | okay, yeah if they service was in the deleted state then it could be skipped. but how could an active instance have a deleted service record? | |
| 19:01:46 | cfriesen | superdan: yeah, I get that. just bad luck that it triggered another bug | |
| 19:02:03 | jgwentworth | I guess that goes back to our discussion from the other day | |
| 19:02:09 | superdan | jgwentworth: we just went over that you can delete services even when instances are on it right? :) | |
| 19:02:10 | jgwentworth | we allow service delete while instances are alive | |
| 19:02:10 | superdan | yeah that | |
| 19:02:16 | cfriesen | jgwentworth: boot instance, migrate it, delete service, upgrade, add new service with same name, migrate instance | |
| 19:02:33 | superdan | or just boot instance, delete service | |
| 19:02:38 | jgwentworth | okay, I didn't realize that was a thing | |
| 19:03:02 | superdan | jgwentworth: remember me saying I thought it was probably bad to allow this anyway? :P | |
| 19:03:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Make service all-cells min version helper use scatter-gather https://review.openstack.org/539590 | |
| 19:03:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Make host_manager use scatter-gather and ignore down cells https://review.openstack.org/539617 | |
| 19:03:06 | jgwentworth | cfriesen: so you would have a problem if we made it so services can't be deleted if they have running instances? | |
| 19:03:17 | mriedem | superdan: i've got a patch up for that, testing it in devstack atm | |
| 19:03:21 | jgwentworth | superdan: yes | |
| 19:03:24 | mriedem | https://review.openstack.org/#/c/560674/1 | |
| 19:04:12 | cfriesen | jgwentworth: I wouldn't, no. In our case we did the sequence I described...the issue is that the back reference from instance to service is by host name | |
| 19:04:29 | cfriesen | so if you delete a service then make another with the same name, they'll both get matched | |
| 19:04:57 | superdan | which is another reason why that hostname-based keying is kinda bad | |
| 19:05:36 | jgwentworth | oh, okay, so because you migrated the instance already the service delete would be okay with you as no running instances left on it | |
| 19:05:45 | cfriesen | yep | |
| 19:05:54 | jgwentworth | k, cool | |
| 19:29:19 | jgwentworth | superdan: on this patch about multi-cell query for instance group members, mriedem and I had some discussion over whether or not I should make changes to limit the query to the cell we're already in, if it's a move operation. we wanted to get your opinion if there's some reason not to do that, that we might have missed https://review.openstack.org/#/c/540258 | |
| 19:29:23 | mriedem | nice http://paste.openstack.org/show/719188/ shows that n-api doesn't barf when trying to delete allocations during server delete if the n-cpu is down and [placement] isn't configured | |
| 19:42:14 | imacdonn | mriedem: FYI, in case you run into it somewhere .... after re-enabling UCA for Oracle ZFSSA CI, I hit this - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=891315 - 'apt-get upgrade' installed a 'libnettle6 amd64 3.4-1~cloud0' (from the UCA), and i'm hoping that'll fix it | |
| 19:42:15 | openstack | Debian bug 891315 in dnsmasq "[dnsmasq] libnettle6 dependency" [Normal,Fixed] | |
| 19:43:24 | mriedem | jgwentworth: https://review.openstack.org/#/c/557796/ lgtm, i'll +2 once you fix that typo | |
| 19:44:30 | jgwentworth | mriedem: thanks | |
| 19:44:33 | mriedem | imacdonn: which UCA? | |
| 19:44:35 | mriedem | pike or queens? | |
| 19:44:48 | imacdonn | mriedem: queens ... it's devstack master | |
| 19:45:01 | mriedem | imacdonn: haven't had any issues like that here https://review.openstack.org/#/c/554317/ | |
| 19:45:20 | imacdonn | mriedem: perhaps you got the new libnettle6 by some other means | |
| 19:45:30 | mriedem | idk, infra builds the images | |
| 19:45:55 | imacdonn | mriedem: OK, well, just a heads-up in case you see it | |
| 19:46:04 | mriedem | ack | |
| 19:46:29 | imacdonn | other than that, ZFSSA CI seems to work (with multiattach) with the UCA | |
| 19:46:47 | mriedem | outstanding | |
| 19:47:55 | superdan | jgwentworth: sure seems reasonable | |
| 19:48:12 | jgwentworth | k, thanks | |
| 19:49:47 | openstackgerrit | melanie witt proposed openstack/nova-specs master: Document Rocky review priorities https://review.openstack.org/557796 | |
| 19:56:34 | mriedem | any opinions on removing the redundant unit tests from tssurya's patch here? https://review.openstack.org/#/c/554920/10/nova/tests/unit/compute/test_host_api.py@439 the functional test in that change will now cover that | |
| 19:57:54 | openstackgerrit | Jay Pipes proposed openstack/nova master: support multiple member_of qparams for GET /r-ps https://review.openstack.org/561315 | |
| 19:58:34 | leakypipes | superdan: that should help when we need >1 placement req filters... | |
| 20:00:24 | openstackgerrit | Jay Pipes proposed openstack/nova master: support multiple member_of qparams for GET /r-ps https://review.openstack.org/561315 | |
| 20:02:31 | openstackgerrit | Jay Pipes proposed openstack/nova master: support multiple member_of qparams for GET /r-ps https://review.openstack.org/561315 | |
| 20:08:41 | jgwentworth | mriedem: yeah, seems like that unit test can go | |
| 20:09:05 | jgwentworth | I don't see it covering anything additional unless I'm missing something | |
| 20:09:28 | superdan | leakypipes: that's implementing what we said int the spec revision right? | |
| 20:10:33 | leakypipes | superdan: yessir | |
| 20:10:45 | superdan | cools | |
| 20:16:10 | jgwentworth | wow, assert_called_once() is a thing. *mind blown* | |
| 20:42:44 | mriedem | using fixtures as context managers - put that in your bag of tricks too for when a test needs to conditionally use a fixture | |
| 20:43:35 | jgwentworth | I think I've done that before already with MonkeyPatch fixture | |
| 20:43:43 | jgwentworth | cause I'm cool like that | |
| 20:49:03 | mriedem | unfortunately our api fixtures might not be setup properly to handle running them as fixtures... | |
| 20:49:05 | mriedem | dagnabbit | |
| 20:55:15 | mriedem | cdent: i know you probably want to figure this out on your friday night, but it's not clear to me that we stop the wsgi app when PlacementFixture or OSAPIFixture tear down | |
| 20:56:41 | mriedem | specifically, i get a timeout b/c the request seems to hang on a connection error at L77 http://paste.openstack.org/show/719198/ | |
| 20:57:11 | mriedem | http://paste.openstack.org/show/719199/ | |
| 21:01:48 | openstackgerrit | Jackie Truong proposed openstack/nova master: Implement certificate_utils https://review.openstack.org/479949 | |
| 21:05:28 | jgwentworth | jackie-truong: hey, thanks for the quick update :) I'm going through the last patch now and to peruse the tempest test logs. sorry for the delay on review | |
| 21:06:18 | jackie-truong | jgwentworth: np! thanks :-) | |
| 21:21:29 | openstackgerrit | Jackie Truong proposed openstack/nova master: Add certificate validation docs https://review.openstack.org/560158 | |
| 21:22:18 | openstackgerrit | Jackie Truong proposed openstack/nova master: Add certificate validation docs https://review.openstack.org/560158 | |
| 21:34:37 | cdent | mriedem: Imma gonna need you to parse that a bit more for me, or I can think harder about it some other time, but at the moment, I don't get what you're saying supposed to be happening there | |
| 21:35:42 | mriedem | i'm just going to leave it as a note in my test for now, | |
| 21:35:51 | mriedem | but in that test i'm using the placement fixture and the compute api fixture, | |
| 21:36:03 | mriedem | but i need the placement fixture to be used conditionally based on what i'm doing, | |
| 21:36:25 | mriedem | so i use it when i'm creating a server because the compute service needs it, but then i don't want placement fixture in place when i delete the server | |
| 21:36:36 | mriedem | but when i go to delete the server i get this weird connection error, | |
| 21:36:48 | mriedem | so i'm not sure if/why the placement fixture is messing with the osapifixture | |
| 21:37:06 | mriedem | i figured maybe the placement fixture isn't cleaning up after itself properly | |
| 21:37:23 | cdent | that a url for the compute api though? | |
| 21:38:11 | mriedem | Max retries exceeded with url: /v2.1/6f70656e737461636b20342065766572//servers/ee29ce1f-ca1e-4466-84cd-23d57bb70206 | |
| 21:38:11 | mriedem | yeah | |
| 21:38:54 | cdent | is there a check behind that compute api call for the placement api? that is, at that step of the game, how is placement involved? | |
| 21:38:59 | mriedem | wsgi-intercept is working on unique endpoints between placement and osapifixture so i'm not sure how things could overlap | |
| 21:39:20 | cdent | is there any asynchrony/eventlet/similar magic going on? | |
| 21:39:22 | mriedem | the delete flow in this patch now adds a call to the scheduler report client to delete allocations for the instance | |
| 21:39:36 | mriedem | which i was hoping would bomb out and @safe_connect would handle it | |
| 21:39:49 | mriedem | since placement fixture wouldn't be in place when the server delete call happens | |
| 21:40:02 | mriedem | but...it's likely trying to hit ksa...? | |
| 21:40:47 | cdent | if you push the code up somewhere and add me to the review I'll look at it tomorrow (eagerly even, because I'm curious) | |
| 21:41:21 | mriedem | ok; i might know how to do this w/o the ugly stub i've got in place too, will test | |
| 21:41:23 | cdent | it's entirely possible there are some global bits in wsgi-intercept being naughty (it is old and gross) | |
| 21:41:37 | cdent | and if we can find them that would be a nice fix | |
| 21:42:27 | cdent | if nothing else, I'll play with the paste tomorrow | |
| 21:43:15 | mriedem | it's bugging me so i'll have something up tonight most likely and just add yo uto the review | |
| 21:43:58 | cdent | I'm going to not get sucked in for of fear of being glared at by family | |
| 21:54:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Block deleting compute services which are hosting instances https://review.openstack.org/560674 | |