Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-12-11
15:06:07 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Add functional regression test for bug 1806064 https://review.openstack.org/623935
15:06:07 openstack bug 1806064 in OpenStack Compute (nova) queens "Volume remains in attaching/reserved status, if the instance is deleted after TooManyInstances exception in nova-conductor" [Medium,In progress] https://launchpad.net/bugs/1806064 - Assigned to Matt Riedemann (mriedem)
15:10:26 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Create BDMs/tags in cell with instance when over-quota https://review.openstack.org/623937
15:10:39 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Create BDMs/tags in cell with instance when over-quota https://review.openstack.org/623937
15:12:15 mriedem dansmith: looks like https://review.openstack.org/#/c/623282/ last failed on the test added,
15:12:24 mriedem so we likely need to reset the global somewhere more....global
15:12:31 mriedem maybe in the test setUp?
15:12:54 mriedem or in the base TestCase setup where we reset other globals?
15:13:10 dansmith mriedem: ah, I applied a fix for that to the next patch by accident I think
15:13:22 dansmith I don't think we need to reset it anywhere other than just in that test
15:14:49 stephenfin sean-k-mooney: RE: https://bugs.launchpad.net/nova/+bug/1805891, this is where things break down https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L209-L210
15:14:49 openstack Launchpad bug 1805891 in OpenStack Compute (nova) "pci numa polices are not followed" [High,Triaged] - Assigned to sean mooney (sean-k-mooney)
15:15:15 stephenfin Looks like we're not persisting that field in the DB :-\
15:16:34 openstackgerrit Dan Smith proposed openstack/nova master: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/623282
15:16:35 openstackgerrit Dan Smith proposed openstack/nova master: Make compute rpcapi version calculation check all cells https://review.openstack.org/623284
15:16:35 openstackgerrit Dan Smith proposed openstack/nova master: Make service.get_minimum_version_all_cells() cache the results https://review.openstack.org/623283
15:16:41 dansmith mriedem: moved that down a patch, mah bad
15:18:47 mriedem ack
15:26:21 openstackgerrit Chris Dent proposed openstack/nova master: Add python 3.7 unit and functional tox jobs https://review.openstack.org/624055
15:26:54 cdent gibi: there are some functional test failures related to waiting for notifications in ^
15:27:22 cdent dansmith, mriedem: that ^ fixes some unit test failures in py37 and a real issue in the multi cell query stuff with StopIteration
15:28:07 gibi cdent: ack
15:28:34 cdent gibi: I haven't investigate the failures closely yet, but I'm assuming some kind of deadlock
15:29:02 gibi cdent: I made a note to look at it at some point
15:29:10 mriedem moar jubz
15:29:41 gibi cdent: is it in PS3 as PS2 passed the functional jobs
15:29:41 mriedem is there a tl;dr from the tc on when we're going to stop running 20 different python unit test/functional jobs per change?
15:29:46 cdent mriedem: i have it on good authority from clarkb that unit test jobs are not "the problem"
15:30:03 mriedem we still have race failures in py jobs that reset the gate
15:30:16 mriedem http://status.openstack.org/elastic-recheck/#1793364
15:30:25 mriedem http://status.openstack.org/elastic-recheck/#1800472
15:30:48 mriedem just wondering if at some point py27 goes to experimental queue or something, and py35 gets dropped
15:30:48 cdent mriedem: heh, on the tl;dr: I'm not really sure as I lost interest when "stop running 20 different..." was taken off the table
15:31:08 cdent I, perhaps, wanted somethign a bit too aggressive: one distro, one python version
15:31:13 cdent but you know
15:31:15 cdent <- me
15:31:42 cdent mriedem: anyway, if we don't want another gate job, that's fine, but we still need the tox jobs and the bug fixes are real (and backwards compatible)
15:31:42 mriedem i'm just not sure why we couldn't move py27 stuff to experimental,
15:31:49 mriedem so we can test fixes for py27, but not gate on it for every change
15:32:11 mriedem sure i'm just in this mindset about the broken gate
15:32:57 cdent in the discussions that I was party to, people were wed to this notion of "testing what people are using (now)" rather than "testing what people will be using when the code is released"
15:33:04 cdent given the former, 2.7 matters
15:34:13 cdent mriedem: I'd be super keen on moving 2.7 to experimental
15:34:37 mriedem commented in the change
15:34:49 mriedem you can say to your tc buddies, "the troll hath spoken in my patch"
15:35:18 mriedem if 2.7 matters, can we decide which of the 20 py3 versions matters?
15:35:24 mriedem and move the others to experimental?
15:36:39 mriedem speaking of gate bug fixes, need another +2 on https://review.openstack.org/#/c/615347
15:39:09 coreycb cdent: will placement be removed from nova in stein?
15:39:40 cdent coreycb: that's the goal, yes
15:40:15 cdent coreycb: that's part of what is driving the goals in http://lists.openstack.org/pipermail/openstack-dev/2018-September/134541.html
15:40:23 coreycb cdent: ok. I just won't be able to get the new placment package into main (ie. get security support etc) until it's removed from nova.
15:40:46 cdent coreycb: depending on who you talk to, we're relatively close
15:41:03 coreycb cdent: ok :)
15:41:16 cdent this needs to merge https://review.openstack.org/#/c/617941/ and then it's child can be considered
15:42:38 coreycb cdent: seems pretty close if that's the case. thanks, I'll keep an eye on it.
15:44:11 cdent thank you
15:45:51 openstackgerrit Merged openstack/nova stable/queens: Add functional regression test for bug 1806064 https://review.openstack.org/623933
15:45:51 openstack bug 1806064 in OpenStack Compute (nova) queens "Volume remains in attaching/reserved status, if the instance is deleted after TooManyInstances exception in nova-conductor" [Medium,In progress] https://launchpad.net/bugs/1806064 - Assigned to Matt Riedemann (mriedem)
15:46:05 openstackgerrit Silvan Kaiser proposed openstack/nova master: Exec systemd-run without --user flag in Quobyte driver https://review.openstack.org/554195
15:48:43 cdent melwitt: I've proposed some gerrit acl config changes to os-resource-classes and os-traits in https://review.openstack.org/624387 which will need your +1 (if you're in fact +1). The justification is effectively: why wait?
15:53:23 mriedem huh, this would probably be annoying for anyone pointing nova-api at cell0 for [database]/connection http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22The%20nova-osapi_compute%20service%20version%20is%20from%20before%20Ocata%5C%22%20AND%20tags%3A%5C%22screen-n-api.txt%5C%22&from=7d
15:59:53 mriedem dansmith: looks like something must have regressed this recently since we're seeing it in master and queens only http://logs.openstack.org/82/623282/2/check/tempest-full/0a4f66a/controller/logs/screen-n-api.txt.gz?level=WARNING#_Dec_11_06_09_01_928385 - not sure why tests aren't failing though, since when we hit that warning we are pulling instances from cell0 and not even looking for an instance mapping
16:00:52 ShilpaSD gibi: Hi
16:02:29 dansmith mriedem: hmm, with creating the record or looking for them I wonder?
16:04:54 mriedem don't really understand since nova-osapi_compute service should be created in the cell0 db here https://github.com/openstack/nova/blob/master/nova/api/openstack/wsgi_app.py#L59
16:05:14 mriedem and that would use https://github.com/openstack/nova/blob/master/nova/objects/service.py#L219
16:09:52 mriedem wtf, same request, we lookup the instance twice, first time we don't get that warning
16:09:52 mriedem http://logs.openstack.org/82/623282/2/check/tempest-full/0a4f66a/controller/logs/screen-n-api.txt.gz#_Dec_11_06_09_01_468549
16:10:05 mriedem 2nd time we do http://logs.openstack.org/82/623282/2/check/tempest-full/0a4f66a/controller/logs/screen-n-api.txt.gz#_Dec_11_06_09_01_919760
16:10:58 openstackgerrit Chris Dent proposed openstack/nova master: Add python 3.7 unit and functional tox jobs https://review.openstack.org/624055
16:11:08 cdent made 3.7 experimental
16:29:20 openstackgerrit Balazs Gibizer proposed openstack/nova master: Ensure that allocated PF matches the used PF https://review.openstack.org/623543
16:32:24 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: debug old compute API service version lookup https://review.openstack.org/624438
16:34:35 mriedem dansmith: comments/questions for your patch here https://review.openstack.org/#/c/623284/
16:34:45 cdent mriedem: do you have a working theory on ^^ or is it mostly fishing at this point
16:34:52 mriedem cdent: fishing
16:34:59 mriedem i'm surprised it doesn't break the gate
16:35:21 mriedem it == the api thinking we have old api services and looking up instances from cell0 rather than the actual cell they are in
16:35:54 mriedem unless coincidentally those instances *are* in cell0
16:36:11 mriedem or tempest just doesn't care about a 404
16:37:37 openstackgerrit Jack Ding proposed openstack/nova-specs master: Select cpu model from a list of cpu models https://review.openstack.org/620959
16:39:36 cdent mriedem: ah, okay, so you're going to log all the service creations to fill in some blanks: who is doing this!? roger
16:39:44 mriedem yeah
16:40:30 cdent good luck
16:40:45 cdent i'm trying to get the mysql opportunistic to fail, but struggling
16:43:55 dansmith mriedem: replied
17:03:35 mdbooth mriedem: Here's an interesting thing. You noted the other day that setting created_disks = False doesn't make the evacuate tests fail. After working it through it turns out that's correct behaviour for all 4 cases, and if you want to make that fail something you'd need to make the initial spawn() fail instead... Except that a spawn failure actually calls cleanup *twice*.
17:04:35 mdbooth Basically I added a similar functional test making spawn fail instead, and I was trying to debug why my Rbd disks were being deleted anyway when I was setting created_disks = False.
17:05:23 openstackgerrit Stephen Finucane proposed openstack/nova master: Fix typo https://review.openstack.org/624442
17:05:24 mdbooth It's because we call _shutdown_instance() in ComputeManager, which calls driver.destroy()
17:05:36 mdbooth mriedem: Which leaves that code in a weird limbo.
17:06:52 mdbooth That is, the code ensures the correct behaviour of _cleanup_failed_start(), but it's irrelevant because we call it again anyway.
17:10:14 mdbooth mriedem: Yeah.
17:10:29 mriedem my goal is to add tests that don't pass without the code fix,
17:10:33 mriedem so however that needs to happen...
17:11:25 mdbooth mriedem: That's kinda the thing. I wrote those, but because the code structure is weird they still didn't fail. However, if you took the code out, the resulting code would be fragile.
17:11:29 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: Store InstancePCIRequest.numa_policy in DB https://review.openstack.org/624444
17:11:29 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Call out where pci.alias should be set https://review.openstack.org/624445
17:11:33 stephenfin sean-k-mooney: ^

Earlier   Later