Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-12-11
13:12:03 sean-k-mooney anyone know how to use logstash and write querries
13:12:38 sean-k-mooney i cant get the message filtering to work
13:12:40 sean-k-mooney tags:"screen-n-cpu.txt" and message:"os_vif error: [Errno 24] Too many open files" and project:"openstack/neutron"
13:13:39 cdent sean-k-mooney: sorry, I'm no help. I generally just flail around, mostly using message: and that's it
13:14:24 sean-k-mooney for some reason its ignoreing the message section and retruning message strings without the message in it
13:15:11 cdent maybe it is confused by [ or ]?
13:15:56 sean-k-mooney maybe i tried excaping them ill try a few more things
14:18:28 openstackgerrit Matt Riedemann proposed openstack/nova master: Handle tags in _bury_in_cell0 https://review.openstack.org/621856
14:20:35 openstackgerrit Matt Riedemann proposed openstack/nova stable/queens: Create BDMs/tags in cell with instance when over-quota https://review.openstack.org/623934
14:22:10 mriedem lyarwood: ready for you https://review.openstack.org/#/q/topic:bug/1806064+branch:stable/queens
14:22:52 mriedem gibi: do you remember why we don't include tags in the instance payload in versioned notifications?
14:23:26 mriedem i guess it's on instance create and update, but not the regular action paylodas
14:23:28 mriedem *payloads
14:25:36 gibi mriedem: maybe because they doesn't change in other cases
14:35:44 gibi mriedem: maybe this helps remebering https://review.openstack.org/#/c/407228/36/nova/notifications/objects/instance.py@119
14:36:00 jangutter mriedem: thanks, and it's Kandarian, not Greek. Sincere sympathies and hope your soul remains intact.
14:36:34 gibi mriedem: I think we wanted to avoid some lazy load issues
14:53:01 mriedem gibi: yeah i figured that was the reason
14:53:07 mriedem and we could add it in later if someone needed it
14:53:22 gibi mriedem: yeah, it is not hard to add
14:53:26 mriedem reminds me i have an old patch to optimize the instance tags creation stuff since we do a lot of unnecessary db stuff
14:53:51 gibi mriedem: btw vitrage has a patch up to consume nova versioned notifications
14:54:08 gibi mriedem: so we will have one modul that trasition from legacy to versioned
14:54:14 mriedem cool - are they migrating from legacy or adding versioned from scratch?
14:54:17 mriedem ah ok
14:54:31 gibi mriedem: https://review.openstack.org/#/c/623274/
15:06:07 openstack bug 1806064 in OpenStack Compute (nova) queens "Volume remains in attaching/reserved status, if the instance is deleted after TooManyInstances exception in nova-conductor" [Medium,In progress] https://launchpad.net/bugs/1806064 - Assigned to Matt Riedemann (mriedem)
15:06:07 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Add functional regression test for bug 1806064 https://review.openstack.org/623935
15:10:26 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Create BDMs/tags in cell with instance when over-quota https://review.openstack.org/623937
15:10:39 openstackgerrit Matt Riedemann proposed openstack/nova stable/pike: Create BDMs/tags in cell with instance when over-quota https://review.openstack.org/623937
15:12:15 mriedem dansmith: looks like https://review.openstack.org/#/c/623282/ last failed on the test added,
15:12:24 mriedem so we likely need to reset the global somewhere more....global
15:12:31 mriedem maybe in the test setUp?
15:12:54 mriedem or in the base TestCase setup where we reset other globals?
15:13:10 dansmith mriedem: ah, I applied a fix for that to the next patch by accident I think
15:13:22 dansmith I don't think we need to reset it anywhere other than just in that test
15:14:49 openstack Launchpad bug 1805891 in OpenStack Compute (nova) "pci numa polices are not followed" [High,Triaged] - Assigned to sean mooney (sean-k-mooney)
15:14:49 stephenfin sean-k-mooney: RE: https://bugs.launchpad.net/nova/+bug/1805891, this is where things break down https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L209-L210
15:15:15 stephenfin Looks like we're not persisting that field in the DB :-\
15:16:34 openstackgerrit Dan Smith proposed openstack/nova master: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/623282
15:16:35 openstackgerrit Dan Smith proposed openstack/nova master: Make service.get_minimum_version_all_cells() cache the results https://review.openstack.org/623283
15:16:35 openstackgerrit Dan Smith proposed openstack/nova master: Make compute rpcapi version calculation check all cells https://review.openstack.org/623284
15:16:41 dansmith mriedem: moved that down a patch, mah bad
15:18:47 mriedem ack
15:26:21 openstackgerrit Chris Dent proposed openstack/nova master: Add python 3.7 unit and functional tox jobs https://review.openstack.org/624055
15:26:54 cdent gibi: there are some functional test failures related to waiting for notifications in ^
15:27:22 cdent dansmith, mriedem: that ^ fixes some unit test failures in py37 and a real issue in the multi cell query stuff with StopIteration
15:28:07 gibi cdent: ack
15:28:34 cdent gibi: I haven't investigate the failures closely yet, but I'm assuming some kind of deadlock
15:29:02 gibi cdent: I made a note to look at it at some point
15:29:10 mriedem moar jubz
15:29:41 mriedem is there a tl;dr from the tc on when we're going to stop running 20 different python unit test/functional jobs per change?
15:29:41 gibi cdent: is it in PS3 as PS2 passed the functional jobs
15:29:46 cdent mriedem: i have it on good authority from clarkb that unit test jobs are not "the problem"
15:30:03 mriedem we still have race failures in py jobs that reset the gate
15:30:16 mriedem http://status.openstack.org/elastic-recheck/#1793364
15:30:25 mriedem http://status.openstack.org/elastic-recheck/#1800472
15:30:48 cdent mriedem: heh, on the tl;dr: I'm not really sure as I lost interest when "stop running 20 different..." was taken off the table
15:30:48 mriedem just wondering if at some point py27 goes to experimental queue or something, and py35 gets dropped
15:31:08 cdent I, perhaps, wanted somethign a bit too aggressive: one distro, one python version
15:31:13 cdent but you know
15:31:15 cdent <- me
15:31:42 mriedem i'm just not sure why we couldn't move py27 stuff to experimental,
15:31:42 cdent mriedem: anyway, if we don't want another gate job, that's fine, but we still need the tox jobs and the bug fixes are real (and backwards compatible)
15:31:49 mriedem so we can test fixes for py27, but not gate on it for every change
15:32:11 mriedem sure i'm just in this mindset about the broken gate
15:32:57 cdent in the discussions that I was party to, people were wed to this notion of "testing what people are using (now)" rather than "testing what people will be using when the code is released"
15:33:04 cdent given the former, 2.7 matters
15:34:13 cdent mriedem: I'd be super keen on moving 2.7 to experimental
15:34:37 mriedem commented in the change
15:34:49 mriedem you can say to your tc buddies, "the troll hath spoken in my patch"
15:35:18 mriedem if 2.7 matters, can we decide which of the 20 py3 versions matters?
15:35:24 mriedem and move the others to experimental?
15:36:39 mriedem speaking of gate bug fixes, need another +2 on https://review.openstack.org/#/c/615347
15:39:09 coreycb cdent: will placement be removed from nova in stein?
15:39:40 cdent coreycb: that's the goal, yes
15:40:15 cdent coreycb: that's part of what is driving the goals in http://lists.openstack.org/pipermail/openstack-dev/2018-September/134541.html
15:40:23 coreycb cdent: ok. I just won't be able to get the new placment package into main (ie. get security support etc) until it's removed from nova.
15:40:46 cdent coreycb: depending on who you talk to, we're relatively close
15:41:03 coreycb cdent: ok :)
15:41:16 cdent this needs to merge https://review.openstack.org/#/c/617941/ and then it's child can be considered
15:42:38 coreycb cdent: seems pretty close if that's the case. thanks, I'll keep an eye on it.
15:44:11 cdent thank you
15:45:51 openstack bug 1806064 in OpenStack Compute (nova) queens "Volume remains in attaching/reserved status, if the instance is deleted after TooManyInstances exception in nova-conductor" [Medium,In progress] https://launchpad.net/bugs/1806064 - Assigned to Matt Riedemann (mriedem)
15:45:51 openstackgerrit Merged openstack/nova stable/queens: Add functional regression test for bug 1806064 https://review.openstack.org/623933
15:46:05 openstackgerrit Silvan Kaiser proposed openstack/nova master: Exec systemd-run without --user flag in Quobyte driver https://review.openstack.org/554195
15:48:43 cdent melwitt: I've proposed some gerrit acl config changes to os-resource-classes and os-traits in https://review.openstack.org/624387 which will need your +1 (if you're in fact +1). The justification is effectively: why wait?
15:53:23 mriedem huh, this would probably be annoying for anyone pointing nova-api at cell0 for [database]/connection http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22The%20nova-osapi_compute%20service%20version%20is%20from%20before%20Ocata%5C%22%20AND%20tags%3A%5C%22screen-n-api.txt%5C%22&from=7d
15:59:53 mriedem dansmith: looks like something must have regressed this recently since we're seeing it in master and queens only http://logs.openstack.org/82/623282/2/check/tempest-full/0a4f66a/controller/logs/screen-n-api.txt.gz?level=WARNING#_Dec_11_06_09_01_928385 - not sure why tests aren't failing though, since when we hit that warning we are pulling instances from cell0 and not even looking for an instance mapping
16:00:52 ShilpaSD gibi: Hi
16:02:29 dansmith mriedem: hmm, with creating the record or looking for them I wonder?
16:04:54 mriedem don't really understand since nova-osapi_compute service should be created in the cell0 db here https://github.com/openstack/nova/blob/master/nova/api/openstack/wsgi_app.py#L59
16:05:14 mriedem and that would use https://github.com/openstack/nova/blob/master/nova/objects/service.py#L219
16:09:52 mriedem http://logs.openstack.org/82/623282/2/check/tempest-full/0a4f66a/controller/logs/screen-n-api.txt.gz#_Dec_11_06_09_01_468549
16:09:52 mriedem wtf, same request, we lookup the instance twice, first time we don't get that warning
16:10:05 mriedem 2nd time we do http://logs.openstack.org/82/623282/2/check/tempest-full/0a4f66a/controller/logs/screen-n-api.txt.gz#_Dec_11_06_09_01_919760
16:10:58 openstackgerrit Chris Dent proposed openstack/nova master: Add python 3.7 unit and functional tox jobs https://review.openstack.org/624055
16:11:08 cdent made 3.7 experimental
16:29:20 openstackgerrit Balazs Gibizer proposed openstack/nova master: Ensure that allocated PF matches the used PF https://review.openstack.org/623543
16:32:24 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: debug old compute API service version lookup https://review.openstack.org/624438

Earlier   Later