Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-02-03
14:20:42 figleaf cdent: regarding https://review.openstack.org/#/c/416752/8/nova/api/openstack/placement/handlers/inventory.py, what should be the value for 'max_unit' in the INVENTORY_DEFAULTS (line 109) ?
14:22:06 cdent figleaf: I would think since we want to set a schema minimum of 1 (since less than that is meaningless or wrong) then the default should be 1 as well?
14:22:09 cdent bauzas: looking
14:23:18 bauzas mriedem: when are you removing -W for https://review.openstack.org/#/c/428531/ ?
14:24:39 mriedem bauzas: probably after https://review.openstack.org/#/c/428375/ is merged now
14:24:46 mriedem johnthetubaguy: +2 on ^ now
14:25:01 johnthetubaguy looking...
14:25:11 figleaf cdent: didn
14:25:13 mriedem cdent: figleaf: a functional report client test as noted by bauzas for deleting RPs would be good, but i'd do that in a follow up
14:25:13 figleaf doh
14:25:38 mriedem bauzas: i already talked to tonyb about blocking requirements yesterday, it's not a big deal right now
14:25:52 figleaf cdent: didn't seem right to set the default 'max' to 1
14:25:52 mriedem dims_: was telling everyone that yesterday was a soft date for rc1 and monday would be ok
14:26:03 mriedem bauzas: but i'll get rc1 out today if we get this last fix in
14:26:03 cdent figleaf: it is weird yeah, but is better than 0
14:26:29 figleaf cdent: sure, but even db.MAX_INT felt better (even though that seems wrong, too)
14:26:48 bauzas mriedem: okay, cool with me about RC1
14:26:59 mriedem i've got to get my kid on the bus and head into the office
14:27:02 bauzas cdent: mriedem: about https://review.openstack.org/#/c/428375/8/nova/scheduler/client/report.py@772 my question is rather
14:27:49 bauzas cdent: mriedem: if we're deleting a compute node, are we also deleting the nested instances elsewhere ?
14:27:49 cdent figleaf: hmmm. yeah. a good point. that is probably better than 1 as it results in things being left in a working state if a max is not provided
14:28:22 cdent figleaf: which is what DEFAULTS ought to be going for
14:28:54 cdent figleaf: so yeah, step_size 1, min_unit 1, max_unit MAX_INT seems right for the defaults
14:29:00 figleaf cdent: "Hi, I'd like 1 MEMORY_MB for my VM, please"
14:29:29 cdent it's more that 640k so it's gotta be enough
14:31:08 cdent metoo@aol.com
14:35:51 bauzas johnthetubaguy: I had a concern on https://review.openstack.org/#/c/428375/
14:36:07 bauzas I did +1 because I didn't wanted to pollute and say it was a blocker
14:36:19 bauzas but I'm still wondering if all goes the right way
14:37:21 johnthetubaguy bauzas: most of the time the list of instances will be empty, I think
14:37:45 johnthetubaguy bauzas: its hitting the 409 from placement I guess, that would cause an issue
14:38:16 bauzas johnthetubaguy: I need to look at code, because I wonder that
14:38:42 bauzas johnthetubaguy: say I'm booting inst1, then deleting computeA, do I get a nova list empty?
14:38:50 bauzas I suppose so
14:38:58 bauzas so I'm wondering where the thing is done
14:44:47 openstackgerrit Chris Dent proposed openstack/nova master: Add more debug logging on RP inventory update failures https://review.openstack.org/414230
14:44:47 openstackgerrit Chris Dent proposed openstack/nova master: Add placement request id to log when GET or POST rps https://review.openstack.org/427740
14:44:48 openstackgerrit Chris Dent proposed openstack/nova master: Add placement request id to log when GET aggregates https://review.openstack.org/426874
14:44:48 openstackgerrit Chris Dent proposed openstack/nova master: Add more debug logging on RP inventory delete failures https://review.openstack.org/426290
14:45:41 cdent johnthetubaguy: the bottom of that stack ^ was +W (+2 from you) but failing in the gate a few times, so I've rebased but now it is all bereft of its approval
14:46:27 openstackgerrit Sylvain Bauza proposed openstack/nova master: Destroy the ReqSpec object when deleting the instance https://review.openstack.org/391060
14:46:27 openstackgerrit Sylvain Bauza proposed openstack/nova master: Cover again the ReqSpec record creation testing https://review.openstack.org/391059
14:54:22 openstackgerrit Ed Leafe proposed openstack/nova master: Add check for invalid allocation amounts https://review.openstack.org/416752
14:54:24 figleaf cdent: ^^ Addressed your concerns
14:57:38 johnthetubaguy cdent: ah, doh, will take a peak
15:06:30 johnthetubaguy cdent: I do wonder if we have too many new lines in some of the error messages in the logs, BTW
15:07:22 johnthetubaguy cdent: its a micro nit, it might be more to do with the error message omitted by the placement API and how that looks in the logs
15:07:31 mriedem that's the nature of the requests.response isn't it?
15:09:25 johnthetubaguy it could be
15:12:22 cdent johnthetubaguy, mriedem: that could be the way that webob is formatting error responses. the json_error_formatter stuff subclasses something which puts newlines in here and there for no apparently reason
15:12:57 johnthetubaguy cdent: I was looking at that with pumaranikar the other day, I think it might be a bit new line happy
15:13:06 cdent new line happy
15:13:06 cdent yes
15:13:11 openstackgerrit Sarafraj Singh proposed openstack/nova-specs master: Send service token along with user tokens https://review.openstack.org/426933
15:13:32 cdent i'm not sure I'm keen on the decorator idea, but the current middleware idea won't work
15:13:44 cdent (but I've mostly been ignoring that)
15:16:10 cdent johnthetubaguy: in any case, the logging on the rt side shouldn't care. we should just fix it on the api side, yeah?
15:16:59 johnthetubaguy cdent: yeah, certainly if the extra new lines are from the API, which I think they are
15:17:22 johnthetubaguy cdent: it was more your patches reminded my about the niggle I spotted debugging that bug the other day
15:17:56 cdent let's make pumaranikar fix it :)
15:18:39 johnthetubaguy good idea
15:18:47 johnthetubaguy pumaranikar: we found you another placement bug
15:19:23 EmilienM mriedem: good morning, regarding the issue with Nova in WSGI in TripleO - does it make sense we stop doing that in Ocata (and revisit later)?
15:19:26 mriedem so on the tripleo failures from yesterday, with a run on my debug patch, i see the instance that heat is trying to create fails
15:19:26 mriedem http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/7a6e5bc/logs/subnode-2/var/log/nova/nova-conductor.txt.gz#_2017-02-03_05_17_35_826
15:19:28 mriedem EmilienM: good timing
15:19:32 EmilienM wow
15:19:43 mriedem scheduler couldn't find a host
15:19:56 EmilienM do you want me to stop deploing nova api in wsgi with apache and see if it helps?
15:20:15 mriedem given we haven't really flushed out testing with that, i think it's prudent at this point in ocata
15:20:26 EmilienM ok let me do it
15:21:03 mriedem the 404 isn't because the instance was created and we just can't find it though, it's not found because it fails to build and the instance isn't created (it looks like), but the dead instance should go into cell0 and we should pull it from cell0
15:21:31 mriedem hrm, dansmith melwitt do we even look in cell0 for compute api get()?
15:21:39 cdent johnthetubaguy, pumaranikar It can probably be built right in with the json_formatter stuff because it's all of a piece
15:22:12 melwitt mriedem: we should, I think. do we not? checking
15:22:21 dansmith melwitt: yes
15:22:23 dansmith er, mriedem yes
15:22:31 dansmith mriedem: the mapping tells us what cell to look in
15:22:38 melwitt oh yeah
15:22:39 dansmith which would be cell0 if it's there
15:24:15 mriedem what's odd is in this debug run i expect to see the 'found x cells' thing from https://review.openstack.org/#/c/428360/2/nova/compute/api.py@2526
15:24:22 mriedem but it's not in http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/7a6e5bc/logs/subnode-2/var/log/nova/nova-api.txt.gz
15:24:30 kashyap johnthetubaguy: "But asking for a friendlier looking supernova seems a bit... unscientific" :-)
15:24:52 johnthetubaguy kashyap: :)
15:25:19 kashyap Meanwhile, if there are any Virtualization maintainers of Ubuntu, you may want to triage this - https://bugs.launchpad.net/nova/+bug/1643911
15:25:19 openstack Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed]
15:25:24 kashyap lease see my last comment there.
15:25:26 melwitt mriedem: I thought we had found there weren't any 'GET .../servers' in those logs
15:25:31 kashyap (s/lease/Please/)
15:26:01 mriedem melwitt: my debug patch also did the cells thing in _get_instance()
15:26:23 melwitt oh, you added logging for _get_instance
15:26:47 mriedem melwitt: http://logs.openstack.org/60/428360/2/experimental/gate-tripleo-ci-centos-7-nonha-multinode/7a6e5bc/logs/subnode-2/var/log/nova/nova-api.txt.gz#_2017-02-03_05_17_30_873 is the GET
15:27:31 mriedem but it's not hitting the logging i added here https://review.openstack.org/#/c/428360/2/nova/compute/api.py
15:28:13 mriedem so i'm thinking we're trying to pull it from a cell but it's not there, which would be nuts, but i'll add some more logging
15:28:58 mriedem oh well i guess that would make sense if we pull the instance here https://review.openstack.org/#/c/428360/2/nova/compute/api.py@2269
15:29:04 mriedem because the service version is 0
15:29:41 melwitt yeah, I was about to say, we had found bc they're running through apache they're getting service version 0, or something
15:31:51 mriedem yup,
15:31:52 mriedem AND
15:31:59 mriedem http://logs.openstack.org/15/428415/1/experimental/gate-tripleo-ci-centos-7-nonha-multinode/4a76477/

Earlier   Later