Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-04-04
13:06:26 efried ileixe: I also seem to recall there was going to be a forum session following on from one we had in Berlin on this topic...
13:07:08 efried ileixe: https://www.openstack.org/summit/denver-2019/summit-schedule/events/23675/change-ownership-of-resources-followup
13:08:01 adrianc sean-k-mooney: have some meetings and prior engagements will probably get to it on Sunday.
13:08:54 sean-k-mooney adrianc: ill read your comments if you have no objections to useing syncronised instead of util.lock then i can make the changes today
13:09:42 adrianc sean-k-mooney: well i intentionally defined the utils.lock() so ill be able to use as a context manager and save the wrapping
13:09:46 openstackgerrit Takashi NATSUME proposed openstack/nova master: Change a log level for overwriting allocation https://review.openstack.org/649788
13:10:49 adrianc sean-k-mooney: but if decorators are preferred then I have no objection.
13:13:21 mriedem anyone hit this grenade failure yet? http://logs.openstack.org/12/648912/4/check/grenade-py3/19a6549/logs/grenade.sh.txt.gz#_2019-04-03_03_50_29_324
13:14:21 mriedem efried: cdent: ^?
13:15:31 cdent mriedem: I remember hitting that some months ago and thinking I fixed it
13:15:59 cdent It happens when the openstack client is reinstalled, at a different python version, so the plugins are no longer visible
13:16:45 mriedem hmm, only happening on stable/stein and master http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22cval%3D'openstack%3A%20'%5C%5C''resource%20provider%20inventory%20show%20openstack%3A%20VCPU%20-f%20value%20-c%20total'%5C%5C''%20is%20not%20an%20openstack%20command.%20See%20'%5C%5C''openstack%20--help'%5C%5C''.%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d
13:16:50 mriedem grenade was maybe branched?
13:17:04 cdent so, somewhere in that log will be something causing whichever openstack command is on the path to become a different one
13:18:30 cdent this was the original bug https://bugs.launchpad.net/grenade/+bug/1805156
13:18:31 openstack Launchpad bug 1805156 in grenade "In a python3 grenade run, openstack client plugins are lost after swift install" [Undecided,Fix released] - Assigned to Chris Dent (cdent)
13:18:32 openstackgerrit Hamdy Khader proposed openstack/nova master: Do not perform port update in case of baremetal instance. https://review.openstack.org/649345
13:18:56 mriedem cdent: looks like osc-placement is installed into py2.7 and python-openstackclient is installed into py3.6
13:19:42 efried mriedem: Yes, I was looking at that last night.
13:20:20 efried okay, looks like you guys are further along than I got
13:20:46 mriedem WARNING: Did not find python 3 classifier for remote package osc-placement
13:20:52 mriedem that's why osc-placement isn't installed on py3
13:20:59 cdent mriedem: swift is reinstalling python-openstackclient here: http://logs.openstack.org/12/648912/4/check/grenade-py3/19a6549/logs/grenade.sh.txt.gz#_2019-04-03_03_26_57_820
13:21:18 mriedem https://github.com/openstack/osc-placement/blob/master/setup.cfg#L18
13:21:26 mriedem assuming this is a rax-dfw node...
13:22:01 cdent are you sure that's more than a warning?
13:23:28 mriedem that makes me think it's bug 1820892
13:23:29 openstack bug 1820892 in devstack "Intermittent "Error starting thread.: ModuleNotFoundError: No module named 'etcd3gw'" in grenade-py3 jobs since March 14" [High,In progress] https://launchpad.net/bugs/1820892 - Assigned to Dr. Jens Harbott (j-harbott)
13:23:53 mriedem devstack failing to parse the py3 classifier on pypi for rax-dfw nodes and thinks it's a py2.7 package and installs there
13:23:57 mriedem https://review.openstack.org/#/c/649096/
13:24:42 cdent I agree with that fix
13:25:06 cdent I'll fix osc-placement in the meantime
13:25:33 mriedem osc-placement looks OK, except it doesn't have a 3.6 classifier, but devstack isn't looking for that
13:25:41 frickler cdent: mriedem: hmm, maybe there's more to it with the "swift reinstalling osc" thing. might be swift shouldn't be installed at all for a py3 job as long as they don't do py3
13:26:26 cdent frickler: I've wondered that too
13:26:40 mriedem frickler: that would be a pretty big change to the integrated gate i think since by default swift backs glance
13:26:43 frickler because some time soon we also might want to make sure that py3 jobs are able to run without py2 at all
13:26:43 mriedem in devstack i mean
13:27:41 mriedem this was a rax-dfw node as well btw
13:27:52 mriedem it's something about the networks to pypi for those nodes
13:28:07 cdent that's...bizarre?
13:28:47 frickler https://review.openstack.org/#/c/649096/ should help with that. if we can get it to pass all those unstable tests, that is
13:30:02 cdent mriedem: is there anything I need to do here, or is the situation in hand?
13:32:58 mriedem cdent: if the underlying issue is the same bug that ^ is fixing then i think we're ok
13:33:24 efried mriedem: if we're trying to cut RC2 today, we need to resolve https://review.openstack.org/#/c/649775/ ASAP, right?
13:33:27 efried What's the next step there?
13:34:01 mriedem efried: i'd like input from you and dansmith on that one, seems gibi thinks bumping the major and dropping the fields is better (alternative 1) and i tend to agree
13:34:05 mriedem i wrote that patch before bed in haste
13:34:23 mriedem taking my kid to the bus stop
13:34:53 efried mriedem: I've only skimmed, but was there a concern that the malicious user would be able to opt into the insecure version? and how was that allayed?
13:35:09 efried I'll read it more deeply here...
13:35:42 dansmith those fields have no place in a notification, best I can tell
13:38:38 efried dansmith: I think that's agreed; the question is just whether we can get away with removing them from the existing version since nobody's using it yet, or if we need a new version
13:38:50 efried in which case can we still get hacked by people asking for the old one
13:39:35 dansmith the *only* reason to do the 1.1 version is if we really want to backport that or something
13:40:12 dansmith but since, as matt said on the review, the user doesn't get to choose what they get, I think it'd be better to signal "no, these really aren't going to be in your notification" with a major version bump
13:40:58 dansmith what I meant by "want to backport" is "want to backport without the appearance of a major bump and breaking change"
13:41:02 efried there's not like a way to spoof a client asking for the old version?
13:41:03 dansmith but I don't think we should do that
13:41:17 dansmith you don't ask not notification versions
13:41:20 dansmith you get what you get
13:41:27 dansmith the version is just a signal to tell you something has changed
13:41:36 efried okay, neat. Then I'm on board.
13:41:40 dansmith it's not like objects where we try to backport a version to the one you want
13:43:00 efried cool
13:45:45 mriedem ok i'll bump the major and drop the fields
13:45:54 mriedem do we still by default emit both legacy and versioned....
13:46:06 mriedem yup
13:46:12 mriedem https://docs.openstack.org/nova/latest/configuration/config.html#notifications.notification_format
13:46:37 mriedem the amount of object translation work to build these payloads is likely a detriment to scheduling performance as wel
13:46:42 mriedem *well
13:46:58 mriedem anywho, that's a different known issue
13:49:25 dansmith mriedem: the base object has some version that increments with each subclass change?
13:49:49 dansmith oh, nm
13:49:52 dansmith event_type
13:49:56 dansmith which won't need to change with this
13:55:15 efried Nova meeting in five minutes in #openstack-meeting
14:03:13 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove CellMappingPayload database_connection and transport_url fields https://review.openstack.org/649775
14:03:23 mriedem gibi_off: stephenfin: dansmith: melwitt: efried: ^
14:03:35 stephenfin ack
14:03:39 efried thx mriedem
14:16:06 openstackgerrit Stephen Finucane proposed openstack/nova master: devstack: Remove 'tempest-dsvm-tempest-xen-rc' https://review.openstack.org/650018
14:22:23 openstackgerrit Matt Riedemann proposed openstack/nova master: Add docs on what not to include in notifications https://review.openstack.org/650020
14:27:56 sean-k-mooney jaypipes: looking at adrianc responce https://review.openstack.org/#/c/620115/29/nova/compute/manager.py@7193 i think its a resonable argument but im also happy to create the new function and use the synchronized decorator.
14:29:13 sean-k-mooney ill leave the decission up to you. the current decorator cant be used as a context manager but i think that could be added in the future to oslo concurancy
14:30:26 sean-k-mooney i think we would jsut have to change this return to a yeild https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/lockutils.py#L328
14:31:36 sean-k-mooney and add the contextlib.contextmanager decorator https://docs.python.org/3/library/contextlib.html#contextlib.contextmanager
14:32:32 sean-k-mooney its something i need to tink about but its a sepreate form this change
14:42:22 mriedem efried: i shared a link to those charts i mentioned, you can peruse them at your leisure and if you think they are fine as further reading (albeit targeted topics) i can make them public and share on the ML or something
14:47:00 openstackgerrit Matt Riedemann proposed openstack/nova stable/stein: Remove CellMappingPayload database_connection and transport_url fields https://review.openstack.org/650043
14:47:36 efried mriedem: thank you.
15:16:34 mriedem hmm, because of that recent devstack change to add a 2nd shared public network we're always getting this in n-cpu logs during tempest runs now
15:16:35 mriedem http://logs.openstack.org/58/637058/15/check/nova-next/f4e8140/logs/screen-n-cpu.txt.gz?level=TRACE#_Apr_04_00_07_20_968719
15:16:56 mriedem seems that should not trace, nor be an error really - the user made a bad request essentially but we catch it in compute rather than the api
15:18:06 mriedem yeah it's just a 400 INFO response in the api http://logs.openstack.org/58/637058/15/check/nova-next/f4e8140/logs/screen-n-api.txt#_Apr_04_00_07_20_975380
15:20:00 mriedem huh surprising we even get that far, the request is empty Body: {"interfaceAttachment": {}}
15:25:45 mriedem https://bugs.launchpad.net/nova/+bug/1823198 is anyone has opinions
15:25:46 openstack Launchpad bug 1823198 in OpenStack Compute (nova) "NetworkAmbiguous traceback in nova-compute logs even though it's a user error" [Low,Triaged]
15:25:48 mriedem *if
15:53:22 openstackgerrit Matt Riedemann proposed openstack/nova master: Drop pre-cinder 3.44 version compatibility https://review.openstack.org/621061

Earlier   Later