Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-02
14:36:26 zaneb mriedem: o/
14:36:48 mriedem zaneb: hi, so i read through all of https://governance.openstack.org/tc/resolutions/20170317-cloud-applications-mission.html again per your email on getting nova dev attendance at the session https://www.openstack.org/summit/boston-2017/summit-schedule/events/18951/cloud-aware-application-support
14:37:05 openstackgerrit Sean Dague proposed openstack/nova master: remove unused functions https://review.openstack.org/461813
14:37:08 mriedem zaneb: and am i correct in saying that for nova it really just boils down to talking about this again? https://review.openstack.org/#/c/222293/
14:37:44 mriedem b/c of the footnote that says "and provide a secure method to provision, rotate, and deliver credentials to the application."
14:37:49 zaneb mriedem: that's one thing that we'll want to discuss, but there are a couple of others that touch on Nova
14:39:02 zaneb mriedem: https://review.openstack.org/#/c/427872/ and https://review.openstack.org/#/c/438134/ are others
14:39:05 mriedem zaneb: so before we talk about that again,
14:39:13 mriedem this has come up several times in the past https://etherpad.openstack.org/p/newton-instance-users
14:39:19 mriedem ^ is the most recent that i remember, from austin,
14:39:37 mriedem but there is nothing in there about what was decided, or next steps
14:39:54 jaypipes edleafe, cdent: k, responded.
14:40:16 cdent thanks jay, looking. Also came up with yet another concern, which I'll inject while reading your repsonses
14:40:22 jaypipes kk
14:40:33 mriedem johnthetubaguy: do you know if you're going to be at the summit?
14:43:38 johnthetubaguy mriedem: I will be there
14:43:54 openstackgerrit Dan Smith proposed openstack/nova master: WIP: Debug race in delete-while-booting test https://review.openstack.org/461513
14:46:50 mriedem and we have lost zane
14:48:25 jaypipes mriedem: ?
14:49:15 mriedem jaypipes: see above
14:50:27 mriedem the john hopkins crew should be attending https://www.openstack.org/summit/boston-2017/summit-schedule/events/18951/cloud-aware-application-support
14:50:38 mriedem since it's all about rbac and auth and secrets
14:53:57 openstackgerrit Sergey Nikitin proposed openstack/nova master: Added PCI NUMA policies https://review.openstack.org/390520
14:55:47 openstackgerrit Alex Szarka proposed openstack/nova master: [WIP]Transform instance.exists notification https://review.openstack.org/403660
14:57:57 openstackgerrit Matt Riedemann proposed openstack/nova master: re-Allow adding computes with no ComputeNodes to aggregates https://review.openstack.org/460616
14:58:21 cdent jaypipes: responded, I hope I'm making a modicum of sense
15:00:05 johnthetubaguy mriedem: zaneb: my memory of the instance-users stuff was the idea to build a prototype that didn't need any Nova integration, so it was clear what bits would be needed in Nova to make the user experience (or security) better
15:00:34 johnthetubaguy mriedem: I don't 100% remember who promised to build the prototype, but I don't remember hearing about what happened
15:00:50 zaneb yeah, so the conclusion of the session was to hack it in as wsgi middleware
15:01:04 zaneb johnthetubaguy: it was kfox1111
15:01:22 johnthetubaguy zaneb: that sounds right
15:01:45 zaneb IIUC he went away and decided that wouldn't work, because of (IIRC) key rotation
15:02:29 johnthetubaguy zaneb: but surely that means we need a prototype that supports key rotation?
15:02:30 zaneb then he did some further updates to the spec, but eventually momentum died
15:04:10 openstackgerrit Sergey Nikitin proposed openstack/nova-specs master: PCI NUMA Policies https://review.openstack.org/361140
15:04:21 zaneb johnthetubaguy: you're not going to get that from wsgi middleware though, because that only gets to run once, when you create the server
15:05:30 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Reword 'reserved_host_memory_mb' help text https://review.openstack.org/461054
15:05:30 openstackgerrit Stephen Finucane proposed openstack/nova master: Replace reserved_host_disk_mb with reserved_host_disk_gb https://review.openstack.org/460661
15:05:48 sdague zaneb: wsgi middleware can have code run on every request
15:07:36 cdent sdague: I would guess an issue with that is predictability of rotation, but for a prototype may not be a concern
15:07:38 mriedem what does the wsgi middleware need to do? fetch a token from keystone and make it available to the guest?
15:07:44 dansmith sdague: right but they're talking about jamming creds into the request to boot, so you'd never have a change to update those
15:07:52 jaypipes cdent: responded.
15:07:54 zaneb sdague: the middleware in question was to inject the keys into the metadata at boot time. there's no way to get that to update the metadata periodically even when there are no API requests. (is metadata even mutable?)
15:07:57 cdent thanks jaypipes
15:08:01 jaypipes cdent: and yes, you're totally making sense.
15:08:09 zaneb mriedem: that was the idea, yeah
15:08:18 zaneb see above^
15:08:22 sdague zaneb: ah, gotcha
15:08:28 mriedem zaneb: i thought that was the same reason for the vendordata v2 stuff in the metadata api
15:08:35 jaypipes cdent: I agree with you and edleafe. just want to isolate these new changes for a little while so we can really test it out before merging into the _get_all_by_filters() method.
15:08:54 mriedem guest makes a request to the vendoradata v2 meta api, which is pluggable,
15:09:04 cdent jaypipes: yeah, that's cool, but was not immediately apparent
15:09:05 mriedem and that thing goes out and gets whatever you need to give to the guest
15:09:07 sdague zaneb: yeh, the vendordata v2 bits should open an avenue here
15:09:11 johnthetubaguy zaneb: the solution always needed an in VM agent to do the rotation AFAIK, thats not part of what Nova does really, Nova can help bootstrap an agent, but mostly you should be able to do that without Nova changes
15:09:25 mriedem sdague: you're ceph fix just passed
15:09:30 jaypipes cdent: and once we Q.E.D. all the SQL constructs for these new shared providers and traits stuff, merge into a single SQL-constructing function will be easier to stomach.
15:09:31 sdague mriedem: cool
15:09:48 sdague mriedem: ship it!
15:10:05 zaneb sdague: great, so hopefully this might be easier to tackle now that's in place
15:10:08 mriedem i need to see if mikal updated the docs after his vendoradata v2 enhancements
15:10:17 mriedem ah https://review.openstack.org/#/c/429431/
15:10:58 openstackgerrit Matt Riedemann proposed openstack/nova master: Update devref with vendordata changes. https://review.openstack.org/429431
15:11:18 cdent jaypipes: does "Yeah, noted." mean "yeah, I agree that this will need to be addressed somewhere in this stack"?
15:11:48 jaypipes cdent: yup.
15:11:58 cdent cool, then I think I can manage to leave you alone for a little while
15:12:14 jaypipes cdent: lol
15:12:58 johnthetubaguy zaneb: the vendor data thing shouldn't affect the core of the prototype though, its just injecting things into configdrive/metadata service
15:13:56 cdent In fact I think it is time for a walk
15:16:24 dansmith mriedem: FYI I think we have another delete race thing, which I'm only able to poke with multiple cells, and only able to poke in the gate (can't repro locally)
15:16:40 mriedem dansmith: i've been seeing your patch
15:17:00 dansmith mriedem: not sure if it's a regression or a latent thing.. it seems to be actually something in the compute at this point, so might be latent and we're just now tickling it
15:17:01 dansmith mriedem: okay cool
15:24:06 jaypipes not sure how I feel about all of dansmith's poking and tickling.
15:24:25 dansmith jaypipes: that's not what you said ... before.
15:24:51 jaypipes oh, dansmith.
15:25:00 dansmith jaypipes: <3
15:29:33 edleafe get a room, you two
15:31:21 mriedem melwitt: your unicode decode error thing has taken off more since 5/1 http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22instance_update_at_top%5C%22%20AND%20message%3A%5C%22UnicodeDecodeError%5C%22%20AND%20tags%3A%5C%22screen-n-cell-region.txt%5C%22&from=7d
15:32:19 melwitt it's mine *eyes shimmering*
15:32:49 mriedem it looks like it blows up in the compute first http://logs.openstack.org/03/427903/7/check/gate-tempest-dsvm-cells-ubuntu-xenial/87da64a/logs/screen-n-cpu.txt.gz?level=TRACE
15:32:58 mriedem which puts the instance into error state
15:33:07 mriedem and that's causing some stuff to blow up at the top when logging the state change
15:33:20 mriedem File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 961, in unplug
15:33:24 mriedem yup it's that same bug as before
15:33:50 mriedem https://bugs.launchpad.net/nova/+bug/1580728
15:33:50 openstack Launchpad bug 1580728 in OpenStack Compute (nova) "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128) in nova.virt.libvirt.vif:unplug with unicode instance.display_name" [Medium,Triaged]
15:33:51 melwitt oh, the unplug vif you talked about
15:34:40 mriedem yeah, whatever is making that blow up in n-cpu and put the instance into error state,
15:34:50 mriedem also makes n-cell-region instance_update_at_top blow up
15:34:58 mriedem b/c we're logging the full instance
15:35:02 mriedem and something in there is unicode
15:35:20 melwitt the lp bug says the test uses a unicode display name for the instance
15:35:35 melwitt if that's still true then that would do it. I don't get why it doesn't always happen though
15:40:47 mriedem melwitt: yeah test_update_server_name_in_stop_state does create a server, stop it, then update the server name to unicode and then delete it
15:41:09 melwitt mriedem: sdague
15:41:11 mriedem i don't know why it wouldn't blow up everytime though
15:42:08 mriedem melwitt: also, when this fails http://logs.openstack.org/03/427903/7/check/gate-tempest-dsvm-cells-ubuntu-xenial/87da64a/console.html#_2017-05-02_08_22_13_375688
15:42:24 mriedem it's while waiting for the server to be active, before the name is updated to be unicode

Earlier   Later