| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-02 | |||
| 15:07:38 | mriedem | what does the wsgi middleware need to do? fetch a token from keystone and make it available to the guest? | |
| 15:07:44 | dansmith | sdague: right but they're talking about jamming creds into the request to boot, so you'd never have a change to update those | |
| 15:07:52 | jaypipes | cdent: responded. | |
| 15:07:54 | zaneb | sdague: the middleware in question was to inject the keys into the metadata at boot time. there's no way to get that to update the metadata periodically even when there are no API requests. (is metadata even mutable?) | |
| 15:07:57 | cdent | thanks jaypipes | |
| 15:08:01 | jaypipes | cdent: and yes, you're totally making sense. | |
| 15:08:09 | zaneb | mriedem: that was the idea, yeah | |
| 15:08:18 | zaneb | see above^ | |
| 15:08:22 | sdague | zaneb: ah, gotcha | |
| 15:08:28 | mriedem | zaneb: i thought that was the same reason for the vendordata v2 stuff in the metadata api | |
| 15:08:35 | jaypipes | cdent: I agree with you and edleafe. just want to isolate these new changes for a little while so we can really test it out before merging into the _get_all_by_filters() method. | |
| 15:08:54 | mriedem | guest makes a request to the vendoradata v2 meta api, which is pluggable, | |
| 15:09:04 | cdent | jaypipes: yeah, that's cool, but was not immediately apparent | |
| 15:09:05 | mriedem | and that thing goes out and gets whatever you need to give to the guest | |
| 15:09:07 | sdague | zaneb: yeh, the vendordata v2 bits should open an avenue here | |
| 15:09:11 | johnthetubaguy | zaneb: the solution always needed an in VM agent to do the rotation AFAIK, thats not part of what Nova does really, Nova can help bootstrap an agent, but mostly you should be able to do that without Nova changes | |
| 15:09:25 | mriedem | sdague: you're ceph fix just passed | |
| 15:09:30 | jaypipes | cdent: and once we Q.E.D. all the SQL constructs for these new shared providers and traits stuff, merge into a single SQL-constructing function will be easier to stomach. | |
| 15:09:31 | sdague | mriedem: cool | |
| 15:09:48 | sdague | mriedem: ship it! | |
| 15:10:05 | zaneb | sdague: great, so hopefully this might be easier to tackle now that's in place | |
| 15:10:08 | mriedem | i need to see if mikal updated the docs after his vendoradata v2 enhancements | |
| 15:10:17 | mriedem | ah https://review.openstack.org/#/c/429431/ | |
| 15:10:58 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update devref with vendordata changes. https://review.openstack.org/429431 | |
| 15:11:18 | cdent | jaypipes: does "Yeah, noted." mean "yeah, I agree that this will need to be addressed somewhere in this stack"? | |
| 15:11:48 | jaypipes | cdent: yup. | |
| 15:11:58 | cdent | cool, then I think I can manage to leave you alone for a little while | |
| 15:12:14 | jaypipes | cdent: lol | |
| 15:12:58 | johnthetubaguy | zaneb: the vendor data thing shouldn't affect the core of the prototype though, its just injecting things into configdrive/metadata service | |
| 15:13:56 | cdent | In fact I think it is time for a walk | |
| 15:16:24 | dansmith | mriedem: FYI I think we have another delete race thing, which I'm only able to poke with multiple cells, and only able to poke in the gate (can't repro locally) | |
| 15:16:40 | mriedem | dansmith: i've been seeing your patch | |
| 15:17:00 | dansmith | mriedem: not sure if it's a regression or a latent thing.. it seems to be actually something in the compute at this point, so might be latent and we're just now tickling it | |
| 15:17:01 | dansmith | mriedem: okay cool | |
| 15:24:06 | jaypipes | not sure how I feel about all of dansmith's poking and tickling. | |
| 15:24:25 | dansmith | jaypipes: that's not what you said ... before. | |
| 15:24:51 | jaypipes | oh, dansmith. | |
| 15:25:00 | dansmith | jaypipes: <3 | |
| 15:29:33 | edleafe | get a room, you two | |
| 15:31:21 | mriedem | melwitt: your unicode decode error thing has taken off more since 5/1 http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22instance_update_at_top%5C%22%20AND%20message%3A%5C%22UnicodeDecodeError%5C%22%20AND%20tags%3A%5C%22screen-n-cell-region.txt%5C%22&from=7d | |
| 15:32:19 | melwitt | it's mine *eyes shimmering* | |
| 15:32:49 | mriedem | it looks like it blows up in the compute first http://logs.openstack.org/03/427903/7/check/gate-tempest-dsvm-cells-ubuntu-xenial/87da64a/logs/screen-n-cpu.txt.gz?level=TRACE | |
| 15:32:58 | mriedem | which puts the instance into error state | |
| 15:33:07 | mriedem | and that's causing some stuff to blow up at the top when logging the state change | |
| 15:33:20 | mriedem | File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 961, in unplug | |
| 15:33:24 | mriedem | yup it's that same bug as before | |
| 15:33:50 | openstack | Launchpad bug 1580728 in OpenStack Compute (nova) "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128) in nova.virt.libvirt.vif:unplug with unicode instance.display_name" [Medium,Triaged] | |
| 15:33:50 | mriedem | https://bugs.launchpad.net/nova/+bug/1580728 | |
| 15:33:51 | melwitt | oh, the unplug vif you talked about | |
| 15:34:40 | mriedem | yeah, whatever is making that blow up in n-cpu and put the instance into error state, | |
| 15:34:50 | mriedem | also makes n-cell-region instance_update_at_top blow up | |
| 15:34:58 | mriedem | b/c we're logging the full instance | |
| 15:35:02 | mriedem | and something in there is unicode | |
| 15:35:20 | melwitt | the lp bug says the test uses a unicode display name for the instance | |
| 15:35:35 | melwitt | if that's still true then that would do it. I don't get why it doesn't always happen though | |
| 15:40:47 | mriedem | melwitt: yeah test_update_server_name_in_stop_state does create a server, stop it, then update the server name to unicode and then delete it | |
| 15:41:09 | melwitt | mriedem: sdague | |
| 15:41:11 | mriedem | i don't know why it wouldn't blow up everytime though | |
| 15:42:08 | mriedem | melwitt: also, when this fails http://logs.openstack.org/03/427903/7/check/gate-tempest-dsvm-cells-ubuntu-xenial/87da64a/console.html#_2017-05-02_08_22_13_375688 | |
| 15:42:24 | mriedem | it's while waiting for the server to be active, before the name is updated to be unicode | |
| 15:42:34 | melwitt | mriedem: sdague's comment on the lp bug mentions how devstack setting LC_ALL=C for grepping causes this, I wonder if there's some reason we couldn't use en_US.utf8 instead | |
| 15:43:45 | sdague | oh, that would totally explain why it's a gate only thing | |
| 15:43:58 | sdague | though, I thought the gate was C.utf8 | |
| 15:44:29 | Swanson | I see n-crt failing on my CI since yesterday. There something up with it? | |
| 15:44:43 | mriedem | Swanson: n-crt was deleted yesterday | |
| 15:44:53 | mriedem | Swanson: it's been deprecated since newton | |
| 15:45:02 | Swanson | mriedem, Whee! Easy fix. | |
| 15:45:20 | melwitt | sdague: looks like a mix in devstack? https://github.com/openstack-dev/devstack/search?utf8=%E2%9C%93&q=lc_all&type= | |
| 15:45:25 | Swanson | mriedem, Would explain my difficulty actually finding it. | |
| 15:46:02 | sdague | melwitt: yeh | |
| 15:46:26 | sdague | melwitt: if you want to propose change to en_US.utf8 everywhere, I'm good with that | |
| 15:46:37 | sdague | especially if it fixes this | |
| 15:48:10 | melwitt | sdague: yup, let's try that. C is ascii only so probably should be using utf8 anyhow. not 100% sure if it will fix this though | |
| 15:48:27 | sdague | yeh, I feel like I had that patch up before for other reasons | |
| 15:48:32 | sdague | but we never decided to land it | |
| 15:48:39 | sdague | but if it fixes this thing, win! | |
| 15:49:12 | melwitt | ++ | |
| 15:52:11 | mriedem | seems we should still not be logging something that's going to blow up here yeah? | |
| 15:54:50 | melwitt | mriedem: I'm not sure. I had been thinking the failure is environment specific, i.e. if the environment doesn't support utf8 | |
| 15:55:35 | mriedem | melwitt: we put lots of things in six.text_type though for the same reason | |
| 15:55:41 | mriedem | regardless of env | |
| 15:55:42 | melwitt | but I'm still not clear on why it doesn't happen all the time and what we could do the prevent it blowing up in a non utf8 env? | |
| 15:55:51 | melwitt | oh | |
| 15:57:33 | sdague | mriedem: it's one of those things that is more fundamental | |
| 15:57:56 | sdague | I feel like I went through a weird set of poking on this the last time | |
| 15:57:58 | mriedem | The String field in ovo has a stringify method which is used for __repr__, seems that should always be doing six.text_type | |
| 15:58:03 | melwitt | so six.text_type returns a unicode string. if the env doesn't support unicode, then how does it help? | |
| 15:58:10 | sdague | right, like melwitt said | |
| 15:58:33 | sdague | stuff gets super borky when you are in a non utf8 env | |
| 15:58:42 | melwitt | it seems like it would just do the same thing, but I dunno | |
| 15:58:48 | clarkb | C.utf8 is the locale I think people should be using based on what is built into libcs | |
| 15:59:20 | sdague | clarkb: well, that has a different issue in that "nobody" uses that except the dib images for infra | |
| 15:59:47 | clarkb | sdague: because its relatively new but aiui its what things are switching to? | |
| 15:59:56 | sdague | clarkb: are they? | |
| 16:00:01 | clarkb | ya fedora was aiui | |
| 16:00:10 | sdague | ubuntu? | |
| 16:00:10 | clarkb | which should mean the rest of the red hat world is too? | |
| 16:00:17 | mriedem | melwitt: maybe you're right, since it looks like the base __repr__ in ovo does encode it if on py27 https://github.com/openstack/oslo.versionedobjects/blob/master/oslo_versionedobjects/base.py#L318 | |
| 16:00:26 | clarkb | ubuntu already had it but being desktop first I'm not sure if they will hcnage the default | |
| 16:00:42 | sdague | anyway, lets see if the old tried and true en_US.utf8 works | |