Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-02
15:08:09 zaneb mriedem: that was the idea, yeah
15:08:18 zaneb see above^
15:08:22 sdague zaneb: ah, gotcha
15:08:28 mriedem zaneb: i thought that was the same reason for the vendordata v2 stuff in the metadata api
15:08:35 jaypipes cdent: I agree with you and edleafe. just want to isolate these new changes for a little while so we can really test it out before merging into the _get_all_by_filters() method.
15:08:54 mriedem guest makes a request to the vendoradata v2 meta api, which is pluggable,
15:09:04 cdent jaypipes: yeah, that's cool, but was not immediately apparent
15:09:05 mriedem and that thing goes out and gets whatever you need to give to the guest
15:09:07 sdague zaneb: yeh, the vendordata v2 bits should open an avenue here
15:09:11 johnthetubaguy zaneb: the solution always needed an in VM agent to do the rotation AFAIK, thats not part of what Nova does really, Nova can help bootstrap an agent, but mostly you should be able to do that without Nova changes
15:09:25 mriedem sdague: you're ceph fix just passed
15:09:30 jaypipes cdent: and once we Q.E.D. all the SQL constructs for these new shared providers and traits stuff, merge into a single SQL-constructing function will be easier to stomach.
15:09:31 sdague mriedem: cool
15:09:48 sdague mriedem: ship it!
15:10:05 zaneb sdague: great, so hopefully this might be easier to tackle now that's in place
15:10:08 mriedem i need to see if mikal updated the docs after his vendoradata v2 enhancements
15:10:17 mriedem ah https://review.openstack.org/#/c/429431/
15:10:58 openstackgerrit Matt Riedemann proposed openstack/nova master: Update devref with vendordata changes. https://review.openstack.org/429431
15:11:18 cdent jaypipes: does "Yeah, noted." mean "yeah, I agree that this will need to be addressed somewhere in this stack"?
15:11:48 jaypipes cdent: yup.
15:11:58 cdent cool, then I think I can manage to leave you alone for a little while
15:12:14 jaypipes cdent: lol
15:12:58 johnthetubaguy zaneb: the vendor data thing shouldn't affect the core of the prototype though, its just injecting things into configdrive/metadata service
15:13:56 cdent In fact I think it is time for a walk
15:16:24 dansmith mriedem: FYI I think we have another delete race thing, which I'm only able to poke with multiple cells, and only able to poke in the gate (can't repro locally)
15:16:40 mriedem dansmith: i've been seeing your patch
15:17:00 dansmith mriedem: not sure if it's a regression or a latent thing.. it seems to be actually something in the compute at this point, so might be latent and we're just now tickling it
15:17:01 dansmith mriedem: okay cool
15:24:06 jaypipes not sure how I feel about all of dansmith's poking and tickling.
15:24:25 dansmith jaypipes: that's not what you said ... before.
15:24:51 jaypipes oh, dansmith.
15:25:00 dansmith jaypipes: <3
15:29:33 edleafe get a room, you two
15:31:21 mriedem melwitt: your unicode decode error thing has taken off more since 5/1 http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22instance_update_at_top%5C%22%20AND%20message%3A%5C%22UnicodeDecodeError%5C%22%20AND%20tags%3A%5C%22screen-n-cell-region.txt%5C%22&from=7d
15:32:19 melwitt it's mine *eyes shimmering*
15:32:49 mriedem it looks like it blows up in the compute first http://logs.openstack.org/03/427903/7/check/gate-tempest-dsvm-cells-ubuntu-xenial/87da64a/logs/screen-n-cpu.txt.gz?level=TRACE
15:32:58 mriedem which puts the instance into error state
15:33:07 mriedem and that's causing some stuff to blow up at the top when logging the state change
15:33:20 mriedem File "/opt/stack/new/nova/nova/virt/libvirt/vif.py", line 961, in unplug
15:33:24 mriedem yup it's that same bug as before
15:33:50 openstack Launchpad bug 1580728 in OpenStack Compute (nova) "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128) in nova.virt.libvirt.vif:unplug with unicode instance.display_name" [Medium,Triaged]
15:33:50 mriedem https://bugs.launchpad.net/nova/+bug/1580728
15:33:51 melwitt oh, the unplug vif you talked about
15:34:40 mriedem yeah, whatever is making that blow up in n-cpu and put the instance into error state,
15:34:50 mriedem also makes n-cell-region instance_update_at_top blow up
15:34:58 mriedem b/c we're logging the full instance
15:35:02 mriedem and something in there is unicode
15:35:20 melwitt the lp bug says the test uses a unicode display name for the instance
15:35:35 melwitt if that's still true then that would do it. I don't get why it doesn't always happen though
15:40:47 mriedem melwitt: yeah test_update_server_name_in_stop_state does create a server, stop it, then update the server name to unicode and then delete it
15:41:09 melwitt mriedem: sdague
15:41:11 mriedem i don't know why it wouldn't blow up everytime though
15:42:08 mriedem melwitt: also, when this fails http://logs.openstack.org/03/427903/7/check/gate-tempest-dsvm-cells-ubuntu-xenial/87da64a/console.html#_2017-05-02_08_22_13_375688
15:42:24 mriedem it's while waiting for the server to be active, before the name is updated to be unicode
15:42:34 melwitt mriedem: sdague's comment on the lp bug mentions how devstack setting LC_ALL=C for grepping causes this, I wonder if there's some reason we couldn't use en_US.utf8 instead
15:43:45 sdague oh, that would totally explain why it's a gate only thing
15:43:58 sdague though, I thought the gate was C.utf8
15:44:29 Swanson I see n-crt failing on my CI since yesterday. There something up with it?
15:44:43 mriedem Swanson: n-crt was deleted yesterday
15:44:53 mriedem Swanson: it's been deprecated since newton
15:45:02 Swanson mriedem, Whee! Easy fix.
15:45:20 melwitt sdague: looks like a mix in devstack? https://github.com/openstack-dev/devstack/search?utf8=%E2%9C%93&q=lc_all&type=
15:45:25 Swanson mriedem, Would explain my difficulty actually finding it.
15:46:02 sdague melwitt: yeh
15:46:26 sdague melwitt: if you want to propose change to en_US.utf8 everywhere, I'm good with that
15:46:37 sdague especially if it fixes this
15:48:10 melwitt sdague: yup, let's try that. C is ascii only so probably should be using utf8 anyhow. not 100% sure if it will fix this though
15:48:27 sdague yeh, I feel like I had that patch up before for other reasons
15:48:32 sdague but we never decided to land it
15:48:39 sdague but if it fixes this thing, win!
15:49:12 melwitt ++
15:52:11 mriedem seems we should still not be logging something that's going to blow up here yeah?
15:54:50 melwitt mriedem: I'm not sure. I had been thinking the failure is environment specific, i.e. if the environment doesn't support utf8
15:55:35 mriedem melwitt: we put lots of things in six.text_type though for the same reason
15:55:41 mriedem regardless of env
15:55:42 melwitt but I'm still not clear on why it doesn't happen all the time and what we could do the prevent it blowing up in a non utf8 env?
15:55:51 melwitt oh
15:57:33 sdague mriedem: it's one of those things that is more fundamental
15:57:56 sdague I feel like I went through a weird set of poking on this the last time
15:57:58 mriedem The String field in ovo has a stringify method which is used for __repr__, seems that should always be doing six.text_type
15:58:03 melwitt so six.text_type returns a unicode string. if the env doesn't support unicode, then how does it help?
15:58:10 sdague right, like melwitt said
15:58:33 sdague stuff gets super borky when you are in a non utf8 env
15:58:42 melwitt it seems like it would just do the same thing, but I dunno
15:58:48 clarkb C.utf8 is the locale I think people should be using based on what is built into libcs
15:59:20 sdague clarkb: well, that has a different issue in that "nobody" uses that except the dib images for infra
15:59:47 clarkb sdague: because its relatively new but aiui its what things are switching to?
15:59:56 sdague clarkb: are they?
16:00:01 clarkb ya fedora was aiui
16:00:10 sdague ubuntu?
16:00:10 clarkb which should mean the rest of the red hat world is too?
16:00:17 mriedem melwitt: maybe you're right, since it looks like the base __repr__ in ovo does encode it if on py27 https://github.com/openstack/oslo.versionedobjects/blob/master/oslo_versionedobjects/base.py#L318
16:00:26 clarkb ubuntu already had it but being desktop first I'm not sure if they will hcnage the default
16:00:42 sdague anyway, lets see if the old tried and true en_US.utf8 works
16:00:46 clarkb the idea behind C.utf8 is have the "sanity" of C (sorting etc) while supporting utf8 too
16:02:22 melwitt fwiw I get the error when I try the same example C.utf8 $ LC_ALL=C.utf8 python -c "a = u'\u00CD\u00F1st\u00E1\u00F1c\u00E9'; print a"
16:04:12 clarkb melwitt: what error? (that works for me locally)
16:04:35 clarkb I get "Íñstáñcé"
16:04:39 jaypipes cdent: yay, your request for additional functional tests identified an issue with the WHERE condition on shared providers... the difference of where a single parentheses is makes a world of difference :)
16:04:54 melwitt clarkb: hm, interesting. the error from the bug "UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)"

Earlier   Later