| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-01 | |||
| 14:07:36 | johnthetubaguy | BobBall: is this a general problem we are seeing at the moment? | |
| 14:08:11 | BobBall | Checking | |
| 14:08:42 | BobBall | general problem meaning unrelated to this patch? | |
| 14:09:08 | johnthetubaguy | BobBall: it feels very related to this patch, but we haven't changed the code that XenAPI uses, AFAIK | |
| 14:09:19 | johnthetubaguy | BobBall: just wanted to check its not a general issue | |
| 14:09:35 | BobBall | hmmmz - actually we are failing all tests in the last couple of hours... | |
| 14:10:40 | johnthetubaguy | BobBall: looks like the plugins have changed or something? | |
| 14:10:52 | BobBall | Big sadness from the Citrix CI at the moment :( | |
| 14:11:45 | BobBall | Investigating. Plugins certainly shouldn't have changed since we're pulling in from a fixed version of os-xenapi from nova's requirements... | |
| 14:12:18 | BobBall | Made the mistake of trying to open a 1MB log file in firefox. Apparently it doesn't like a 1MB html file and has decided it needs a really long think about it | |
| 14:13:06 | johnthetubaguy | BobBall: yeah, I had to download it | |
| 14:13:13 | johnthetubaguy | BobBall: I am seeing 404 from glance | |
| 14:13:28 | BobBall | Not ideal. | |
| 14:20:04 | BobBall | Looks like glance upload is broken | |
| 14:20:50 | mriedem | BobBall: like this? | |
| 14:20:51 | mriedem | 2017-02-01 12:47:40.884 20207 ERROR nova.compute.manager [instance: 797cb75d-5ab2-4231-b972-db08257716f4] Failure: ['XENAPI_PLUGIN_FAILURE', 'upload_vhd2', 'PluginError', 'Got Permanent Error response [404] while uploading image [79f5d221-0e61-4ee9-82e4-69f507219b78] to glance [http://192.168.33.1:9292/v2/images/79f5d221-0e61-4ee9-82e4-69f507219b78/file]'] | |
| 14:21:02 | BobBall | Tis only upload, yes, not download. | |
| 14:21:12 | mriedem | what changed? | |
| 14:21:12 | BobBall | I think. | |
| 14:21:15 | BobBall | Trying to figure that out. | |
| 14:23:59 | mriedem | johnthetubaguy: sfinucan: i've flip flopped and decided to nack anymore of https://blueprints.launchpad.net/nova/+spec/use-service-tokens | |
| 14:24:01 | BobBall | first failure was 11 hours ago on change 427536 | |
| 14:24:09 | mriedem | cinder and neutron are done, let's do the glance stuff in pike | |
| 14:24:21 | johnthetubaguy | mriedem: yeah, I am fine with that, thats like Friday right? | |
| 14:24:39 | mriedem | johnthetubaguy: yeah | |
| 14:24:41 | mriedem | i think so anyway | |
| 14:24:45 | sfinucan | mriedem: What he said ^ | |
| 14:24:46 | mriedem | i believe stable/ocata is created from rc1 | |
| 14:26:13 | bauzas | mriedem: it is | |
| 14:26:19 | bauzas | mriedem: we have to write the cut | |
| 14:26:34 | bauzas | mriedem: have you seen dhellmann's emails about that ? | |
| 14:26:41 | bauzas | I can do a WIP if you wish | |
| 14:26:54 | mriedem | bauzas: yes i know about it | |
| 14:26:55 | bauzas | and you would just update with the right SHA1 | |
| 14:27:50 | BobBall | I have a meeting in 3 minutes... Will have to look into the CI after the meeting. | |
| 14:28:07 | BobBall | mriedem / johnthetubaguy - suggest you remove the Citrix CI from voting until I can fix? | |
| 14:28:16 | bauzas | mriedem: okay, but I mean, you okay with writing the releases change ? | |
| 14:28:22 | BobBall | (or figure out what's going on...) | |
| 14:28:29 | bauzas | mriedem: that would create the branch | |
| 14:28:37 | mriedem | bauzas: yes | |
| 14:28:40 | bauzas | kewl | |
| 14:28:57 | mriedem | we have a lot of shit to fix before then :( | |
| 14:29:12 | mriedem | BobBall: looks like that's already taken care of https://review.openstack.org/#/admin/groups/511,members | |
| 14:29:21 | BobBall | +1 | |
| 14:30:59 | jroll | mriedem: think we should be using the same service token for ironic interactions (rather than separate auth configs we use today) | |
| 14:31:05 | jroll | slash johnthetubaguy ^ | |
| 14:32:50 | johnthetubaguy | mriedem: this one seems ready now, and good to have: https://review.openstack.org/#/c/421760/ | |
| 14:33:35 | mriedem | jroll: is that a question? | |
| 14:33:42 | mriedem | johnthetubaguy: yeah it's on the list | |
| 14:33:45 | johnthetubaguy | jroll: possibly, we need to work out what we want keystone roll wise. I quite like the "i am nova" service token being used for all the cases we want for that | |
| 14:34:37 | johnthetubaguy | jroll: I would rather we move to user token + nova service user, for all interactions where possible, so it clear where the request came from at all times | |
| 14:34:49 | jroll | mriedem: oh yeah, punctuation is hard, that's a question | |
| 14:34:54 | johnthetubaguy | jroll: need some oslo.policy tweaks to get that working though | |
| 14:35:24 | jroll | johnthetubaguy: yeah, need to think more about it, user token would be nice for e.g. hitting neutron on our side | |
| 14:35:38 | mriedem | we also don't know what the performance implications are yet | |
| 14:36:05 | johnthetubaguy | mriedem: very true, it very much depends on that not being total suck | |
| 14:36:09 | mriedem | i think the hope is, in pike we pick a ci job and turn the service user stuff on | |
| 14:36:24 | mriedem | *etherpad | |
| 14:36:39 | cdent | it's cuz you're talking to a brit | |
| 14:36:44 | johnthetubaguy | mriedem: I am asking our internal CI to do some proper performance trials with/without, but they don't sound too excited yet | |
| 14:36:56 | johnthetubaguy | cdent: heh | |
| 14:39:30 | openstackgerrit | Lee Yarwood proposed openstack/nova: WIP - libvirt: Remove redundant bdm serial mangling and saving during swap_volume https://review.openstack.org/427364 | |
| 14:39:45 | openstackgerrit | Chris Dent proposed openstack/nova: Add placement request id to log when GET or POST rps https://review.openstack.org/427740 | |
| 14:41:45 | jroll | johnthetubaguy: mriedem: ok, thanks, will revisit later on :) | |
| 14:42:17 | mriedem | jroll: didn't you get your fill of ironic features into nova in ocata? | |
| 14:42:30 | mriedem | i feel like i reviewed an endless stream of feature parity patches :) | |
| 14:43:09 | jroll | mriedem: not features, quality and such | |
| 14:45:44 | cdent | jaypipes: I'm gonna go get some air, but if you want me (because you're busy or whatever) to get those tweaks on the ironic inventory stuff, leave me a message somewhere and I will | |
| 14:45:51 | mriedem | panda: bnemec: did you guys get your issue resolved with novaclient 7.1.0? | |
| 14:46:05 | jaypipes | cdent: nah, I'll take care of em. thanks for reviewing. | |
| 14:46:11 | cdent | ✔ | |
| 14:46:17 | mriedem | johnthetubaguy: jaypipes: how realistic is that patch at this point? | |
| 14:46:47 | jaypipes | mriedem: trying my best. I think I should be able to get it ready today. | |
| 14:46:57 | jaypipes | mriedem: I added a bunch of functional test to the patch yesterday. | |
| 14:47:01 | openstack | Launchpad bug 1660878 in OpenStack Compute (nova) "test_reboot_deleted_server fails with 409 "Cannot 'reboot' instance while it is in vm_state building"" [High,Confirmed] | |
| 14:47:01 | bauzas | mriedem: mmm, just saw https://bugs.launchpad.net/nova/+bug/1660878 | |
| 14:47:17 | mriedem | bauzas: yes it's in the ML too | |
| 14:47:21 | mriedem | i have a debug patch up for it | |
| 14:47:27 | mriedem | https://review.openstack.org/#/c/427544/ | |
| 14:48:23 | bauzas | mriedem: not sure I saw this thread, looking | |
| 14:48:23 | panda | mriedem: we think yes, but we have problem merging the change since puppet syntax gate is failing due to a new deprecation warning. | |
| 14:49:14 | mriedem | bauzas: it's the papercuts thread | |
| 14:49:35 | openstack | Launchpad bug 1660878 in OpenStack Compute (nova) "test_reboot_deleted_server fails with 409 "Cannot 'reboot' instance while it is in vm_state building"" [High,Confirmed] | |
| 14:49:35 | mriedem | bauzas: the thread doesn't matter, the bug has the details https://bugs.launchpad.net/nova/+bug/1660878 | |
| 14:49:40 | bauzas | oh ok | |
| 14:49:58 | bauzas | not [nova] tagged :( | |
| 14:50:04 | bauzas | I saw it, but passed it | |
| 14:50:09 | bauzas | s/passed/skipped | |
| 14:50:11 | sfinucan | bauzas: Just spotted your review on https://review.openstack.org/#/c/392643/. Thank you :) | |
| 14:50:19 | bauzas | I should hire an assistant | |
| 14:50:21 | mriedem | bauzas: melwitt: dansmith: looks like i got a hit on my debug patch http://logs.openstack.org/44/427544/2/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/f50a5ee/logs/screen-n-cond.txt#_2017-02-01_04_49_11_702 | |
| 14:50:46 | bauzas | like the one our French Presidency candidate is hiring | |
| 14:53:08 | jaypipes | "Harry Harrington" is officially the best-named new OpenStack core nominee. | |
| 14:54:53 | mriedem | dansmith: so i've got a recreate on that failure, we know the build request is gone before we create the instance in conductor, now i'm just not sure what i do with it - we don't put that in cell0 since if the build request is deleted before the instance is created, we just treat it like the instance was deleted, i.e. it wouldn't be in ERROR state in cell0 | |
| 14:55:11 | mriedem | i think i can just return | |
| 14:55:27 | dansmith | mriedem: ah I misunderstood your assertion last night then | |
| 14:55:43 | dansmith | mriedem: yeah, if build request is gone before we create the point is we're supposed to just bail quietly.. we do that in at least one place now | |
| 14:55:53 | mriedem | yup here https://review.openstack.org/#/c/427544/2/nova/conductor/manager.py@936 | |
| 14:55:58 | mriedem | which for this bug is too late | |
| 14:56:33 | dansmith | mriedem: ah, yep, indeed | |