| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-08 | |||
| 17:31:15 | johnthetubaguy | pumaranikar: there was a good comment on the git commit message too I think | |
| 17:31:37 | pumaranikar | johnthetubaguy, yeah. I will address those two and let you know. | |
| 17:31:48 | johnthetubaguy | pumaranikar: cool, thank you | |
| 17:37:59 | EmilienM | cdent: we've unpinned nova in rdo and I did recheck on https://review.openstack.org/#/c/430571/ and it pass | |
| 17:38:11 | EmilienM | cdent: which means we might drink gin tonic tonight | |
| 17:38:23 | EmilienM | but i'll confirm when I'll see tripleo Ci promotion jobs | |
| 17:38:38 | cdent | What does this mean about why it was failing before? | |
| 17:39:22 | openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove redundant bdm serial mangling and saving during swap_volume https://review.openstack.org/427364 | |
| 17:40:12 | lyarwood | jaypipes: ^ thanks for review, I've updated the comment and added a quick check before we stash the serial (as we do elsewhere during attach). | |
| 17:40:16 | johnthetubaguy | cdent: does that sound like a race to you, or is that just me? | |
| 17:40:30 | jaypipes | lyarwood: cool, thanks! will re-review shortly. | |
| 17:40:50 | EmilienM | cdent: tbh, I don't know | |
| 17:40:55 | cdent | johnthetubaguy: it sounds like we have insufficient data to say much of anything :( | |
| 17:41:07 | johnthetubaguy | cdent: thats more accurate | |
| 17:41:16 | cdent | we can't fully articulate what is different from attempt to attempt | |
| 17:41:21 | dansmith | cdent: if it passes some of the time and doesn't other times, I think that's the definition of a race :) | |
| 17:41:22 | cdent | just that things are different | |
| 17:41:36 | cdent | dansmith: "unpinned nova" | |
| 17:41:57 | dansmith | cdent: this was on the run to unpin nova yes? it failed once, and passed on a recheck | |
| 17:42:34 | cdent | Oh, I didn't get that from what EmilienM said. Was it the exact same stuff in each run? I thought not? | |
| 17:42:48 | dansmith | that's my understanding of it | |
| 17:43:35 | cdent | Because if you're right dansmith then I think EmilienM is going to the g&t's early, because we've learned nothing. | |
| 17:44:30 | dansmith | [07:27:24] <EmilienM>dansmith: ok I'm doing recheck and in 40m if I still see multinode failing, I'll unpin Nova in RDO and see how it goes | |
| 17:44:30 | dansmith | maybe I misunderstood this: | |
| 17:44:43 | dansmith | I thought he meant he was rechecking the unpin | |
| 17:44:47 | EmilienM | I did the recheck and it passed | |
| 17:44:55 | EmilienM | which means, I could deploy tripleo with nova from master | |
| 17:45:24 | EmilienM | so we're unpinning now in RDO and i'll check how tripleo work | |
| 17:45:33 | EmilienM | we should know in ~1h | |
| 17:45:42 | EmilienM | cdent: if I see you offline, i'll just email you | |
| 17:45:54 | cdent | EmilienM: when you did the recheck, was that the exact same code (everywhere) as earlier in the day, or some other code? | |
| 17:46:21 | dansmith | I don't even understand how 430571 tests anything different actually | |
| 17:46:28 | dansmith | it depends on the wsgi patch | |
| 17:46:36 | dansmith | but nothing to unpin nova afaict | |
| 17:46:42 | dansmith | even though it says it does | |
| 17:48:07 | EmilienM | cdent: the exact same in theory | |
| 17:48:17 | cdent | "in theory"? | |
| 17:48:27 | EmilienM | yeah, the version of openstack-nova rpm was the same | |
| 17:48:36 | dansmith | it doesn't depend on the wsgi patch actually | |
| 17:48:45 | dansmith | it depends on the nova canary that was depending on the wsgi patch | |
| 17:48:47 | EmilienM | but 'delorean' builds packages from git master so maybe something merged in the meantime? | |
| 17:48:57 | EmilienM | dansmith: yeah it's a canary | |
| 17:49:13 | dansmith | EmilienM: but why depend on the nova patch that does nothing from the tripleo patch? | |
| 17:49:32 | EmilienM | dansmith: to run a job that didn't run in nova gate | |
| 17:49:34 | EmilienM | dansmith: ovb-ha | |
| 17:49:54 | EmilienM | check experimental in nova doesn't run ovb-ha | |
| 17:49:56 | dansmith | EmilienM: how does the depends-on accomplish that? | |
| 17:50:26 | dansmith | EmilienM: I don't think anything is likely to have merged related to this in the last few hours, and even still, if you're not testing the rc1 branch you're not testing ocata anyway | |
| 17:50:59 | EmilienM | dansmith: our Ci tooling read Zuul env and build packages | |
| 17:51:15 | dansmith | EmilienM: if you wanted the nova experimental job to run on the tripleo change the depends-on should go the other way no? | |
| 17:51:24 | EmilienM | dansmith: right, gimme some hours to figure if it works now and if it's not racy & if it's stable | |
| 17:51:55 | EmilienM | I wanted to see ovb-ha so I did https://review.openstack.org/#/c/430571/ which depends-on a nova patch | |
| 17:52:43 | EmilienM | so our ci tooling built the nova patch and ran ovb-ha | |
| 17:52:51 | EmilienM | and it failed. That's the logs we investigated this morning | |
| 17:53:03 | EmilienM | but I did recheck and it pass | |
| 17:53:09 | EmilienM | that's where we are now | |
| 17:53:56 | dansmith | that's not how depends-on works for any other jobs that I know of, but okay | |
| 17:54:33 | oomichi | mriedem: one comment on https://review.openstack.org/#/c/430497 | |
| 18:02:18 | oomichi | Kevin_Zheng: gmann puts a question on https://review.openstack.org/#/c/403568/18/nova/compute/manager.py | |
| 18:16:45 | mriedem | oomichi: replied | |
| 18:16:56 | oomichi | sfinucan: can you take a look at https://review.openstack.org/#/c/389338/ again ? | |
| 18:17:29 | dansmith | mriedem: can you re-hit https://review.openstack.org/430354 ? | |
| 18:18:34 | sfinucan | oomichi: Done and done | |
| 18:18:43 | oomichi | sfinucan: thanks :) | |
| 18:18:56 | cfriesen | dansmith: can you check my question earlier about commit 2c0a306? I think that was your commit. | |
| 18:18:58 | sfinucan | any time :) | |
| 18:19:22 | mriedem | dansmith: yeah | |
| 18:19:42 | mriedem | gabor_antal: fyi https://review.openstack.org/#/c/389338/ | |
| 18:19:46 | openstackgerrit | John Garbutt proposed openstack/nova master: Default live_migration_progress_timeout to off https://review.openstack.org/429798 | |
| 18:20:10 | gabor_antal | mriedem: thanks, I already saw that patchset :) | |
| 18:20:29 | gabor_antal | And built mine on the top of it: https://review.openstack.org/#/c/431030/ | |
| 18:22:14 | dansmith | cfriesen: yeah, it probably did drop a little of that, although libvirt was spawn'ing in its driver too, so just adding those to the helper thread might affect more behavior than you expect | |
| 18:24:13 | cfriesen | dansmith: looking at @wrap_instance_event in particular, won't that generate start/finish events potentially before we even do any work? | |
| 18:24:42 | dansmith | cfriesen: yeah, but that didn't wait until it was complete before anyway because libvirt was doing its own spawn'ing | |
| 18:25:28 | cfriesen | dansmith: fair enough, but we used to at least do the pre_live_migration stuff before that spawn in libvirt | |
| 18:25:57 | dansmith | cfriesen: well, we also used to exhaust the rpc worker pool, but.. yeah :) | |
| 18:26:55 | cfriesen | dansmith: the reason I noticed this was we tried using the same model with cold migrations, but got bit by no longer having the errors_out_migration wrapper | |
| 18:28:08 | dansmith | cfriesen: looks like the other drivers were all synchronous, so maybe just moving it to the thread function would be okay, but it's definitely another change | |
| 18:29:10 | cfriesen | dansmith: okay, thanks for the retrospective. | |
| 18:35:35 | oomichi | mriedem: thanks, replied again | |
| 18:35:50 | cfriesen | mdbooth: you around? | |
| 18:36:14 | oomichi | sdague: can you take a look at https://review.openstack.org/#/c/389338/ again ? | |
| 18:39:47 | openstackgerrit | John Garbutt proposed openstack/nova master: Remove stuck live-migration detection https://review.openstack.org/430218 | |
| 19:00:18 | jaypipes | mdbooth, dansmith, mriedem1: in your opinion, has boot-from-volume been the "feature" added to Nova that has resulted in the most bugs, non-cloudy behaviour, and general hackiness? | |
| 19:00:42 | dansmith | jaypipes: it's definitely up there on the list | |
| 19:00:45 | dansmith | and one of my most hated | |
| 19:02:02 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Run api sample tests against 2.latest https://review.openstack.org/430352 | |
| 19:02:42 | alaski | jaypipes: dansmith +1 | |
| 19:04:30 | melwitt | it's so notorious we had to acronym it: bfv | |
| 19:04:45 | mriedem | non-cloudy behavior? | |
| 19:04:57 | mriedem | seems more cloudy than shelve... | |
| 19:05:14 | mriedem | the combination of ways to boot from volume and attach block devices is insane though | |
| 19:05:31 | dansmith | to me, bfv is 100% non-cloudy | |
| 19:05:43 | dansmith | so I'm not sure how something can be less cloudy | |
| 19:05:56 | dansmith | equally uncloudy perhaps | |
| 19:06:24 | mriedem | because there is a volume involved? | |
| 19:06:58 | alaski | I think it would be a perfectly cromulent feature if Nova was not involved in managing the lifecycle of the volume at all | |
| 19:07:56 | dansmith | no, volumes are fine | |
| 19:08:06 | mriedem | yeah so i'm asking to distinguish the orchestration vs having a volume attached | |
| 19:08:08 | dansmith | volumes in general I mean | |