| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-18 | |||
| 16:12:59 | mriedem | jgwentworth: i think there always is, | |
| 16:13:00 | superdan | jgwentworth: yeah, there's the one that is talking about the actual progress being made, | |
| 16:13:08 | mriedem | because the entire first session is recapping everything already said before | |
| 16:13:08 | superdan | and then another about ... not that | |
| 16:14:14 | superdan | mriedem: that's a nice way of putting it | |
| 16:14:32 | jgwentworth | superdan: are you referring to the fast-forward upgrades talk? because that is a THIRD thing | |
| 16:14:32 | superdan | and the first one has two parts, IIRC | |
| 16:14:47 | superdan | jgwentworth: oh you were just commenting that there are two parts to the first one? | |
| 16:15:04 | mriedem | I and Ii | |
| 16:15:14 | jgwentworth | oh, yes. I didn't think to think of it as one | |
| 16:15:15 | mriedem | FFU IV is when FFU fights the russians | |
| 16:15:26 | superdan | based on previous history, I think we need two hours because we'll spend a bunch of time talking about what FFU isn't actually doing but the moderator wants to be doing | |
| 16:15:54 | jgwentworth | :) | |
| 16:16:11 | superdan | or rather, what the moderator wants other people to do | |
| 16:16:27 | mriedem | upgrade fastly with no downtime? | |
| 16:17:03 | leakypipes | mriedem: he will crush you. | |
| 16:17:16 | mriedem | FFU is like iron | |
| 16:17:43 | jgwentworth | debating whether to put the two-part FFU session on this mail | |
| 16:19:07 | jgwentworth | maybe not, it's not too nova-related last time I've gone | |
| 16:20:10 | superdan | agree | |
| 16:20:29 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send resource allocations in the port binding https://review.openstack.org/569459 | |
| 16:26:54 | leakypipes | gibi: safe travels mate. | |
| 16:28:28 | openstackgerrit | Dan Smith proposed openstack/nova master: DNM: Testing a thing https://review.openstack.org/569461 | |
| 16:28:33 | gibi | leakypipes: thanks | |
| 16:28:45 | openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova master: Remove the remaining of the removed option https://review.openstack.org/569462 | |
| 16:29:17 | openstackgerrit | Merged openstack/nova master: Update the deprecate os_region_name option https://review.openstack.org/567829 | |
| 16:30:52 | openstackgerrit | Merged openstack/nova master: Fix inconsistency in docs https://review.openstack.org/569348 | |
| 16:34:09 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: Localdisk https://review.openstack.org/549300 | |
| 16:54:42 | superdan | mriedem: so, I don't think my instance change actually caused that error | |
| 16:54:54 | superdan | mriedem: here's another example from a different unrelated patch: http://logs.openstack.org/38/564838/6/gate/nova-cells-v1/5fc0493/logs/screen-n-cell-child.txt.gz?level=TRACE#_May_18_14_09_53_542120 | |
| 16:56:08 | superdan | mriedem: my DNM testing patch above should help uncover what it's actually choking on | |
| 17:38:32 | superdan | mriedem: that cells failure is trying to load keypairs | |
| 17:39:06 | superdan | mriedem: which is probably because the instance that came out of the cell database can't have loaded that from the api db down there or some such | |
| 17:40:30 | jgwentworth | I wonder why the cells v1 job is otherwise on the fritz today | |
| 17:40:48 | jgwentworth | seeing a lot of fails and rechecks in my gerrit notifications | |
| 17:40:59 | superdan | jgwentworth: what I just said :) | |
| 17:41:17 | jgwentworth | oh, that change already landed? I had thought it was stuck on the cells v1 job | |
| 17:41:30 | jgwentworth | if so, nevermind | |
| 17:41:35 | superdan | no | |
| 17:41:40 | superdan | it's unrelated to that change | |
| 17:42:16 | jgwentworth | oh. I thought when you mentioned failure to load keypairs it was in the context of the "load everything" patch | |
| 17:42:38 | jgwentworth | so weird that it's suddenly acting up if not for that | |
| 17:42:56 | superdan | we thought it was caused by that patch but it's not | |
| 17:43:06 | superdan | yeah I dunno why it just started coming up, | |
| 17:43:10 | superdan | but it's clearly racy so .. | |
| 17:43:16 | jgwentworth | yeah ++ | |
| 17:44:55 | superdan | I'm going to put it underneath the bottom one though just so it will squash the issue and make it easier to land that set | |
| 17:47:01 | superdan | heh we actually have a test that ensures that refresh does this exactly | |
| 17:47:36 | superdan | but I don't think the cells code can do anything about this, so probably better to just alter the refresh() behavior | |
| 17:51:58 | jgwentworth | guh, I gotta pack | |
| 18:12:59 | openstackgerrit | Dan Smith proposed openstack/nova master: Make instance.refresh() avoid recursion better https://review.openstack.org/569461 | |
| 18:13:00 | openstackgerrit | Dan Smith proposed openstack/nova master: Make instance able to lazy-load almost everything https://review.openstack.org/569267 | |
| 18:13:01 | openstackgerrit | Dan Smith proposed openstack/nova master: Expose instance_get_all_uuids_by_host() from DB API and use it https://review.openstack.org/569268 | |
| 18:13:18 | superdan | first one there ^ should fix the cellsv1 thing | |
| 18:18:10 | mriedem | superdan: ok i was wondering how your change would have caused that | |
| 18:18:18 | openstackgerrit | Merged openstack/nova master: Fix typo in NUMATopologyFilter docs https://review.openstack.org/543505 | |
| 18:26:19 | mriedem | jgwentworth: superdan: this is why we're seeing the spike in cells v1 job failures https://review.openstack.org/#/c/563851/ | |
| 18:26:21 | mriedem | the test is new | |
| 18:26:33 | superdan | ah | |
| 18:26:51 | superdan | how fail-y is it? | |
| 18:27:19 | mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ServerShowV254Test%5C%22%20AND%20message%3A%5C%22rebuild_instance%5C%22%20AND%20message%3A%5C%22OrphanedObjectError%5C%22%20AND%20tags%3A%5C%22screen-n-cell-child.txt%5C%22&from=7d | |
| 18:27:44 | mriedem | it's a new rebuild test + key_name change | |
| 18:27:51 | mriedem | that doesn't really need to live in tempest... | |
| 18:27:53 | mriedem | gmann: ^ | |
| 18:28:47 | superdan | mriedem: that didn't load anything for me | |
| 18:29:06 | mriedem | yeah fixing the query | |
| 18:31:13 | mriedem | we seem to have lost multiline error logging, which is why that doesn't work | |
| 18:31:32 | superdan | is it 100% or no? | |
| 18:31:55 | mriedem | idk, but i'm seeing a lot of stuff failing with that | |
| 18:32:04 | superdan | okay | |
| 18:32:25 | mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ServerShowV254Test%5C%22%20AND%20message%3A%5C%22Error%20processing%20message%20locally%3A%20OrphanedObjectError%3A%20Cannot%20call%20obj_load_attr%20on%20orphaned%20Instance%20object%5C%22%20AND%20tags%3A%5C%22screen-n-cell-child.txt%5C%22&from=7d | |
| 18:32:37 | mriedem | https://bugs.launchpad.net/nova/+bug/1772088 | |
| 18:32:38 | openstack | Launchpad bug 1772088 in OpenStack Compute (nova) "ServerShowV254Test.test_rebuild_server fails with OrphanedObjectError in cells v1" [High,In progress] - Assigned to Dan Smith (danms) | |
| 18:33:28 | jgwentworth | mriedem: ack, nice find | |
| 18:34:21 | superdan | 45 since yesterday so probably not 100% | |
| 18:34:23 | superdan | but clearly bad | |
| 18:34:24 | mriedem | fried_rice: happy with this now https://review.openstack.org/#/c/549300/ | |
| 19:02:27 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: Update PowerVM hypervisor docs https://review.openstack.org/569497 | |
| 19:04:06 | openstackgerrit | Arvind Nadendla proposed openstack/nova master: WIP:Handle rebuild of instances with image traits https://review.openstack.org/569498 | |
| 19:04:10 | mriedem | superdan: +2 on https://review.openstack.org/#/c/569461/ if you can find another core | |
| 19:05:38 | superdan | cool | |
| 19:07:57 | fried_rice | mriedem: ack, reviewing the bug fix that's standing in the way first... | |
| 19:36:38 | openstackgerrit | Eric Fried proposed openstack/nova master: Normalize inventory from update_provider_tree https://review.openstack.org/569132 | |
| 19:36:39 | openstackgerrit | Eric Fried proposed openstack/nova master: libvirt: get_inventory => update_provider_tree https://review.openstack.org/560444 | |
| 19:36:40 | openstackgerrit | Eric Fried proposed openstack/nova master: libvirt: Don't report DISK_GB if sharing https://review.openstack.org/560459 | |
| 19:52:38 | openstackgerrit | Matthew Edmonds proposed openstack/nova master: Update PowerVM hypervisor docs https://review.openstack.org/569497 | |
| 19:57:20 | openstackgerrit | Arvind Nadendla proposed openstack/nova master: WIP:Handle rebuild of instances with image traits https://review.openstack.org/569498 | |
| 21:05:41 | openstackgerrit | Arvind Nadendla proposed openstack/nova master: WIP:Handle rebuild of instances with image traits https://review.openstack.org/569498 | |
| 21:08:45 | mriedem | boo live migration job failed | |
| 21:12:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update PowerVM hypervisor docs https://review.openstack.org/569497 | |
| 21:20:24 | fried_rice | son of a | |
| 21:21:19 | fried_rice | superdan: mriedem: I noticed that patch seems to be parented pretty far back in history. Perhaps since we need to recheck it anyway, we could rebase it? | |
| 21:34:23 | superdan | fried_rice: eh? | |
| 21:34:35 | fried_rice | The cells recursion fix | |
| 21:34:42 | superdan | fried_rice: it's the base of a series | |
| 21:34:47 | superdan | it can't go any earlier | |
| 21:35:02 | fried_rice | I'm saying the series could be rebased on a more current version of master. | |
| 21:35:19 | superdan | that will only further delay it getting in | |
| 21:35:33 | fried_rice | superdan: how so? It needs to be rechecked anyway. | |
| 21:35:44 | superdan | does it? | |