| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-21 | |||
| 18:42:25 | cfriesen | sean-k-mooney: cool, thanks | |
| 18:46:08 | mriedem | dansmith: commented on mdbooth's change https://review.openstack.org/#/c/551302/ | |
| 18:46:15 | mriedem | dansmith: overall it's ok, but i think we also have a problem in rollback | |
| 18:48:02 | dansmith | mriedem: cool thanks | |
| 18:53:56 | mriedem | i looked at his live migration test patch that enabled the volume-backed live migration tests, | |
| 18:54:04 | mriedem | the grenade live migration job passed, which uses mixed computes | |
| 18:54:16 | mriedem | and goes back and forth (rocky->queens->rocky) | |
| 18:54:20 | mriedem | and vice-versa | |
| 18:54:34 | mriedem | the other non-grenade live migration job failed, looks like all rpc messaging timeouts | |
| 18:57:04 | mriedem | huh, same thing in melwitt's patch to enable the volume-backed live migratoin tests http://logs.openstack.org/04/528104/6/check/legacy-tempest-dsvm-multinode-live-migration/ff90ecb/logs/subnode-2/screen-n-cpu.txt.gz?level=TRACE | |
| 18:57:25 | mriedem | i wonder if there is just something about doing a volume-backed live migration that takes that much longer such that pre_live_migration rpc call times out | |
| 18:58:19 | melwitt | yeah, I had been rechecking the patch periodically but it never passed reliably upon multiple rechecks | |
| 18:58:35 | mriedem | i'm seeing the same rpc timeouts in your patch and https://review.openstack.org/#/c/553377/ | |
| 18:58:37 | mriedem | which depends on your patch | |
| 18:58:40 | melwitt | yeah | |
| 18:59:26 | melwitt | I just mean I had been checking on it to see if we were going to be able to re-enable those tests but it wasn't panning out. will try depending on your queens uca patch | |
| 18:59:29 | mriedem | you could try setting the rpc timeout to 120 in nova-cpu.conf in the job config and see if that makes a difference | |
| 18:59:44 | mriedem | i don't think libvirt is the problem here | |
| 18:59:51 | melwitt | okay, will do that instead | |
| 19:00:42 | mriedem | this is how you do something like that https://review.openstack.org/#/c/549789/9/playbooks/legacy/nova-cells-v1/run.yaml@38 | |
| 19:00:55 | mriedem | except you'll use $NOVA_CPU_CONF | |
| 19:01:10 | mriedem | oh, but, | |
| 19:01:20 | mriedem | first you need to move the live migratoin job defs in tree | |
| 19:01:23 | mriedem | something i've been thinking about doing anyway | |
| 19:02:13 | melwitt | hm, okay. (on a call atm) | |
| 19:04:58 | dansmith | or I could finish my live heartbeating thing in oslo.messaging | |
| 19:09:52 | mriedem | or you could take this over https://review.openstack.org/#/c/452546/ | |
| 19:10:00 | mriedem | like you PROMISED at the PTG | |
| 19:10:31 | dansmith | um, wut? | |
| 19:11:00 | dansmith | I thought you said you were going to do that? | |
| 19:11:20 | mriedem | oh on | |
| 19:11:23 | mriedem | *no | |
| 19:11:28 | mriedem | i said i had tried at one point and was shot down | |
| 19:11:35 | mriedem | and was welcome to others getting shot | |
| 19:12:15 | mriedem | btw i think the cellsv1 + neutron job is ready to go https://review.openstack.org/#/c/549789/ | |
| 19:12:30 | mriedem | i can't drop the old cells v1 job from master until that flushes through | |
| 19:14:15 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: DiskAdapter parent class https://review.openstack.org/549053 | |
| 19:14:34 | openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: Localdisk https://review.openstack.org/549300 | |
| 19:14:46 | dansmith | mriedem: that's a long ways from "promised" :) | |
| 19:15:03 | mriedem | i know, i was hoping the caps would convey the joke | |
| 19:15:38 | mriedem | like, "omfg there was pug shit EVERYWHERE!" | |
| 19:16:31 | dansmith | mriedem: so this patch's cells job came from the in-tree version https://review.openstack.org/#/c/549780/2 ? | |
| 19:16:58 | dansmith | ah, I guess there are two on there | |
| 19:18:21 | dansmith | mriedem: your job's cells-child log has a little extra red in it from the base job: http://logs.openstack.org/80/549780/2/check/nova-cells-v1/66d336e/logs/screen-n-cell-child.txt.gz?level=TRACE | |
| 19:18:31 | dansmith | although that looks vaguely familiar, so maybe not a problem | |
| 19:18:58 | mriedem | i've seen those before | |
| 19:19:03 | mriedem | race in the metadata updates | |
| 19:19:08 | dansmith | okay | |
| 19:19:23 | mriedem | so yeah what i did was move the existing job in-tree | |
| 19:19:28 | mriedem | and renamed it | |
| 19:19:34 | mriedem | b/c that's what we do when we move them in tree, | |
| 19:19:47 | mriedem | and then tweaked it to be the new thing with neutron | |
| 19:20:00 | dansmith | so after we merge that first one, we can remove the base job from infra, right? | |
| 19:20:06 | mriedem | in https://review.openstack.org/#/c/549789/ we are getting both jobs b/c at this point the legacy job is still in openstack-zuul-jobs | |
| 19:20:14 | mriedem | yeah i have all of thoes patches lined up | |
| 19:20:17 | mriedem | https://review.openstack.org/#/q/topic:bp/remove-nova-network+(status:open+OR+status:merged) | |
| 19:24:25 | dansmith | mriedem: cool, +2 on the bottom two | |
| 19:24:37 | dansmith | melwitt: ^ | |
| 19:44:00 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Preserve multiattach flag when refreshing connection_info https://review.openstack.org/555029 | |
| 19:44:13 | openstackgerrit | Matt Riedemann proposed openstack/os-vif stable/pike: ovs: do not delete port if already exists https://review.openstack.org/550080 | |
| 19:53:56 | mriedem | to do that from. | |
| 19:53:56 | mriedem | dansmith: i skimmed the review comments on https://review.openstack.org/#/c/452546/ again and there were some todos that came out of that as prereqs, one of which was tagged attach which we've had since pike. there was another about returning bdm tags out of the volume attachments API. i had a separate spec for that which got held up in committee because of local disk tags and also exposing vifs tags in GET requests but which | |
| 19:54:30 | mriedem | one thing that could move it forward was just remove device_name from the volume attach API (not bfv), because that doesn't have the sneaky ec2 thing that ftersin was -1ing this for | |
| 19:55:35 | dansmith | mriedem: ah, that seems like incremental improvement and probably the 90% case where people attach a volume and expect it to go in a certain place | |
| 19:57:05 | mriedem | the device name is also presumably for correlating on the guest right? | |
| 19:57:26 | mriedem | i know this volume has something in it that my app needs, so i'll attach it at vdc and my guest will expect it to be at vdc | |
| 19:57:34 | dansmith | yeah | |
| 19:57:40 | dansmith | it's the only use for it | |
| 19:59:18 | mriedem | so i think what i'd propose, if i were to redo this, is (1) drop device_name from attach volume API, (2) return tags in GET calls to the os-volume_attachments and os-interface APIs | |
| 19:59:53 | mriedem | and then cross my fingers that john, booth, feodor and artom don't show up to review the spec | |
| 19:59:58 | dansmith | and take tags in volume/interface attach? | |
| 20:00:02 | dansmith | or do we already have that? | |
| 20:00:07 | mriedem | we've had that since pike | |
| 20:00:13 | mriedem | that was one of the pre-reqs for this other removal spec | |
| 20:00:24 | dansmith | on both interface and volume? but not sriov or something? | |
| 20:00:29 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: Network interface attach/detach https://review.openstack.org/546813 | |
| 20:00:34 | mriedem | same | |
| 20:00:47 | mriedem | sriov ports are attached the same way | |
| 20:00:55 | mriedem | they have to be pre-created in neutron is all with the special binding profile i think | |
| 20:01:10 | dansmith | I thought there was some way to attach something where we were still missing tag function | |
| 20:01:12 | mriedem | or are you referring to the bug that artom was trying to fix | |
| 20:01:36 | mriedem | https://review.openstack.org/#/c/533805/ ? | |
| 20:01:45 | dansmith | anyway, yes, I think making those three changes together makes sense. I need to refresh on the bfv/ec2 issue I guess, | |
| 20:01:54 | dansmith | because that doesn't sound like a great reason to remove it from bfv either but.. | |
| 20:02:01 | dansmith | to nop | |
| 20:02:03 | dansmith | not | |
| 20:02:22 | mriedem | yeah, i said the same to ftersin basically, it is completely undocumented and untested behavior | |
| 20:03:05 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: vSCSI volume driver https://review.openstack.org/526094 | |
| 20:04:16 | openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: Snapshot https://review.openstack.org/543023 | |
| 20:04:22 | artom | mriedem, I thought we matches volume tags based on device name? | |
| 20:04:31 | artom | *matched | |
| 20:05:04 | dansmith | mriedem: if we remove it, it prevents them from using the boot call past that microversion forever I guess? | |
| 20:05:27 | dansmith | mriedem: I guess I don't see why a tag on bfv won't work the same for them, | |
| 20:05:29 | mriedem | dansmith: if we removed device_name from the bdm object in server create, yeah. | |
| 20:05:40 | dansmith | but I also don't really understand the exchange between you two either | |
| 20:05:46 | dansmith | I think there's ML context probably | |
| 20:05:52 | mriedem | ftersin pointed out some stuff about image-defined BDMs don't have tags | |
| 20:06:00 | mriedem | it's in the spec review | |
| 20:06:07 | mriedem | artom: not sure | |
| 20:06:10 | mriedem | you wrote that code :) | |