| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-19 | |||
| 19:58:42 | efried | dansmith Super helpful article, thank you. | |
| 20:01:05 | penick_ | dansmith and melwitt thanks for the help, btw | |
| 20:01:32 | dansmith | penick_: no problem. Now I have to go shower the juno off me | |
| 20:02:10 | penick_ | Sorry about that. On the upside you don't need to bathe in the juno-ness all day every day. | |
| 20:04:19 | dansmith | sdague: we kinda lost momentum on mikal's privsepapalooza: https://review.openstack.org/#/c/494423/30 | |
| 20:08:08 | openstackgerrit | Merged openstack/nova master: Pick ironic nodes without VCPU set https://review.openstack.org/511844 | |
| 20:08:15 | openstackgerrit | Merged openstack/nova master: Add attachment_get to refresh_connection_info https://review.openstack.org/512626 | |
| 20:08:24 | openstackgerrit | Merged openstack/nova master: Update live migration to use v3 cinder api https://review.openstack.org/463987 | |
| 20:12:58 | mriedem | stvnoyes: ^ woot | |
| 20:14:20 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Add multiattach support to Nova https://review.openstack.org/499777 | |
| 20:14:42 | mriedem | johnthetubaguy: ^ updated the multiattach spec - i'll proxy your +2 if you're not around to handle it tonight | |
| 20:16:48 | mriedem | oh yeah today is the q-1 tag, suppose i should tag that | |
| 20:20:31 | cdent | mriedem, dansmith: since it is spec freezey day, and this spec is required by a spec already merged, you wanna drop some eyes on it? It was done as a seperate spec from the POST allocation (which is what requires it) because it’s a different microversion: https://review.openstack.org/#/c/508164/ please and thank you. | |
| 20:20:49 | cdent | It’s probably not strictly necessary that it merges as the work is going to be done anyway. It is mostly a bookkeeping thing I guess. | |
| 20:22:48 | mriedem | probably just gets a freeze exception given it's a dependency | |
| 20:23:24 | cdent | cool, thanks | |
| 20:58:37 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Add support for certificate validation https://review.openstack.org/488541 | |
| 21:10:04 | openstackgerrit | Merged openstack/nova-specs master: Add multiattach support to Nova https://review.openstack.org/499777 | |
| 21:12:19 | openstackgerrit | Merged openstack/nova-specs master: Add support for certificate validation https://review.openstack.org/488541 | |
| 21:12:29 | efried | mriedem I think you added me to https://review.openstack.org/#/c/334732/ to render opinion as a driver owner; did that. | |
| 21:13:27 | mriedem | efried: ok, takashi said it was for NFS in his use case too, and it takes "several minutes" for the disk to migrate, | |
| 21:13:40 | mriedem | which to me isn't a good use case for needing to add plumbing to abort the thing, | |
| 21:13:42 | efried | mriedem wha? How could it take minutes to "migrate" NFS? | |
| 21:13:47 | mriedem | if the admin is cold migrating the instance, it's for good reason | |
| 21:14:48 | mriedem | i think the live migrate cancel stuff was needed because live migrate could stall out and we needed a hook in to abort the job in the hypervisor | |
| 21:15:04 | efried | Okay, that's not what the spec says. | |
| 21:15:08 | mriedem | this is just like, whiny user says their cold migrating vm is down too long and tells the admin to stop doing that thing | |
| 21:15:23 | mriedem | right, this doesn't say anything about cold migrate stalling out and we need a way to kill it | |
| 21:15:28 | efried | The spec implies that we're copying gigabytes of local (*actual* local) disk over the wire. | |
| 21:15:36 | efried | ...and want to be able to bail on that. | |
| 21:15:38 | mriedem | this is saying, "admin is cold migrating, user doesn't like that it's taking several minutes, and wants the admin to stop' | |
| 21:16:23 | mriedem | idk maybe if there were actual numbers in here, | |
| 21:16:29 | mriedem | like, i've got a 200GB disk windows vm | |
| 21:16:36 | mriedem | and it's taking me 3 hours to cold migrate that thing | |
| 21:17:15 | mriedem | this is why i asked cburgess this morning b/c i know he uses ceph and i know he loves to migrate stuff | |
| 21:17:25 | cburgess | Oh sorry I missed something... | |
| 21:17:29 | cburgess | Do what now? | |
| 21:17:33 | stvnoyes | mridem. fantastic! :-) thanks for all the help from both you and johnthetubaguy! | |
| 21:18:05 | mriedem | cburgess: do you think https://review.openstack.org/#/c/334732/ is useful? | |
| 21:18:06 | mriedem | cburgess: do you have users with 100+ gb disks and if so, do you cold migrate those and if so, how long does that take? | |
| 21:18:53 | cburgess | mriedem Funny you should ask about 100+ GB disk... I'm just now starting at a 15TB one. | |
| 21:19:03 | cburgess | mriedem Give me a few to read this spec and I'll let you know. | |
| 21:19:09 | mriedem | thanks | |
| 21:21:01 | ildikov | stvnoyes: congrats!!! :) | |
| 21:37:15 | openstackgerrit | Chris Dent proposed openstack/nova master: WIP: [placement] Enable limiting GET /allocation_candidates https://review.openstack.org/513526 | |
| 21:37:31 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Add pagination and changes since filter support for os-instance-action API https://review.openstack.org/507762 | |
| 21:41:12 | mriedem | jaypipes: addressed your comments and sylvain's in https://review.openstack.org/#/c/507762/ - left a todo for alex's thing about sort order which i think we could hash out in review later | |
| 21:42:45 | cdent | night all | |
| 21:50:40 | openstackgerrit | Merged openstack/nova master: [Trivial] Fix up a docstring https://review.openstack.org/513447 | |
| 22:03:11 | cburgess | mriedem So yeah thats useful. We tend to not recommend cold migrations with large amounts of data. Clients that use a lot of space we push towards a volume solution for the large storage and to either use BFV or to have small roots locally and put the bulk storage in volumes. | |
| 22:03:15 | cburgess | But its still a useful feature. | |
| 22:03:37 | mikal | Random fact for the day. Name the commit which added metadata to nova. | |
| 22:04:02 | cburgess | mikal: Which part of metadata? The service? | |
| 22:04:18 | mikal | There has always been a web service | |
| 22:04:23 | mriedem | cburgess: want to comment on the spec? operator input has weight on something like this. | |
| 22:04:28 | mikal | So, the commit which added any instance accessible metadata functionality | |
| 22:04:46 | cburgess | mriedem Sure | |
| 22:04:49 | mriedem | "copy what ec2 / aws does"? | |
| 22:04:54 | cfriesen | mriedem: for what it's worth, the largest EC2 instances have 48TB of disk | |
| 22:05:02 | cburgess | mikal Um... probably somewhere in in the first few commits if not the initial import. | |
| 22:05:39 | cfriesen | mriedem: for the "abort cold migration" thing, what about the case where the dest node crashes mid-transfer? | |
| 22:05:54 | mriedem | cfriesen: wouldn't that auto-kill the cold migrate? | |
| 22:05:56 | cburgess | cfriesen I think thats covered already isn't it? Don't we detect that? | |
| 22:06:15 | cburgess | mriedem cfriesen Oh maybe not now that we can't monitor disk transfer progress. | |
| 22:06:17 | cfriesen | I'm thinking the connection could stay open until the TCP connection times out | |
| 22:06:31 | cfriesen | in the scp case, at least | |
| 22:06:32 | cburgess | Oh right... libvirt will report the error back. | |
| 22:06:39 | cburgess | At some point | |
| 22:06:40 | cburgess | In theory | |
| 22:06:54 | cburgess | mikal So whats the answer? | |
| 22:07:10 | mikal | cburgess: commit bf6e6e718cdc7488e2da87b21e258ccc065fe499 | |
| 22:07:17 | mikal | cburgess: which has the commit message "initial import" | |
| 22:07:32 | mikal | cburgess: at that point, nova was 5,083 lines of code | |
| 22:07:52 | cburgess | mikal Called it! | |
| 22:08:45 | mikal | In other words, its always been weird | |
| 22:08:54 | cburgess | wow 5K lines of code. I have scripts to fix horked up volumes that are longer then that... | |
| 22:09:40 | cburgess | OK thats a lie.. its only 1349 lines of code. | |
| 22:19:20 | cburgess | mriedem Is the existing abort admin only? Or is that policy controlable? | |
| 22:19:46 | mriedem | cburgess: anytihng related to live migration is admin-only by default | |
| 22:19:55 | mriedem | and cold migration | |
| 22:20:16 | cburgess | mriedem OK | |
| 22:20:48 | mikal | cburgess: I'm not sure volumes existed in the first commit, I'd have to check... | |
| 22:21:13 | cburgess | mikal What do volumes have to do with anything? | |
| 22:21:28 | mikal | cburgess: well, your volume fix script... | |
| 22:21:36 | mikal | cburgess: probably wasn't required back then | |
| 22:21:55 | cburgess | mikal My volume fix script wasn't required until the hot mess that is/was os-brick circa liberty. | |
| 22:28:50 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Use neutron's new port binding API for live migration https://review.openstack.org/375580 | |
| 22:34:57 | mriedem | dansmith: i updated ^ based on some review i had after sean updated it from the call this morning | |
| 22:35:09 | mriedem | dansmith: one sticky thing in there yet which i know you won't like, | |
| 22:35:14 | mriedem | but i think we can hammer in code review | |
| 22:36:23 | dansmith | yeah I don't really think the argument of "avoid changing the object due to stuff in ml2" is sound | |
| 22:36:40 | dansmith | but as long as we can argue in the code I guess it's fine | |
| 22:45:11 | openstackgerrit | Merged openstack/nova-specs master: Use neutron's new port binding API for live migration https://review.openstack.org/375580 | |
| 23:26:05 | openstackgerrit | Merged openstack/nova-specs master: Add pagination and changes since filter support for os-instance-action API https://review.openstack.org/507762 | |
| 23:51:18 | gmann | bauzas: i replied on your comment on this spec - https://review.openstack.org/#/c/508101/7 | |
| 23:52:02 | gmann | bauzas: let me know if i need to make it clear about those 'Main policy' in spec. i can do that quickly | |
| #openstack-nova - 2017-10-20 | |||
| 00:30:12 | openstackgerrit | Ghanshyam Mann proposed openstack/nova-specs master: Spec for API extensions policy removal https://review.openstack.org/508101 | |
| 00:30:29 | gmann | bauzas: ^^ | |
| 01:33:42 | yikun_jiang | liusheng, test | |
| 01:48:36 | openstackgerrit | garyk proposed openstack/nova master: Add debug information to metadata requests https://review.openstack.org/511895 | |