| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-05 | |||
| 14:44:01 | dansmith | mriedem: yeah, but we need to know which instances are ironic and not, so I'm kinda thinking that maybe we should make a dedicated command that takes an ironic compute's hostname and migrates instances that are on that compute | |
| 14:44:11 | dansmith | which makes it not really fit the online-data-migrations model | |
| 14:44:23 | dtantsur | mriedem: hey! when do you folks plan on finalizing your PTG schedule? I specifically wonder about the scheduling discussion, as we have a couple of topics there | |
| 14:44:25 | mriedem | we can figure out the ironic nodes via the hypervisor_type on the compute_nodes table right? | |
| 14:44:37 | mriedem | dtantsur: it's something i need to work on this week | |
| 14:44:40 | dansmith | mriedem: we can yeah | |
| 14:44:52 | mriedem | it's pretty open ended atm, except for thursday morning being cinder time | |
| 14:45:58 | dtantsur | mriedem: we have a few topics depending on the scheduling discussion outcome to an extent. so I guess Wed would work better, probably afternoon. | |
| 14:46:05 | dtantsur | but it's up to you of course :) | |
| 14:46:08 | dtantsur | vdrok: ^^ | |
| 14:46:10 | dansmith | mriedem: so your preference would be to do it fully automatic like that? we kinda need to backport this if we're going to remove the migration from queens | |
| 14:46:25 | mriedem | dansmith: so the problem with the offline data migrations command with this is we don't really have a marker i don't think | |
| 14:46:29 | dtantsur | or just don't remove the migration from queens... | |
| 14:46:34 | mriedem | since the extra specs are a json blob | |
| 14:46:48 | mriedem | but i haven't thought it through fully | |
| 14:46:55 | dansmith | mriedem: yup, hence my thinking that a one-shot command for a host would be good | |
| 14:47:15 | dansmith | dtantsur: yeah, but we really want to stop allowing the old model too, which we *could* do separate from the migration piece, but... | |
| 14:47:20 | mriedem | for all compute nodes where hypervisor_type == 'ironic'; find all instances where host/node in that list; do migrate instance extra spec | |
| 14:47:36 | dansmith | the migration piece in compute also brings overhead | |
| 14:47:39 | mriedem | dansmith: i wouldn't want to require a host, but it could be an option | |
| 14:48:22 | dansmith | mriedem: well, for a dedicated command I was thinking make it specifically "run for instances on this host" so you're responsible for iterating hosts and therefore not running twice | |
| 14:50:35 | mriedem | how do you know you've hit all hosts? | |
| 14:50:45 | mriedem | and then someone will say, how do i get the list of hosts | |
| 14:51:03 | dansmith | well, I'm thinking about tripleo where that's easy to determine | |
| 14:51:03 | mriedem | running twice isn't a problem either right? it's just a no-op if already migrated | |
| 14:51:10 | dansmith | sure, it's just super expensive | |
| 14:51:12 | mriedem | i'm generally never thinking about tripleo | |
| 14:51:23 | mriedem | :) | |
| 14:51:26 | dansmith | heh | |
| 14:51:47 | dtantsur | :D | |
| 14:51:48 | mriedem | my preference would be, host is optional, so if someone thinks they know better and want to iterate hosts client side, fine | |
| 14:51:59 | mriedem | otherwise we do it | |
| 14:52:10 | dansmith | how about we make it expected, else there's an --all option, | |
| 14:52:20 | dansmith | so it's clear that it's not just idempotent and zero impact? | |
| 14:52:35 | dansmith | and to be clear, this is a dedicated command with specific behaviors, yes? | |
| 14:52:39 | mriedem | it is idempotent except for the db read hit right? | |
| 14:53:07 | dansmith | right, idempotent is the wrong word, because it is, I meant "resumable" or "zero impact if nothing to do" | |
| 14:53:15 | dansmith | like, don't just run this on every puppet run | |
| 14:53:50 | mriedem | how is tripleo going to determine that it's done all of the migrations? | |
| 14:53:56 | mriedem | and can stop | |
| 14:54:00 | dansmith | tripleo won't, | |
| 14:54:03 | dtantsur | so, are you going to call into ironic in this command? | |
| 14:54:09 | mriedem | dtantsur: no | |
| 14:54:19 | dtantsur | sorry, I don't quite understand how you're going to know the resource class of the node | |
| 14:54:22 | mriedem | dtantsur: this is just duplicating what we do in nova-compute startup | |
| 14:54:32 | dansmith | it'll be a specific skip-level upgrades script | |
| 14:54:37 | dtantsur | well, in nova compute we're using data from ironic | |
| 14:54:39 | dansmith | dtantsur: yeah, we'll have to take a RC as well | |
| 14:54:47 | dansmith | it's going to be super offline | |
| 14:54:55 | dtantsur | I know, hence my concern :) | |
| 14:54:59 | dansmith | yup | |
| 14:55:02 | dtantsur | one mistake - and the cloud is in a very strange state | |
| 14:55:29 | dansmith | but that's the risk of skip level upgrades, | |
| 14:55:37 | dansmith | you're taking the complexity of skipping into your own hands | |
| 14:55:47 | dansmith | it's a tradeoff of complexity and downtime for infrequent upgrades | |
| 14:55:56 | dtantsur | okie, but how is the command invokation going to look? | |
| 14:55:59 | dansmith | that's been a core part of my messaging on the topic at least :) | |
| 14:56:25 | dansmith | nova-manage db migrate-ironic-things --resource_class 'undercloud' --do-all-make-my-db-hurt | |
| 14:56:25 | dtantsur | $ nova-migrate-to-pike <host> --node <uuid>=<class> --node <uuid2>=<class2> | |
| 14:56:28 | dtantsur | ? | |
| 14:56:41 | dtantsur | dansmith: wait, tripleo uses one resource class, but that does not mean everyone does | |
| 14:56:41 | jaypipes | dansmith: would you be able to attend this session in Denver? https://etherpad.openstack.org/p/queens-PTG-skip-level-upgrades | |
| 14:56:44 | dansmith | or if you have different ones, then: | |
| 14:56:52 | dansmith | nova-manage db migrate-ironic-things --resource_class 'undercloud' --host foo1 | |
| 14:57:05 | dansmith | dtantsur: sure, that means you have to do your hosts one at a time with a rc then | |
| 14:57:25 | dtantsur | dansmith: host == ironic node, not nova-compute process, right? | |
| 14:57:29 | dansmith | jaypipes: um yes? been working on this with lyarwood :) | |
| 14:57:31 | dtantsur | (sorry, I always confuse the terms) | |
| 14:57:39 | dansmith | dtantsur: ah, well, that's a fair point | |
| 14:57:54 | dansmith | dtantsur: I was meaning compute, but yeah --node is probably the better option | |
| 14:58:44 | rabel | mriedem: i answered to your questions on https://review.openstack.org/#/c/494169/ . could you have a look at it again? | |
| 14:59:48 | vdrok | dtantsur: yeah everything except friday works for me | |
| 14:59:53 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Mark LXC as missing for swap volume support https://review.openstack.org/482216 | |
| 15:01:19 | dansmith | dtantsur: mriedem: anyway, let me cook something up along these lines and ya'll can rip on it | |
| 15:01:29 | dtantsur | yes please :) | |
| 15:01:36 | mriedem | we do have this compute_nodes.mapped column now | |
| 15:01:43 | mriedem | wonder if we could use that for dumb paging | |
| 15:02:09 | dansmith | mriedem: not really a good idea, I don't think | |
| 15:02:25 | dansmith | mriedem: we'd have to bump that for all the non-ironic computes too if we did | |
| 15:02:40 | mriedem | true | |
| 15:03:13 | mriedem | this is probably going to have to use all sqla orm code, | |
| 15:03:13 | dansmith | and | |
| 15:03:23 | mriedem | since we don't have some query methods you'd need on the objects | |
| 15:03:34 | mriedem | unless you just did ComputeNodeList.get_all and filter in python | |
| 15:03:34 | dansmith | maybe yeah | |
| 15:03:51 | dansmith | just some non-remotable object queries to make it clean is fine I think | |
| 15:03:55 | dansmith | no rpc impact there | |
| 15:04:40 | mriedem | taking a uuid would be nice, but you don't get the uuid out of the rest api until you're at pike | |
| 15:04:47 | mriedem | so that's out of the question probably | |
| 15:05:24 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Add attachment_complete call to volume/cinder.py https://review.openstack.org/493323 | |
| 15:05:25 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Tweak connection_info translation for the new Cinder attach/detach API https://review.openstack.org/493324 | |
| 15:05:26 | openstackgerrit | Ildiko Vancsa proposed openstack/nova master: Implement new attach Cinder flow https://review.openstack.org/330285 | |
| 15:11:22 | melwitt | abhi89: the cell with uuid of all zeros is "cell0" which contains only instances that failed to be scheduled. it's not associated with any compute host. see doc https://docs.openstack.org/nova/latest/user/cellsv2_layout.html | |
| 15:29:35 | efried | avolkov yt? | |
| 15:34:14 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add some inline code docs tracing the cold migrate flow https://review.openstack.org/496861 | |
| 15:37:20 | avolkov | efried: hi | |
| 15:37:39 | efried | avolkov Was just about to respond to your ML post about HA/anti-affinity of SR-IOV VFs... | |
| 15:38:59 | efried | avolkov I think we can do a bit better even within the bounds of the existing placement API (assuming we get nested RPs). | |
| 15:39:14 | efried | ...by chunking the PFs into HA groups. | |
| 15:39:54 | efried | So I don't think you need the ports labeled P1, P2, P3, P4. I think you can label them G1 and G2, spread across your switches. | |