| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-07 | |||
| 15:00:13 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send resource allocations in the port binding https://review.openstack.org/569459 | |
| 15:28:58 | johnthetubaguy | bauzas looking at the comments you made on the rebuild cell0 instance spec: https://review.openstack.org/#/c/554218 | |
| 15:29:09 | johnthetubaguy | and gibi: ^ | |
| 15:29:21 | bauzas | yup ? | |
| 15:29:22 | johnthetubaguy | you make some great points, but not sure how we move things forward | |
| 15:29:58 | johnthetubaguy | I was wondering about data migrations, after adding a new field, is that needed? probably not till we want to bump the object version I guess? | |
| 15:30:24 | bauzas | johnthetubaguy: about the persisted networks you mean ? | |
| 15:32:59 | bauzas | johnthetubaguy: the online data migration you ask is for setting the field for the existing req_spec records, right? | |
| 15:33:17 | bauzas | johnthetubaguy: that's why I thought about a lazy-loadable attribute | |
| 15:33:38 | bauzas | because there are two things | |
| 15:34:05 | bauzas | 1/ the instance is being created but then not getting a compute, then going into cell0 | |
| 15:34:18 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Change consecutive build failure limit to a weigher https://review.openstack.org/573248 | |
| 15:34:20 | bauzas | so, then, we're sure, the requested_networks is good | |
| 15:34:44 | mriedem | dansmith: ^ updated the commit message to note the other difference in that backport | |
| 15:34:54 | bauzas | 2/ the instance is now rebuilded, so there are some new possible networks | |
| 15:35:02 | bauzas | then, the field could be stale | |
| 15:35:30 | bauzas | johnthetubaguy: but for the data migration just because of a new field, we don't need it | |
| 15:35:57 | johnthetubaguy | bauzas: yeah, I just went through a similar though process, I think its fine without the migration | |
| 15:36:07 | dansmith | mriedem: oh actually I didn't notice that got faulted in | |
| 15:36:19 | dansmith | not sure that's going to work as-is | |
| 15:36:25 | mriedem | we don't have requested_networks on request spec today, and gibi is adding them in but not persisting them for the bw based scheduling stuff | |
| 15:36:37 | johnthetubaguy | bauzas: so block_device_mapping, isn't it the same deal, its just the initial one in there? | |
| 15:36:51 | mriedem | bdms aren't in reqspec eithre | |
| 15:37:09 | mriedem | however, | |
| 15:37:13 | johnthetubaguy | so I keep getting build_request and request_spec mixed up maybe | |
| 15:37:13 | mriedem | i believe bdms get stored in cell0 o nfailure | |
| 15:37:20 | mriedem | build_request is gone once the instance is in a cell | |
| 15:37:33 | gibi | johnthetubaguy, mriedem: yes, as I want to avoid the situation when a later operation reads the requested_networks from the RequestSpec taht is stale | |
| 15:37:47 | johnthetubaguy | yeah, maybe build request is where we want this | |
| 15:38:07 | mriedem | https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L1074 | |
| 15:38:18 | mriedem | bdms get put into cell0 on failure so we can detach volumes when you delete the failed server | |
| 15:38:30 | mriedem | s/detach/unreserve/ | |
| 15:38:48 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L1971 | |
| 15:38:59 | bauzas | the question is, can we discover the requested networks for existing req_spec records ? | |
| 15:39:13 | bauzas | we have the instance field, right? | |
| 15:39:16 | mriedem | requested networks are not persisted | |
| 15:39:17 | bauzas | oops | |
| 15:39:20 | bauzas | the instance record | |
| 15:39:38 | bauzas | mriedem: aren't we persisting the requested networks into the instance record ? | |
| 15:40:18 | mriedem | no | |
| 15:40:22 | mriedem | the info cache is in the instance | |
| 15:40:27 | mriedem | which is after you've allocated netowrks | |
| 15:40:28 | mriedem | o na host | |
| 15:40:49 | bauzas | if we aren't able to discover the networks, then https://review.openstack.org/#/c/554218/ needs to explain that for existing records, we should just rebuild the instance like if the user wasn't asking for networks | |
| 15:40:56 | bauzas | mriedem: gotcha | |
| 15:41:36 | mriedem | if we rebuild from cell0 ,we have to persist requested networks | |
| 15:41:40 | mriedem | otherwise you've changed the original user request | |
| 15:41:46 | mriedem | which you might as well just not support the feature | |
| 15:42:13 | mriedem | especially if i'm creating a server with 20 ports for NFV megaawesome | |
| 15:42:24 | bauzas | mriedem: sure | |
| 15:42:30 | gibi | mriedem: could we keep the BuildRequest in the db if the instance lands in cell0 and persist the networks there? | |
| 15:42:42 | bauzas | mriedem: but for existing instances, how can we know what the user asked ? | |
| 15:42:51 | bauzas | mriedem: if we haven't persisted that | |
| 15:43:09 | gibi | bauzas: port.device_id == instance.uuid | |
| 15:43:19 | bauzas | that's my question | |
| 15:43:32 | bauzas | and then a possible data migration | |
| 15:45:05 | mriedem | gibi: that fucks up a lot of logic in the api | |
| 15:45:22 | mriedem | build request needs to remain temporary until we pick a cell | |
| 15:45:29 | openstackgerrit | Curt Moore proposed openstack/nova-specs master: Add spec for downloading images via RBD https://review.openstack.org/572805 | |
| 15:45:44 | mriedem | bauzas: you'd likely have to persist the requested networks in the request spec | |
| 15:45:50 | mriedem | like everything else that comes in on the POST /servers request | |
| 15:46:05 | mriedem | except bdms and config drive and the other 50 things you can request when creating a server... | |
| 15:46:25 | mriedem | but that stuff gets stored on the instance, where as requested_networks don't | |
| 15:47:10 | mriedem | i'll also say, for the record, that i don't think given the timing that pre-emptible instances / rebuild from cell0 are priorities for rocky | |
| 15:47:20 | mriedem | it's way too big at this point | |
| 15:47:21 | mriedem | imo | |
| 15:47:56 | mriedem | dansmith: you were right, that pike backport failed http://logs.openstack.org/48/573248/2/check/nova-tox-functional/fd4d7a5/testr_results.html.gz | |
| 15:48:04 | dansmith | mriedem: yeah I'm running tests now to fi | |
| 15:48:06 | dansmith | *fix | |
| 15:48:08 | mriedem | ack | |
| 15:48:15 | johnthetubaguy | mriedem: yeah, that's fair, I mean its been in review for months, but we only just getting close on the details now | |
| 15:48:46 | bauzas | mriedem: don't get me wrong, I agree with persisting requested_networks in the ReqSpec | |
| 15:48:57 | melwitt | mriedem: +1 on pre-emptible instances, I was wondering similar yesterday when I was looking over the spec to add the PENDING state | |
| 15:49:01 | bauzas | mriedem: that's exactly the reason why we have this object | |
| 15:49:29 | melwitt | if it's too early to be starting on those? I dunno | |
| 15:49:37 | bauzas | mriedem: but my concern is not about persisting the field for the new instances that are created once we start persisting this info | |
| 15:49:47 | melwitt | mriedem: unrelated but just updated the hash for the ocata release https://review.openstack.org/571522 | |
| 15:50:10 | bauzas | mriedem: my wonders come from the situation where we also need to discover the networks for the instances that are *already* created and having a request_spec record | |
| 15:50:38 | mriedem | bauzas: so something already in cell0 that doesn't have requested networks in cell0 | |
| 15:50:41 | mriedem | in that case, you can't rebuild it | |
| 15:50:45 | bauzas | that's why I'm wondering how we can get this info, and populate the req_spec record accordingly thru a data migration or something like that | |
| 15:50:46 | johnthetubaguy | bauzas: I think we just fail the API call for old instances | |
| 15:50:48 | mriedem | 409 | |
| 15:50:52 | johnthetubaguy | yeah, +1 | |
| 15:50:59 | johnthetubaguy | just added that comment on the spec | |
| 15:51:08 | bauzas | mriedem: johnthetubaguy: I don't disagree, I just mean we need to say this in the specd | |
| 15:51:14 | johnthetubaguy | yup | |
| 15:51:19 | mriedem | easy workaround - delete the ERROR server and re-create it | |
| 15:51:19 | bauzas | mriedem: johnthetubaguy: ie. either we find it, or we fail | |
| 15:51:52 | johnthetubaguy | mriedem: +1 | |
| 15:52:23 | johnthetubaguy | given gibi's concerns, I wonder about calling it initial_requested_networks? | |
| 15:52:34 | johnthetubaguy | i.e. its not updated when you add ports later | |
| 15:52:40 | bauzas | johnthetubaguy: no | |
| 15:52:48 | gibi | johnthetubaguy: or update it whenever it changes :) | |
| 15:52:55 | bauzas | johnthetubaguy: requested_networks looks to me the exact naming | |
| 15:53:01 | bauzas | you *requested* a network | |
| 15:53:16 | gibi | bauzas: interface_attach is used to request additional networks | |
| 15:53:21 | bauzas | that doesn't imply you'll get *this* network | |
| 15:53:40 | johnthetubaguy | do we update the security_groups list that is in request_spec today? | |
| 15:53:41 | bauzas | gibi: sure, but then when rebuilding, we only need to know about what the user asked | |