| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-04 | |||
| 14:37:17 | mriedem | heh that's not really a bug | |
| 14:38:12 | lyarwood | mriedem: ah just code review? I assumed it came from a customer, just reading the scollback it looks like you are reviewing the QEMU LUKS spec right? | |
| 14:38:37 | mriedem | yes | |
| 14:38:46 | lyarwood | kk, don't worry about the bug then | |
| 14:39:01 | mriedem | already written | |
| 14:39:07 | mriedem | i meant your os-brick encryptors change was not a bug | |
| 14:39:14 | mriedem | to make nova use os-brick encryptors code | |
| 14:39:15 | mriedem | is not a bug | |
| 14:39:25 | mriedem | the NameError is definitely a bug :) | |
| 14:39:28 | openstack | Launchpad bug 1679717 in OpenStack Compute (nova) "Potential for NameError when logging volume encryptor" [Undecided,New] | |
| 14:39:28 | mriedem | https://bugs.launchpad.net/nova/+bug/1679717 | |
| 14:39:41 | lyarwood | ah | |
| 14:47:12 | mriedem | lyarwood: your spec reminds me that the cinder API reference doesn't even mention the encryption provider stuff in the volume type api docs | |
| 14:47:15 | openstackgerrit | Jeremy Stanley proposed openstack/nova master: Switch from pip_missing_reqs to pip_check_reqs https://review.openstack.org/453208 | |
| 14:51:44 | namnh | dansmith: Hi mr Smith, are you free, could I ask you a question about rolling upgrade? | |
| 14:52:22 | dansmith | namnh: just ask and someone here will answer if we know | |
| 14:53:19 | lyarwood | mriedem: yeah isn't it just stashed in the type as in extra_specs? ./me doesn't have an env to check | |
| 14:53:39 | lyarwood | type in extra_specs* | |
| 14:53:49 | namnh | dansmith: According to this link [1], we have to stop and start all Nova's serices except compute at the same time. | |
| 14:53:55 | openstackgerrit | Pushkar Umaranikar proposed openstack/nova-specs master: Stop using Neutron admin token https://review.openstack.org/439890 | |
| 14:54:05 | namnh | [1] https://docs.openstack.org/developer/nova/upgrade.html#rolling-upgrade-process | |
| 14:54:24 | namnh | dansmith: But IMO, there is another way to upgrade is that I will upgrade the services except compute sequentially by pinning version each services like this file [2], at this time we have a new code with the value of "pin version" is old release, after that I will unpin version by restart all services at the same time. If we do this way, we only have downtime when unpining version. How do you think? | |
| 14:54:35 | namnh | [2] https://github.com/openstack/nova/blob/master/nova/conf/upgrade_levels.py | |
| 14:55:07 | dansmith | namnh: as we said yesterday, that only works for compute | |
| 14:55:33 | dansmith | namnh: the other services talk directly to the database, which doesn't honor the upgrade level (those are only for rpc) | |
| 14:55:47 | dansmith | namnh: compute is not allowed to talk directly to the database, which is why it can be old, but the others cannoy | |
| 14:55:49 | dansmith | *cannot | |
| 15:00:56 | namnh | dansmith: but if I upgrade one conductor (N ->O), I will pin version is that: conductor=newton. That mean although conductor at new release, but it will continue talk with other services due to pin version. | |
| 15:01:59 | namnh | dansmith: I understand this file [1] configuration to do this. [1] https://github.com/openstack/nova/blob/master/nova/conf/upgrade_levels.py | |
| 15:02:27 | namnh | dansmith: we can pin version for each service | |
| 15:04:50 | bauzas | dansmith: mriedem: so I thought about the claims spec and I pushed my thoughts in the spec https://review.openstack.org/#/c/437424/ | |
| 15:05:02 | bauzas | dansmith: mriedem: tl;dr I think there are 2 things | |
| 15:05:33 | bauzas | 1/ rescheduling is a cellsv2 problem, not really related to my spec, so we should discuss that separately | |
| 15:05:53 | bauzas | because rescheduling (even without scheduler claims) is not possible in a cellsv2 world | |
| 15:06:41 | bauzas | 2/ I think it would be a still good behaviour if computes delete allocations directly by calling placemnt API, given it would be quicker than waiting for the scheduler during the reschedule | |
| 15:08:03 | namnh | hieulq: ping | |
| 15:08:52 | frickler | from a deployment perspective, is there any sense in providing the placement API to customers? or would one rather create an internal endpoint only? | |
| 15:10:08 | frickler | I'm assuming the latter and then it might make sense to change the default of placement.os_interface from public to internal, too | |
| 15:10:29 | pumarani_ | sdague, checking your comment on https://review.openstack.org/#/c/439890/. I understand your concern of getting neutron changes in place first. | |
| 15:10:54 | pumarani_ | sdague, we have already added RFE in neutron https://review.openstack.org/#/c/439890/ | |
| 15:11:15 | openstack | Launchpad bug 1674349 in neutron "[RFE] Introduce a new rule with service user role in Neutron policy" [Wishlist,Triaged] - Assigned to Akihiro Motoki (amotoki) | |
| 15:11:15 | pumarani_ | i mean https://bugs.launchpad.net/neutron/+bug/1674349 | |
| 15:19:35 | sdague | pumarani_: ok, that's a wishlist bug, does that mean it's commited to be accepted? | |
| 15:19:41 | sdague | also, where is implementation code for it | |
| 15:19:46 | frickler | sdague: mriedem: this has a nicer (imho) list of all the versions in Ocata UCA: http://reqorts.qa.ubuntu.com/reports/ubuntu-server/cloud-archive/ocata_versions.html | |
| 15:20:07 | sdague | frickler: ah, cool, I was looking for something like that | |
| 15:20:58 | mriedem | lyarwood: no it's in the volume encryption model in the cinder db | |
| 15:21:50 | sdague | pumarani_: I guess, it's really unclear to me what the neutron status is, and as such I'd really feel a lot better about this depending on merged neutron code. | |
| 15:22:08 | mdbooth | lyarwood: Are you wondering where the provider string comes from? | |
| 15:22:38 | mriedem | bauzas: if we can't reschedule, why do we care about how fast the allocations are deleted? | |
| 15:23:17 | mriedem | that raises the question of when the allocations are deleted, but i think that's just left unchanged, when we delete the instance we remove the allocations via RT in the comptue host | |
| 15:23:19 | mriedem | *compute | |
| 15:23:21 | bauzas | mriedem: it's a separate problem, hence me saying if reschedules are possible, then I'd prefer to delete the allocations sooner than later | |
| 15:23:23 | mdbooth | mriedem: Do you have a spec to delete bdm v1, btw? | |
| 15:23:29 | mriedem | mdbooth: i don't | |
| 15:23:36 | mriedem | bauzas: also, have we taken into account local deletes in the API? | |
| 15:23:47 | mriedem | the compute api would have to delete the allocations there if the compute is dead | |
| 15:23:47 | bauzas | mriedem: I'm just putting a big hypothesis in my spec that is, reschedules are possible | |
| 15:23:50 | mdbooth | mriedem: Why does it linger, btw? | |
| 15:24:07 | mriedem | mdbooth: no good reason probably | |
| 15:24:26 | mdbooth | It's a prime candidate for simplification. | |
| 15:24:33 | bauzas | mriedem: well, that's possibly a good point, if the compute is dead, then the allocation would still be created | |
| 15:24:50 | bauzas | mriedem: so we could possibly clean up "dead" allocations | |
| 15:25:07 | openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova master: The API defines PUT and POST as HTTP methods which need a request body. It is allowed, that this request body might be of zero length. What was missing is, that the "content-length" of the request also might be None. https://review.openstack.org/453220 | |
| 15:25:12 | bauzas | mriedem: that said, the ComputeFilter verifies whether the service is still up | |
| 15:25:39 | pumarani_ | sdague, yes. With neutron changes getting merged, we can also have some code samples on it. we are following up on neutron RFE. | |
| 15:25:50 | bauzas | mriedem: so the real problem is rather, say, a full compute MQ with bad performance | |
| 15:26:10 | bauzas | mriedem: then we would have created an allocation for something not really created | |
| 15:26:26 | bauzas | either way, I need to think more of that | |
| 15:26:34 | openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova master: API: accept None as content-length in HTTP requests https://review.openstack.org/453220 | |
| 15:26:47 | sdague | mriedem: in our "spec review sprint" today, are there particular specs you want eyes on instead of just scatter shotting? | |
| 15:29:13 | openstackgerrit | Peter Hamilton proposed openstack/nova-specs master: Add support for certificate validation https://review.openstack.org/357151 | |
| 15:31:04 | openstack | Launchpad bug 1679736 in Cinder "VolumeTypeEncryption API extension is not documented in api-ref site" [Undecided,New] | |
| 15:31:04 | mriedem | lyarwood: https://bugs.launchpad.net/cinder/+bug/1679736 | |
| 15:31:35 | mriedem | bauzas: i'm talking about after you've landed an instance on a host, so after scheduling, and the compute is down and you delete the instance | |
| 15:31:52 | mriedem | since we can't remove the allocation via RT on the compute host, where do we deallocate for that resource provider for a local delete? | |
| 15:31:59 | mriedem | hell, is that just a latent bug we already have with placement? | |
| 15:32:09 | mriedem | jaypipes: dansmith: ^? | |
| 15:32:26 | bauzas | mriedem: well, me shrugs | |
| 15:32:29 | bauzas | I guess yes | |
| 15:32:30 | mriedem | bauzas: i don't know what "then we would have created an allocation for something not really created" means | |
| 15:32:43 | mriedem | sdague: full monty | |
| 15:33:14 | bauzas | mriedem: I was thinking of the scheduler saying "okay, this one" then posting the allocations, then the boot requests goes trampled by the RPC bus | |
| 15:33:14 | mriedem | bauzas: for the comptue isn't down, but the instance is shelved-offloaded, | |
| 15:33:20 | mriedem | i guess when we shelve offload, do we deallocate? | |
| 15:33:38 | cdent | I think there's some misconceptions going on here | |
| 15:34:31 | mordred | cdent: people have the wrong idea about something? bah | |
| 15:34:45 | jaypipes | mriedem: yeah, it's a latent bug I think. we don't currently deallocate orphaned instances like we should. | |
| 15:34:55 | mriedem | lyarwood: i think we should add a microversion to cinder v3 to fix some of the volume type encryption api stuff, like remove the need for the full classname in the provider entry, just say 'luks' or 'cryptsetup' | |
| 15:35:16 | cdent | yes, known bug: a dead compute node's allocations persist | |
| 15:35:21 | mriedem | lyarwood: and as part of that microversion change in cinder v3, we enforce the actual providers using schema validation, rather than let it be free-form like it is today | |
| 15:35:58 | mriedem | cdent: jaypipes: how about shelve offload? | |
| 15:36:26 | openstack | Launchpad bug 1661312 in OpenStack Compute (nova) "Evacuation will corrupt instance allocations" [Undecided,New] | |
| 15:36:26 | cdent | there's also this: https://bugs.launchpad.net/nova/+bug/1661312 | |
| 15:36:27 | mriedem | because when we unshelve, the instance could end up on a different host | |
| 15:36:41 | jaypipes | mriedem: no idea. would need to check. not sure if the compute manager calls an RT method on shelve_offload(). | |
| 15:36:46 | mriedem | and if it's on a different host, we're then counting allocatoins twice for the same instnace | |
| 15:38:10 | cdent | since allocations are keyed by consumer id, and we set them (as in fully replace them) the situation is a bit cleaner than it might initially appear | |
| 15:38:55 | mriedem | but when we do claims in the scheduler, | |
| 15:39:04 | mriedem | aren't we aggregating the total allocations for a given resource provider? | |