| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-08-11 | |||
| 22:00:17 | sean-k-mooney | artom: cool but i would really like to get that merged before the end of the week | |
| 22:00:24 | artom | No shade on the original author, I realize writing commit messae (or anything, really), in a second language is hard | |
| 22:00:49 | artom | sean-k-mooney, it's not your patch though... are you going to iterate on it? | |
| 22:01:22 | sean-k-mooney | if need but i want to get this downstream into 16.1 sooner rather then later | |
| 22:01:30 | sean-k-mooney | i have just been trying to get people to review | |
| 22:01:51 | sean-k-mooney | i tested the first verision but i need to retet the latest version | |
| 23:45:43 | artom | nova-grenade-multinode just failed twice in a row with apparently the same errors | |
| #openstack-nova - 2020-08-12 | |||
| 02:12:31 | artom | Dammit, again. Looks like it's a real thing, will debug tomorrow. | |
| 02:50:13 | melwitt | artom: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016446.html | |
| 07:34:39 | brinzhang | sean-k-mooney, dansmith, stephenfin, gmann: report bug 1891274 | |
| 07:34:39 | openstack | bug 1891274 in OpenStack Compute (nova) "100% of nova-grenade-multinode zuul task run failed" [High,New] https://launchpad.net/bugs/1891274 | |
| 07:35:11 | melwitt | I already reported that here https://launchpad.net/bugs/1891244 | |
| 07:35:11 | openstack | Launchpad bug 1891244 in OpenStack Identity (keystone) "*-grenade-multinode jobs fail with UnicodeDecodeError: 'utf-8' codec can't decode byte 0x87 in position 3: invalid start byte" [Undecided,In progress] - Assigned to melanie witt (melwitt) | |
| 07:37:46 | brinzhang | melwitt: is it the same thing? bug 1891274 said the nova-grenade-multinode task failed | |
| 07:37:46 | openstack | bug 1891274 in OpenStack Compute (nova) "100% of nova-grenade-multinode zuul task run failed" [High,New] https://launchpad.net/bugs/1891274 | |
| 07:37:49 | brinzhang | will check | |
| 07:38:50 | melwitt | yes, it should be the same thing. and nova-grenade-multinode passes with the fix I proposed to keystone https://review.opendev.org/745752 | |
| 07:42:11 | brinzhang | melwitt:ack, thanks, after this patch merged, and I will set this bug to repeat | |
| 08:13:19 | tosky | melwitt: for the record, cinder is impacted by the msgpack bump too | |
| 08:13:25 | tosky | melwitt: namely here: https://zuul.opendev.org/t/openstack/builds?job_name=cinder-grenade-mn-sub-volbak&project=openstack/cinder | |
| 08:22:34 | sean-k-mooney | this looks like a byte vs text type issue | |
| 08:23:15 | sean-k-mooney | the fix looks straitgh forward if you can find the correct place to do it | |
| 09:44:23 | sean-k-mooney | dumb question | |
| 09:44:40 | sean-k-mooney | is there any reason we cant support resize for shelved instances | |
| 09:45:15 | sean-k-mooney | it would just be a flavor update in the db right and asserting there are no conflict with the image | |
| 10:19:09 | sean-k-mooney | in the shelve offloaded case that is | |
| 13:01:55 | kklimonda | is there a... saner way to extend nova APIs than forking entire repo? We have a rather specific set of requirements for spawning new instances that can't really be done with just the oslo_policy | |
| 13:11:58 | gmann | kklimonda: you cannot extend the APIs with upstream code, forking is anyways you can do anything. if i understand correctly, you do not want oslo policy checks in API right? | |
| 13:12:44 | kklimonda | @gmann well, oslo_policy can't really handle our usecase by itself | |
| 13:13:25 | gmann | policy checks are configurable and you can allow API to everyone acess | |
| 13:42:51 | sean-k-mooney | melwitt: bnemec by the way i finally got aroudn to trying to repoduce https://bugs.launchpad.net/nova/+bug/1854992 since we are talking about adding a ping | |
| 13:42:51 | openstack | Launchpad bug 1854992 in OpenStack Compute (nova) "Frequent instances stuck in BUILD with no apparent failure" [Undecided,Incomplete] | |
| 13:43:18 | sean-k-mooney | melwitt: bnemec deleting the queue will not triggere it because oslo messaging just recreates the queue | |
| 13:43:49 | sean-k-mooney | melwitt: bnemec but unbinding the queue form the topic does break the ablity for the compute node to recive rpcs | |
| 13:44:16 | sean-k-mooney | but its still considered alive since it can still send the heartbeats back fine | |
| 13:53:53 | sean-k-mooney | melwitt: bnemec can you let me know what you think of the two options to fix this? | |
| 13:58:34 | kklimonda | @gmann right, but policy checks don't receive much context about server create request, and without that I can't really say whether it should be allowed or denied | |
| 14:00:22 | gmann | kklimonda: ohk, currently we pass only project-id, user-id and AZ - https://github.com/openstack/nova/blob/5b79ec8670619ece92ac8bb43d84d2412778a182/nova/api/openstack/compute/servers.py#L648 | |
| 14:00:57 | gmann | kklimonda: we can add more targets there to enhance the policy control | |
| 14:01:22 | gmann | or pass instance object itself for wider use case | |
| 14:03:40 | sean-k-mooney | dpending on what api it is you might not have an instance objet at the policy time but you do get one failrly quilcly. the build request or request spec might work in those casess | |
| 14:03:48 | sean-k-mooney | im thinking of create server | |
| 14:04:07 | sean-k-mooney | but for other apis we proably could pass the instance object ya | |
| 14:04:18 | gmann | sean-k-mooney: yeah, the request body of instance request we can pass in case of create server | |
| 14:05:11 | sean-k-mooney | yes ideally we would pass a copy too | |
| 14:05:12 | gmann | or best will be add attr by attr based on use case we get to know | |
| 14:05:15 | sean-k-mooney | so that you can modify anything | |
| 14:05:23 | gmann | true | |
| 14:05:29 | sean-k-mooney | *can't | |
| 14:06:09 | sean-k-mooney | body looks like its a dict | |
| 14:06:36 | sean-k-mooney | for this kind of think i generally prefer to be lazy and pass the whole thing | |
| 14:06:48 | sean-k-mooney | rahter then attr by attr | |
| 14:06:58 | sean-k-mooney | but it depned on how strong a contract we want to give | |
| 14:07:21 | sean-k-mooney | e.g. if we change a field name if we just pass the body it can break things | |
| 14:07:26 | sean-k-mooney | so there is a trade off there | |
| 14:07:35 | gmann | yeah, for some APIs that is ok but for many passing everything is just unnecessary for oslo policy | |
| 14:07:36 | sean-k-mooney | at least form an upgrades point of vew | |
| 14:08:11 | gmann | you mean if API change the request attr etc? | |
| 14:08:35 | gmann | and overridden policy were relying on that | |
| 14:08:55 | sean-k-mooney | if we renamed say body['os:scheduler_hints'] to body['os:scheduler_hint'] and the polocy was relying on that then it would break | |
| 14:09:09 | gmann | yeah | |
| 14:09:13 | sean-k-mooney | but if we add a scheduler_hints attibe then we have a layer of indirection | |
| 14:09:34 | sean-k-mooney | not that we plan to do that but you know just trying to see what would break | |
| 14:09:37 | gmann | that is good upgrade checks in such case | |
| 14:10:22 | gmann | we can get the override rules and how they are written and compare such case of attr change at API side | |
| 14:10:42 | sean-k-mooney | only for default rules | |
| 14:10:49 | sean-k-mooney | we wont know what customer are using | |
| 14:11:19 | gmann | we can know, we get the policy file and the overridden rule info | |
| 14:11:33 | sean-k-mooney | not upstream | |
| 14:12:06 | sean-k-mooney | if random customer x uses this to create a custom policy rules check any sub element of the requet body | |
| 14:12:16 | sean-k-mooney | we can test for that | |
| 14:13:35 | gmann | yeah, what other case you mean we cannot test? | |
| 14:13:38 | sean-k-mooney | what we could do is pass create_kwargs instead of body that gives us a small laywer of indriection in this case but i think im slowly talking my self into advocating for only passing objects or indivigual atributes | |
| 14:14:26 | sean-k-mooney | we cant test every poicly atibute that coudl be used form the body | |
| 14:14:48 | gmann | ohk, yeah. | |
| 14:15:49 | gmann | but as long as we construct the targets explicitly, we can do. I am in favor of 'add attr based on known use case' instead of complete object/request info | |
| 14:16:18 | gmann | we do not test current allowed attr though but something to improve | |
| 14:16:41 | gmann | project_id is only tested at major/or-all | |
| 14:19:31 | sean-k-mooney | ya if we just add things one by one to the target we can test it | |
| 14:20:08 | sean-k-mooney | well the target is currently | |
| 14:20:10 | sean-k-mooney | target = { | |
| 14:20:13 | sean-k-mooney | 'project_id': context.project_id, | |
| 14:20:15 | sean-k-mooney | 'user_id': context.user_id, | |
| 14:20:17 | sean-k-mooney | 'availability_zone': availability_zone} | |
| 14:20:24 | sean-k-mooney | i dont think we would remove any of those well ever | |
| 14:20:29 | sean-k-mooney | so it pretty safe | |
| 14:21:01 | sean-k-mooney | kklimonda: what did you need for your usecase | |
| 14:21:05 | gmann | yeah, chance of changing request body is very less. addition is there | |
| 14:21:11 | gmann | also scope control and new defaults, can solve the problem at most extend. | |
| 14:27:31 | kklimonda | sean-k-mooney: it's a billing issue - we want to have two types of VMs: pay-as-you-go and prepaid (attached to a contract). I need to limit `compute:create:*` API calls based on whether user has credits, or contract with a open slots for a given flavor type. Also, have a way to return a customized error message back to the user would be nice. | |
| 14:29:02 | sean-k-mooney | kklimonda: that is not really a policy function | |
| 14:29:58 | sean-k-mooney | kklimonda: that feels more like a use case for keystone middelware or quotas | |
| 14:30:36 | kklimonda | keystone middleware as in custom middleware for keystone? | |
| 14:31:07 | gmann | yeah, we cannot control these with policy | |
| 14:31:11 | sean-k-mooney | as in custom mideelware for the nova api | |
| 14:31:32 | sean-k-mooney | sorry not keystone middelware oslo.middleware | |
| 14:31:52 | sean-k-mooney | we have keystone middelware that uses that to handel all our auth | |
| 14:32:02 | sean-k-mooney | so its the example of api middelware that comes to mind | |
| 14:32:06 | kklimonda | can I still add custom middleware to nova? keystone have removed that feature around stein iirc | |
| 14:32:18 | sean-k-mooney | you can do it with paste | |