| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-24 | |||
| 18:24:04 | superdan | heh, okay | |
| 18:24:40 | mriedem | got my landscaping buddy on the way over to triage the mini washout | |
| 18:24:55 | mriedem | stopped by ace, the helpful hardware place, on the way home from lunch to stop up on gutter extenders | |
| 18:24:57 | mriedem | booya! | |
| 18:25:06 | mriedem | *stock up | |
| 18:25:12 | superdan | haha | |
| 18:27:14 | mriedem | just trying to keep this dump afloat | |
| 18:29:30 | sdague | are you collecting animals 2 by 2 yet? | |
| 18:29:37 | mriedem | close | |
| 18:29:51 | mriedem | i'll send you a picture | |
| 18:30:14 | melwitt | ace is the place with the helpful hardware folks | |
| 18:30:32 | mriedem | right | |
| 18:30:34 | mriedem | exactly | |
| 18:30:44 | mriedem | the guy that helped me, his name tag said "Lon" and he was missing some teeth | |
| 18:30:51 | mriedem | but god damn did he know a thing or two about downspouts | |
| 18:31:26 | melwitt | perfect | |
| 18:31:32 | mriedem | and the legality of pumping water into the city line | |
| 18:31:39 | mriedem | wink wink thanks lon | |
| 18:33:15 | sdague | heh | |
| 18:37:44 | openstackgerrit | Anusha Unnam proposed openstack/nova master: Add keypairs field to InstancePayload https://review.openstack.org/419730 | |
| 18:37:54 | openstack | Launchpad bug 1675791 in OpenStack Compute (nova) "Instance created by demo user(non-admin), shelved by admin and unshelved by demo user --> ends up in error state " [Undecided,Invalid] | |
| 18:37:54 | mriedem | nkorabli: in https://bugs.launchpad.net/nova/+bug/1675791 the error in the conductor log shows that the image used to unshelve the instance can't be found | |
| 18:38:01 | mriedem | so was the image deleted? | |
| 18:38:41 | nkorabli | mriedem: it's just got an error state | |
| 18:38:43 | mriedem | nkorabli: you're hitting htis https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L665 | |
| 18:39:16 | nkorabli | so is it a bug? | |
| 18:39:20 | mriedem | no | |
| 18:39:28 | mriedem | nkorabli: not if you deleted the snapshot image | |
| 18:39:53 | mriedem | when you shelve an instance, the compute API creates a snapshot of the instance: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L3355 | |
| 18:40:14 | nkorabli | look into the second screenshot I attached, you can see all the command I run and status of the instance | |
| 18:40:17 | mriedem | that gets stored in the instance metadata when shelving on the compute https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L4326 | |
| 18:40:39 | nkorabli | I don't think I deleted any snapshot | |
| 18:40:55 | mriedem | when you unshelve, conductor looks up the snapshot image from that metadata https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L656 | |
| 18:41:03 | mriedem | and would use that image to recreate the instance on another compute | |
| 18:41:15 | mriedem | you're hitting a problem where the snapshot image ID can't be found | |
| 18:41:18 | mriedem | for the shelved instance | |
| 18:41:30 | openstackgerrit | Anusha Unnam proposed openstack/nova master: Add keypairs field to InstancePayload https://review.openstack.org/419730 | |
| 18:41:55 | mriedem | nkorabli: do you have access to the database? | |
| 18:42:15 | mriedem | i would check the glance database (or api i guess) for the snapshot image with that id in the error | |
| 18:42:22 | mriedem | the 52d95c... id | |
| 18:42:24 | nkorabli | I can check, I tested that it all with devstack | |
| 18:42:53 | mriedem | did you unshelve before the instance was completely marked as 'shelved_offloaded'? | |
| 18:43:01 | mriedem | these are async operations | |
| 18:43:06 | nkorabli | nope | |
| 18:43:40 | mriedem | your command screenshot doesn't show the unshelve command you ran | |
| 18:44:01 | nkorabli | take a look at the second screenshot I attached, I do a check after every action - to see what is the state of instance | |
| 18:44:06 | mriedem | oh wait a minute, | |
| 18:44:09 | nkorabli | hmmm | |
| 18:44:10 | mriedem | i bet i know the issue, | |
| 18:44:13 | mriedem | you shelved as one user, | |
| 18:44:23 | mriedem | so the snapshot image only has access from that user, | |
| 18:44:35 | mriedem | so when you go to unshelve, the unshelve user context doesn't have access to the private snapshot image | |
| 18:44:38 | mriedem | and you get a 404 | |
| 18:45:00 | nkorabli | Yeah, I described this in the bug description | |
| 18:45:22 | nkorabli | I mean, that snapshot is stored under admin | |
| 18:45:29 | mriedem | yeah sorry i missed that | |
| 18:45:34 | mriedem | just connected those dots :) | |
| 18:45:59 | nkorabli | no problems :) it's always good to ask questions and double check :) | |
| 18:46:10 | nkorabli | So, I suppose it's a bug | |
| 18:46:16 | mriedem | i believe the admin user can give membership access to the demo user so they can use the image | |
| 18:46:43 | mriedem | it may be a bug in nova that we aren't using the instance owner for the image | |
| 18:47:39 | nkorabli | I just want to find the right workflow - like if it's shelved by admin, than demo user shoudn't be able to unshelve it? or not? | |
| 18:49:25 | mriedem | i'd need to see what we do for snapshot, since it's a similar case, | |
| 18:49:38 | mriedem | if the admin snapshots a demo user's instance, can the demo user access that snapshot image | |
| 18:49:57 | mriedem | it's slightly different though because the demo user doesn't need to really care about that snapshot image, they may never need it, | |
| 18:50:01 | mriedem | but for unshelve, the instance is tied to it | |
| 18:51:09 | nkorabli | yeaaah :) it's confusing a bit | |
| 18:54:45 | jroll | mriedem: I've got $20 on that behavior coming from some forensics use case | |
| 18:55:02 | mriedem | jroll: like, murder? | |
| 18:55:09 | jroll | maybe! | |
| 18:55:14 | jroll | all sorts of crime, really | |
| 19:02:11 | mriedem | i've got to say i'm not following the CIS angle to this | |
| 19:02:25 | mriedem | is that some kind of michigander joke? | |
| 19:03:56 | nkorabli | mriedem: I also realized, that I have not attached the whole list of commands I ran. Updated it | |
| 19:07:42 | openstackgerrit | Sujitha proposed openstack/nova master: [WIP] Add BDM to InstancePayload https://review.openstack.org/448779 | |
| 19:10:36 | openstackgerrit | Anusha Unnam proposed openstack/nova master: Add description to policies in services.py https://review.openstack.org/449782 | |
| 19:11:12 | openstackgerrit | Sujitha proposed openstack/nova master: [WIP] Add BDM to InstancePayload https://review.openstack.org/448779 | |
| 19:14:09 | mriedem | it looks like when we create a snapshot image, we set is_public=False, which for glance v2 is visibility=private | |
| 19:14:18 | mriedem | looking at the image api docs, the default visibility is shared | |
| 19:14:35 | mriedem | which means we could add the instance owner to the snapshot as a member | |
| 19:15:19 | mriedem | i start to get totally lost when talking about image membership, and the image.owner value | |
| 19:15:31 | mriedem | because i believe it's all configurable in glance to behave differently | |
| 19:15:39 | mriedem | mordred: ^ i'm sure you've experienced this? | |
| 19:16:08 | mriedem | and the image member has to 'accept' membership before they can see the image | |
| 19:17:18 | nkorabli | but maybe demo user just shouldn't have permission to unshelve it once admin shelved it? I guess it make sense | |
| 19:17:38 | mriedem | the default policy on shelve and unshelve is admin or owner | |
| 19:17:50 | mriedem | so if the admin shelves my instance, i as the owner should still be able to unshelve it | |
| 19:18:03 | nkorabli | when demo user is an operator, for example and admin is aware of the resources and stuff.. | |
| 19:18:33 | mriedem | then idk, i guess restrict the policy on unshelve to be admin-only | |
| 19:18:43 | mriedem | but then why shelve at all? | |
| 19:20:03 | nkorabli | well, shelving is a mystery thing for me. As I would say it's kind of a rudiment.. I maybe wrong here :) | |
| 19:22:38 | openstackgerrit | Eric Fried proposed openstack/nova master: PowerVM Driver: config drive https://review.openstack.org/409404 | |
| 19:22:40 | nkorabli | but basically, it is maybe useful when you need to save resources.. Like - this VM won't be used for next week/month, but then you will need it, so you shelve and unshelve it... But I am not sure about it either. | |
| 19:23:58 | mriedem | well that's the idea yeah | |
| 19:24:07 | mriedem | why the admin would do it though, i'm not sure | |
| 19:25:24 | mordred | mriedem: looking | |
| 19:26:08 | mordred | mriedem: I have not played with shared images yet, no - although I'm starting to feel like maybe I shoud ... | |
| 19:26:19 | openstack | Launchpad bug 1675791 in OpenStack Compute (nova) "Instance created by demo user(non-admin), shelved by admin and unshelved by demo user --> ends up in error state " [Undecided,Invalid] | |
| 19:26:19 | mriedem | nkorabli: see if this makes sense https://bugs.launchpad.net/nova/+bug/1675791/comments/10 | |
| 19:26:23 | mriedem | but i think image membership is what we need | |