| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-04 | |||
| 18:15:24 | leakypipes | ok dokey | |
| 18:15:32 | superdan | hang on, forgot to run pep8 | |
| 18:15:39 | superdan | stand by rumble | |
| 18:16:03 | leakypipes | hehe | |
| 18:18:00 | mriedem | ERROR [nova.scheduler.client.report] Failed to allocate resources on provider 9f098d35-83b2-46e1-a2c2-b51ee1a33f66 for instance aeb45d5b-4787-4afa-9b19-430da6f33699. Error: | |
| 18:18:00 | mriedem | yes | |
| 18:18:07 | mriedem | Unable to allocate inventory: Inventory for 'DISK_GB' on resource provider '9f098d35-83b2-46e1-a2c2-b51ee1a33f66' invalid. | |
| 18:18:32 | mriedem | what do you mean you can't allocate against a provider with 0 DISK_GB inventory?! | |
| 18:18:48 | superdan | okay here comes | |
| 18:18:56 | openstackgerrit | Dan Smith proposed openstack/nova master: remove provider allocs in confirm/revert resize https://review.openstack.org/488510 | |
| 18:18:57 | openstackgerrit | Dan Smith proposed openstack/nova master: Resource tracker compatibility with Ocata and Pike https://review.openstack.org/491012 | |
| 18:18:57 | openstackgerrit | Dan Smith proposed openstack/nova master: Add resource utilities to scheduler utils https://review.openstack.org/490514 | |
| 18:19:52 | mriedem | +2 on https://review.openstack.org/#/c/490514/ | |
| 18:20:37 | leakypipes | mriedem: I'd +2 that but I added a lot to it... | |
| 18:20:57 | leakypipes | perhaps melwitt or bauzas could look at it? | |
| 18:21:11 | mriedem | bauzas is going to the mediterraneannenaanna coast | |
| 18:21:30 | superdan | bauzas hasn't been on PTO in over 48 hours, so he can't be asked to do things by french law | |
| 18:21:40 | mriedem | zing | |
| 18:21:54 | mriedem | fried_rice: there was no FFE for the service catalog endpoint thingy | |
| 18:22:13 | mriedem | fried_rice: and as far as i know there hasn't been much core reviewer attention on it | |
| 18:22:27 | mriedem | fried_rice: and you keep changing things :) | |
| 18:23:36 | mriedem | superdan: leakypipes: yeah so fun fact, with the shared storage test, we hit https://github.com/openstack/nova/blob/85cd4574b8347d032be2285a277a0abe4d4a6869/nova/scheduler/client/report.py#L1084 all the time b/c i've got the compute node's reporting 0 DISK_GB inventory, so i've at least sorted out and recreated the problem | |
| 18:23:59 | mriedem | it's just not as obvious in the tests until much later | |
| 18:24:13 | leakypipes | k | |
| 18:24:23 | openstackgerrit | Merged openstack/os-vif master: doc: Remove cruft from releasenotes conf.py https://review.openstack.org/480092 | |
| 18:34:24 | mriedem | for someone doing shared storage for their computes, would we expect them to post like DISK_GB with total=100 and reserved=100, or just not post any DISK_GB for the compute node provider? | |
| 18:34:30 | openstackgerrit | Jackie Truong proposed openstack/nova master: Add trusted_certs to instance_extra https://review.openstack.org/457711 | |
| 18:34:39 | mriedem | difference between yes i have some but i'm using it all, vs i want you to think i have no local disk | |
| 18:34:43 | mriedem | even if i do | |
| 19:12:07 | mriedem | leakypipes: just thought of something - is there anything making VCPU/MEMORY_MB indivisable between providers? for example, if i'm creating a server with x cpu and y ram, and CN1 can provide the vcpu but not the ram, and CN2 can provide the ram but not the vcpu, would placement return both compute node providers to the scheduler and the scheduler would pick one and fail? | |
| 19:12:28 | mriedem | i don't think that's possible unless vcpu/memory_mb is marked shared or something | |
| 19:12:38 | mriedem | which is how it can work to get disk from another provider and not the compute node | |
| 19:14:43 | superdan | that would be really bad | |
| 19:14:58 | superdan | but pretty sure that we're looking for single providers that provide all the things we want, | |
| 19:15:07 | superdan | or are related to one via aggregate that does | |
| 19:15:22 | fried_rice | mriedem The changes have been in response to reviews, fwiw. | |
| 19:15:32 | superdan | I forget if/how a shared provider is considered and not just another compute node in the same aggregate | |
| 19:15:37 | superdan | but I think there's a provision for that | |
| 19:15:47 | superdan | misc_shares_via_aggregate or something maybe? | |
| 19:16:14 | mriedem | right the compute node providers have to be in an aggregate relationship with an rp that has the MISC_SHARES_VIA_AGGREGATE trait | |
| 19:16:24 | fried_rice | mriedem At this pace, it won't be fully completed (because neutron & cinder integration still needed) for at least another week, even assuming my latest rev gets +2s | |
| 19:16:37 | fried_rice | I don't have a problem pushing it; your call whether it's important enough to squeeze in late. | |
| 19:16:43 | mriedem | fried_rice: is there no way that can't be staggered per service? | |
| 19:16:59 | mriedem | fried_rice: i don't think there is justification for getting that in after FF and before RC1 | |
| 19:17:21 | fried_rice | mriedem Certainly. That's basically what's happening. The outstanding change set just does glance & ironic. The other services can be added one at a time, or whatever. | |
| 19:18:00 | fried_rice | mriedem Anyway, I'll just keep plugging away at the changes, and let y'all decide whether/when to push 'em. | |
| 19:18:32 | mriedem | superdan: another thing i thought of which complicates the shared storage trampling in the heal task, but if we're doing a move and there are allocations for the instance against 3 providers, like CN1 provides ram/cpu and gets DISK_GB from a shared storage provider (which is provider #2), and then the dest compute provider provides ram/cpu/disk (local disk) | |
| 19:18:36 | mriedem | things get wonky | |
| 19:18:53 | mriedem | because during a move, there is nothing preventing the scheduler from picking a compute node that is sharing storage with the source node | |
| 19:18:59 | mriedem | so it could pike a node with locak disk | |
| 19:19:02 | mriedem | *pick | |
| 19:19:10 | superdan | well, | |
| 19:19:24 | superdan | if there is not request in the reqspec that prevents that, then I would say that's fine | |
| 19:19:44 | superdan | but you'd have to have block migration for that, which I think is automatic now,right? | |
| 19:19:47 | mriedem | right i think it's fine from a scheduler pov | |
| 19:20:13 | superdan | could be a thing with a hypervisor needing shared storage all te time for a migration though, if that's what you mean | |
| 19:20:30 | mriedem | we don't consult block vs shared migration in the scheduler to determine which compute nodes to use based on if they have a shared storage provider aggregate | |
| 19:20:46 | mriedem | i was just thinking resize/cold migrate | |
| 19:20:59 | mriedem | could move the instance from a source node using shared storage to a dest node using locak disk | |
| 19:21:01 | mriedem | *local | |
| 19:21:06 | mriedem | which i think is ok | |
| 19:21:13 | superdan | yeah' | |
| 19:21:27 | mriedem | it's just that things in the report client trying to sort this mess out is going to be...messy | |
| 19:21:54 | fried_rice | edmondsw_ Looking at https://review.openstack.org/#/c/485121 - I think the answer is yes. Stay tuned. | |
| 19:22:32 | edmondsw_ | fried_rice tx | |
| 19:22:33 | superdan | well, as leakypipes has said, we probably need to be providing more info the scheduler in those situations to account for some of this | |
| 19:22:36 | superdan | but yteah | |
| 19:33:18 | fried_rice | edmondsw_ Checkacheckacheck it out. | |
| 19:34:50 | cfriesen | superdan: mriedem: I think that if the end-user hasn't explicitly requested shared storage than doing a block migration to/from a compute with local storage is fine | |
| 19:35:12 | cfriesen | so I'm not sure the scheduler needs to care about the storage the instance is currently on | |
| 19:35:36 | cfriesen | unless the hypervisor physically can't handle that case | |
| 19:38:19 | edmondsw_ | fried_rice ty sir | |
| 19:46:04 | cfriesen | do we even have a way to say "I want to be on a hypervisor with shared storage for instance "localdisk"? | |
| 19:46:42 | cfriesen | almost seems like that would be a host-aggregate thing | |
| 19:57:07 | sean-k-mooney | cfriesen: are you assuming that if the tenant wants there instance to be backed by shared storage then they would request the instance to be backed by a cinder volume | |
| 19:59:00 | superdan | leakypipes: you're going to look at my split of your patch right? | |
| 20:11:31 | cfriesen | sean-k-mooney: either that or they would specify a flavor that the admin has set up to map to a host aggregate that has shared storage. | |
| 20:12:32 | cfriesen | I can't think of any other way to request shared-storage in a mixed shared/local cloud | |
| 20:13:01 | sean-k-mooney | cfriesen: the issue with that is its not programatically discoverable unless we used traits to model that | |
| 20:14:14 | cfriesen | sean-k-mooney: right. so we don't currently support the scenario where we happen to schedule on a hypervisor with shared storage that doesn't support block-migrating to non-shared storage. (are there such hypervisors?) | |
| 20:15:20 | sean-k-mooney | cfriesen: does that work for ceph to non ceph today? | |
| 20:15:53 | cfriesen | sean-k-mooney: I think it'd be a question of whether qemu supports it | |
| 20:16:02 | sean-k-mooney | cfriesen: you can configure nova to always use ceph on one node without the tenant requesting it but not sure if you can do a block migration to another host | |
| 20:16:34 | openstackgerrit | Dan Smith proposed openstack/nova master: Resource tracker compatibility with Ocata and Pike https://review.openstack.org/491012 | |
| 20:17:04 | cfriesen | sean-k-mooney: yeah, I don't know if it works or not | |
| 20:17:09 | sean-k-mooney | cfriesen: well even if qemu could i dont think anything would endup deleteing the ceph voluome in that case unless maybe libvirt would? | |
| 20:17:19 | openstackgerrit | Merged openstack/nova master: Fix getting instance bdms in multiple cells https://review.openstack.org/490340 | |
| 20:18:10 | cfriesen | sean-k-mooney: no clue. I'd hope it'd be covered by the normal post-live-migration cleanup on the source side. | |
| 20:18:21 | sean-k-mooney | it proably is not a good idea to mix nova with default ceph stoage and nova with default local in the same availablity zone in anycase | |
| 20:19:40 | cfriesen | certainly that'd be the simplest option....."don't do that" | |
| 20:21:41 | sean-k-mooney | well if you want to keep your openstack sysadmins sane mixing default storage configs in the same availablity zone is certenly not the way to go | |
| 20:43:36 | openstackgerrit | Eric Fried proposed openstack/nova master: nova.utils.get_ksa_adapter() https://review.openstack.org/488137 | |
| 20:47:31 | leakypipes | superdan: just back from doctor... have you pushed the split yet? | |
| 20:47:57 | superdan | leakypipes: yeah dude.. but I just realized there's still a functional failure in the middle, so fixing that up, | |
| 20:48:15 | leakypipes | superdan: gotcha. just ping me when ready. | |
| 20:48:27 | superdan | leakypipes: you could still look, it's just a minor thing | |
| 20:48:32 | leakypipes | ah, k | |
| 20:48:39 | leakypipes | looking now | |
| 20:48:41 | superdan | leakypipes: just to see if you're okay with it or I missed something major | |
| 20:48:47 | leakypipes | k | |