| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-11-02 | |||
| 16:14:07 | bauwser | mdbooth: ask leakypipes or finucannot | |
| 16:14:16 | bauwser | they'd be glad to help | |
| 16:14:35 | mdbooth | bauwser: I was trying to work out finucannot 's friday nick :) | |
| 16:14:42 | finucannot | mdbooth: done-zo | |
| 16:14:49 | mdbooth | finucannot: Cheers | |
| 16:52:45 | sean-k-mooney | mnaser: with the multi backend support in glance it might be ok for nova to add another location but i would never want nova to delete it form a speficic cluster | |
| 16:53:11 | mnaser | sean-k-mooney: yeah but thinking out loud i don't think its appropriate in case nova wants to do cache eviction | |
| 16:53:15 | sean-k-mooney | mnaser: that said that opens up a lot of corener cases in edge deployment or when you have restiction on where data can be stored | |
| 16:53:37 | mnaser | i've been hacking on it and it doesn't seem as crazy of an idea | |
| 16:54:14 | sean-k-mooney | mnaser: do you have ases were image x is not allowed to leave datacenter y | |
| 16:54:26 | sean-k-mooney | that is where this gets messy | |
| 16:54:38 | mnaser | sean-k-mooney: nope, not in this scenario at least | |
| 16:54:50 | mnaser | it's a deployment that spans 2 physical datacenters | |
| 16:54:59 | sean-k-mooney | nova does not know that so we cant assume nova can relocate teh image to anotehr cluster in general | |
| 16:55:34 | sean-k-mooney | mnaser: sure in your case it praobly fine, im just highlightin one fo the resonce nova would not do this by default | |
| 16:55:52 | mnaser | yep i agree | |
| 16:55:53 | sean-k-mooney | mnaser: haveing a config option on the other hand maybe | |
| 16:58:08 | sean-k-mooney | mnaser: if you get this working and it usefull to you then im sure people would at least consider it but likely the responce would be have a higher level service prechase the image for you out of band of nova | |
| 16:58:45 | mnaser | sean-k-mooney: right but in a public cloud environment.. i cant really do that | |
| 16:58:55 | mnaser | unless i setup replication which replicates *all* images across regions | |
| 16:59:04 | mnaser | sorry, not across region, but across ceph clusters | |
| 16:59:18 | mnaser | i.e. this is the case of a single glance, single nova deployment, multiple ceph clusters | |
| 16:59:40 | sean-k-mooney | am that depends on if you confie you teantes to specific clusers | |
| 17:00:03 | sean-k-mooney | in which case you only need to replicate the images that each of the teannts the use tha cluster need | |
| 17:00:47 | mnaser | right, but this is a public cloud so the answer is "everyone" .. as it presents itself as 2 independent availability zones | |
| 17:01:17 | mnaser | i mean i can see value in this type of feature for those who want to avoid downloading stuff non stop | |
| 17:01:30 | sean-k-mooney | this is specifcly for the ceph image backend by the way? | |
| 17:01:48 | mnaser | yes.. well.. normal backends cache images locally (even ceph) | |
| 17:01:54 | mnaser | but the problem is well.. it will be a local cache | |
| 17:01:58 | sean-k-mooney | mnaser: you basicaly want to tack advantage of cephs ablity to quickly clone volumes internally right | |
| 17:02:08 | mnaser | yes, and avoid the download by every compute node | |
| 17:02:21 | sean-k-mooney | ya thats what i assumed | |
| 17:02:33 | mnaser | otherwise the local storage would have been ok but yeah | |
| 17:03:03 | sean-k-mooney | mnaser: i might be worth seeing if you could do it in os-brick and have it stage the image in the ceph cluster on the first use of that image in that cluster | |
| 17:03:16 | mnaser | you know now what i think about it | |
| 17:03:19 | mnaser | i realize that like | |
| 17:03:30 | mnaser | this is more of a cinder problem because we no longer offer vms without bfv | |
| 17:03:50 | sean-k-mooney | mnaser: ya kindof | |
| 17:03:54 | mnaser | before we'd include ephemeral on storage so it was a nova issue | |
| 17:04:20 | mnaser | and i think this is kinda solved in https://docs.openstack.org/cinder/latest/admin/blockstorage-image-volume-cache.html | |
| 17:05:31 | sean-k-mooney | mnaser ya in the boot from volume case that should help alot | |
| 17:06:09 | sean-k-mooney | mnaser: that said in the ceph case i hope they are using ceph to do the clone internllay wich i assume they are | |
| 17:06:27 | mnaser | i think new cinder volumes from image do use ceph clone | |
| 17:06:42 | mnaser | so it would download from glance over http (in the cases of regions without local glance) | |
| 17:06:58 | mnaser | where the ceph cluster isnt directly accessible | |
| 17:07:10 | sean-k-mooney | provided glance was not backed also by ceph right | |
| 17:07:25 | mnaser | glance is backed by ceph, but it can do cow images in the same datacenter its hosted | |
| 17:07:30 | sean-k-mooney | if you glance is on ceph i think it can just to a direct clone pasing the ceph uri | |
| 17:07:51 | mnaser | yeah but that ceph cluster won't be accessible as it'd be in another datacenter | |
| 17:07:57 | mnaser | so it will fallback to just downloading over http | |
| 17:08:30 | sean-k-mooney | mnaser: right and you want to update glance with the second loaction once its cache by cinder | |
| 17:08:45 | mnaser | i mean, i dont have to, cinder will just know that it has it cached locally anyways | |
| 17:09:07 | mnaser | so bfv, create volume, cinder checks its cache (on local ceph), sees it exists, cow and boot vm | |
| 17:10:03 | sean-k-mooney | right but if you are not doing bfv and instead used the ceph backend for nova then you woudl want the other behavior | |
| 17:10:38 | sean-k-mooney | and in that config the caching and glance update would be nova problem | |
| 17:10:58 | sean-k-mooney | but ya if your using bfv then it sound like it should be good | |
| 17:21:33 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Fail to live migration if instance has a NUMA topology https://review.openstack.org/611088 | |
| 17:31:36 | openstackgerrit | Merged openstack/nova master: Fix libvirt volume tests passing invalid disk_info https://review.openstack.org/529328 | |
| 17:43:02 | openstackgerrit | Jim Rollenhagen proposed openstack/nova-specs master: Use conductor groups to partition nova-compute services for Ironic https://review.openstack.org/609709 | |
| 17:45:20 | leakypipes | fried_rolls, SteelyDan, melwitt: if you have a moment: https://review.openstack.org/#/c/614351/ | |
| 18:04:47 | openstackgerrit | John Garbutt proposed openstack/nova master: WIP: Integrating with unified limits https://review.openstack.org/615180 | |
| 18:15:56 | openstackgerrit | Merged openstack/nova master: Pass disk_info dict to libvirt_info https://review.openstack.org/529329 | |
| 18:41:15 | mriedem | SteelyDan: besides the RequestSpec.is_bfv resource tracking/filtering fix in rocky, can you think of any other major volume-related features/bug fixes we had? | |
| 18:41:22 | mriedem | in rocky specifically | |
| 18:42:07 | SteelyDan | I'd have to look.. I suck at remembering time intervals like that | |
| 18:42:18 | SteelyDan | but nothing jumps out at me as recent, no | |
| 19:19:00 | fried_rice | leakypipes: https://review.openstack.org/#/c/614351/ back atcha. I'm not a full -1 here, but would like the alternative considered. Also weirdness noted inline - I'm sure I'm just blind. | |
| 19:37:10 | leakypipes | fried_rice: that's a good idea. I'll adapt. | |
| 19:37:40 | fried_rice | leakypipes: cool beans. | |
| 19:38:16 | fried_rice | sean-k-mooney, cfriesen, mriedem: I didn't dig too deep, but the tempest-full-py3 placement log on that trust-the-cache change is 154K, vs random other change where it's 305K. | |
| 19:38:47 | fried_rice | leakypipes: --^ (see also ML) | |
| 19:38:49 | mriedem | meaning we just hit the placement API less | |
| 19:38:56 | fried_rice | exactly, which was the point. | |
| 19:39:10 | mriedem | funny how removing a cache means we hit the API *less* | |
| 19:39:16 | fried_rice | No, we didn't remove the cache | |
| 19:39:22 | fried_rice | we removed *refreshing* the cache | |
| 19:39:25 | fried_rice | oh | |
| 19:39:54 | fried_rice | Actually the part that we removed by default was just the updating of the compute rp every minute. | |
| 19:39:55 | mriedem | those sound like married cousins | |
| 19:40:18 | fried_rice | to get the full effect of this thing, I would need to set that config var. | |
| 19:40:24 | fried_rice | does that happen in the devstack project? | |
| 19:40:33 | fried_rice | cause yeah, I totally want to do that. | |
| 19:40:45 | mriedem | it doesn't | |
| 19:40:52 | mriedem | you could post-config that in the nova-next job or something if you wanted | |
| 19:40:55 | mriedem | to see a run with it | |
| 19:41:06 | mriedem | [[post-config|$NOVA_CPU_CONF]] something like that | |
| 19:41:10 | mriedem | there are examples | |
| 19:41:26 | mriedem | you want to blast it up to 1000000 or something? | |
| 19:41:31 | fried_rice | mriedem: No, zero | |
| 19:41:39 | mriedem | i thought the min was 1? | |
| 19:41:41 | mriedem | or are you changing that? | |
| 19:41:42 | fried_rice | the patch lets you do that now. | |
| 19:41:45 | mriedem | ok | |
| 19:41:58 | fried_rice | zero => float('inf') kind of thing | |
| 19:42:01 | mriedem | then you could just put a patch on top that makes the nova-next job disable the refresh | |
| 19:42:30 | fried_rice | Okay, I have the bat and the blindfold. Can you spin me around three times and point me in the right direction? | |
| 19:42:51 | fried_rice | (I don't know where to find that config for the nova-next job) | |
| 19:43:15 | mriedem | it would go in here https://github.com/openstack/nova/blob/master/playbooks/legacy/nova-next/run.yaml#L36 | |
| 19:43:48 | fried_rice | ack, thx. I'm going to make a couple of edits per review comments from sean-k-mooney and then I'll sling that on top. | |
| 19:43:50 | mriedem | but in a [[post-config|$NOVA_CPU_CONF]] section | |