| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-13 | |||
| 13:01:37 | sfinucan | johnthetubaguy: Sure. Gimme 10 | |
| 13:02:12 | johnthetubaguy | sfinucan: sweet, I have got all confused after re-reading them, particular around the good points cdent raised in there | |
| 13:06:33 | sfinucan | johnthetubaguy: Dumb question, but is there another name of the service that boots a VM other than 'nova-compute'? | |
| 13:07:13 | johnthetubaguy | sfinucan: hmm, duno really, n-cpu? | |
| 13:07:19 | sfinucan | i.e. we'd say "the scheduler" rather than 'nova-scheduler' | |
| 13:07:30 | johnthetubaguy | "the compute node" gets used a bit | |
| 13:07:37 | johnthetubaguy | or host | |
| 13:07:51 | johnthetubaguy | or just "in compute" | |
| 13:08:03 | johnthetubaguy | not sure thats a good thing though | |
| 13:09:01 | sfinucan | maybe I should just use nova-compute and nova-scheduler consistently instead | |
| 13:14:17 | johnthetubaguy | sfinucan: I guess thats always clear | |
| 13:17:12 | bauzas | sfinucan: if you talk about a running service having a RPC MQ, it's nova-compute | |
| 13:17:25 | bauzas | sfinucan: if you talk about an hypervisor, it's a node | |
| 13:37:04 | peter-hamilton | jaypipes: if you have any questions on cert validation just ping me | |
| 13:37:28 | jaypipes | peter-hamilton: will do. was on PTO yesterday. catching up today. | |
| 13:38:55 | mriedem | johnthetubaguy: as you probably saw i marked up your remove scope check patch, let me know if you want to talk about it | |
| 13:39:32 | johnthetubaguy | mriedem: so I didn't see that, looking now | |
| 13:39:41 | mriedem | you might want to sit down | |
| 13:40:02 | johnthetubaguy | funny you say that, I just adjusted my desk to sit down mode | |
| 13:45:07 | artom | For tagged attach, I need to invalidate the metadata API's cache - however, as far as I can tell, its cache is just a dict local to that process. | |
| 13:45:42 | artom | I'm trying to thinking of a good way to solve that, without suddenly requiring all deployment to run memcache.' | |
| 13:46:07 | bauzas | jaypipes: in case you have some brain time free, I'd love to get your insights on https://review.openstack.org/#/c/450122/15 | |
| 13:46:53 | bauzas | jaypipes: my biggest concern is how a virt driver can return a single dict for a common RC if it has two distinct devices behind it | |
| 13:47:04 | bauzas | unless it's not really a problem and then meh. | |
| 13:48:04 | mriedem | artom: i wonder if we could set a flag in the db to signal when the cache should be dumped | |
| 13:48:30 | mriedem | and only look it up when requesting a version new enough for the tagged attach stuff, | |
| 13:48:37 | mriedem | although i think with config drive that would be every time | |
| 13:48:38 | artom | mriedem, it's not in the DB anywhere... | |
| 13:48:47 | mriedem | artom: i know | |
| 13:48:52 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: PCI NUMA Policies https://review.openstack.org/361140 | |
| 13:48:59 | sfinucan | johnthetubaguy: & | |
| 13:49:03 | mriedem | artom: is the cache per instance? | |
| 13:49:11 | mriedem | i'm not terribly familiar with this | |
| 13:49:13 | artom | mriedem, I think it's per process | |
| 13:49:14 | johnthetubaguy | sfinucan: cool, queued it | |
| 13:49:14 | artom | https://github.com/openstack/nova/blob/master/nova/cache_utils.py | |
| 13:49:32 | sfinucan | Also: cdent, sean-k-mooney, snikitin_, sgordon - https://review.openstack.org/361140 | |
| 13:49:45 | sfinucan | I went with required, preferred and legacy | |
| 13:49:57 | artom | mriedem, the metadata handler calls get_client(), which, depending on conf, configures a cache region in a certain way | |
| 13:50:32 | artom | It's using dogpile.cache, in which a region is basically a frontend/client: https://dogpilecache.readthedocs.io/en/latest/usage.html | |
| 13:50:38 | sfinucan | jaypipes: You don't get to escape either :P https://review.openstack.org/361140 | |
| 13:51:07 | cdent | ha | |
| 13:51:11 | artom | And I think the backend is basically a dict: https://github.com/openstack/oslo.cache/blob/master/oslo_cache/backends/dictionary.py | |
| 13:51:50 | artom | So unless the REST API request just happened to land on the same process that eventually handles the metadata request, we need some form of IPC | |
| 13:52:44 | mriedem | artom: default cache is just in-memory per process dict? | |
| 13:52:51 | mriedem | i think it's dogpile yeah? | |
| 13:52:59 | artom | mriedem, that's what I gather, yeah | |
| 13:55:02 | mriedem | well this is a sticky wicket | |
| 13:55:19 | mriedem | we have to look up an instance when it's not cache, or the cache is expired | |
| 13:55:48 | mriedem | i was thinking we could have set a flag in instance_extra that would tell metadata api to refresh the cache, but that defeats the purpose because you have to do the db lookup to find out | |
| 13:56:26 | mriedem | default cache expiration is 15 seconds right? | |
| 13:56:48 | snikitin_ | sfinucan: wow. Thank you for update! btw 'legacy' looks good because this behavior it really a little bit strange | |
| 13:56:57 | mriedem | does the guest need to see the tagged block device immediately? | |
| 13:57:10 | artom | mriedem, ah, you're saying just wait it out? | |
| 13:57:13 | frickler | mriedem: regarding your ML question, johnthetubaguy has a patch up to clarify things: https://review.openstack.org/#/c/438887/3 | |
| 13:57:17 | mriedem | artom: that's what i'm asking | |
| 13:57:33 | mriedem | artom: i.e. this is already a latent issue right? | |
| 13:57:39 | sfinucan | snikitin_: no problem | |
| 13:57:55 | mriedem | if your guest is relying on changes via metadata api, and it's already cached on a timer, then your guest logic probably already needs to poll for changes | |
| 13:58:03 | mriedem | i don't really know the use cases though | |
| 13:58:12 | sfinucan | snikitin_: Personally I'd like to drop that policy because it makes no sense. If you care about NUMA affinity, you should have 100% NUMA-aware nodes. If you don't, you clearly don't care enough :) | |
| 13:58:17 | johnthetubaguy | frickler: mriedem: thats right, I new that was familiar, I think bauzas said he would fix up that patch. | |
| 13:58:25 | sfinucan | But this is a good half way measure to avoid breaking existing deployments | |
| 13:58:27 | artom | mriedem, I suppose it is... but... well, did we ever have a user case where the metadata changed during an instance's lifetime? | |
| 13:59:00 | mriedem | artom: i'm sure there are all sorts of things that change | |
| 13:59:04 | snikitin_ | sfinucan: yea, me too. But this damn backward compatibility... | |
| 13:59:14 | mriedem | but let me check | |
| 13:59:38 | bauzas | johnthetubaguy: it just went out of my radar, because I'm not the Gerrit owner :( | |
| 13:59:51 | johnthetubaguy | bauzas: yeah, I have done that a few times also | |
| 13:59:54 | sfinucan | snikitin_: I know, right? How dare our customers ask for backwards compatibility :P Let's hope that naming clears up everything | |
| 13:59:54 | mriedem | artom: security groups could change, password could change | |
| 14:00:10 | mriedem | artom: network metadata could change couldn't it depending on attaching/detaching ports? | |
| 14:00:25 | mriedem | not to mention dynamic vendor data | |
| 14:00:27 | bauzas | mriedem: if your ML thread is related to my az spec, nevermind it, we could just wait for Queens | |
| 14:00:39 | mriedem | bauzas: it was the impetus yes | |
| 14:00:49 | mriedem | i never knew about default_scheduler_zone | |
| 14:00:54 | artom | mriedem, that's probably correct... | |
| 14:01:05 | bauzas | mriedem: default_schedul/e/_zone but yeah | |
| 14:01:15 | mriedem | artom: so if you accept as a limitation that you're not getting the latest changes immediately when using metadata api, then this is probably ok | |
| 14:01:16 | bauzas | mriedem: hope my reply made things clearer for you | |
| 14:01:32 | mriedem | bauzas: it made me feel like there was a purpose | |
| 14:01:43 | mriedem | bauzas: however, the help text on those options is still murky | |
| 14:01:43 | artom | mriedem, I think that's acceptable in the short term, at least | |
| 14:01:46 | mriedem | and should probably be updated | |
| 14:01:53 | bauzas | mriedem: yup, hence johnthetubaguy and frickler point | |
| 14:01:55 | mriedem | bauzas: plus, | |
| 14:01:58 | snikitin_ | sfinucan: :D | |
| 14:01:58 | artom | mriedem, and if someone complains we can always do a "real" cache later | |
| 14:02:01 | mriedem | this big fat scary warning https://docs.openstack.org/developer/nova/aggregates.html#availability-zones-azs | |
| 14:02:04 | mriedem | should probably be in the api-ref | |
| 14:02:07 | mriedem | where api users would see it | |
| 14:02:10 | mriedem | rather than way down in devref | |
| 14:02:11 | artom | Or they can run memcache and we can add a bit where we invalidate it | |
| 14:02:29 | bauzas | mriedem: well, when I wrote by that time, I wasn't seasoned with api-ref :) | |
| 14:02:39 | mriedem | artom: memcache doesn't help the timing problem right? just the consistency issue, | |
| 14:02:40 | bauzas | it should even be in the operators guide | |
| 14:02:51 | mriedem | i.e. depending on which metadata api worker i hit with my request, i could get different results | |
| 14:03:02 | bauzas | anyway, spec deadline day, I'm turned into review day for me | |
| 14:03:07 | mriedem | or would memcache be updated globally for all? i suppose it would | |
| 14:03:09 | mriedem | nevermind then | |