Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-08-22
15:07:15 yonglihe sure, so expose cpu_topology is not related to asyc cpu.
15:08:12 alex_xu sean-k-mooney: but stephenfin point out https://github.com/openstack/nova/blob/f9ba1192c4/nova/virt/libvirt/driver.py#L5219, the vcpu_topology may empty
15:08:22 alex_xu I just doesn't figure out what case we will have yet
15:08:50 sean-k-mooney sure it will be empty unless you use teh hw:cpu_* extrapecs
15:09:22 sean-k-mooney so we need to not just check the vcpu_model but allso all of the sub fields to see if they are set
15:09:38 alex_xu oh, I thought we will have a default one
15:09:52 sean-k-mooney the default is virt driver speficic
15:10:07 sean-k-mooney so unless the virt driver populated it in the instance no
15:10:11 sean-k-mooney if it does we might
15:12:01 melwitt belmoreira: hi, as of rocky, console proxies need to be deployed per cell https://docs.openstack.org/nova/stein/user/cellsv2-layout.html#consoleauth-service-and-console-proxies and then if you need to keep consoles working during a live upgrade, set [workarounds]enable_consoleauth = True to enable nova-consoleauth to be a fall back https://docs.openstack.org/releasenotes/nova/stein.html
15:12:25 efried alex_xu, sean-k-mooney, aspiers: so what's up with this request_filters thing?
15:13:41 alex_xu efried: I thought we always getting the info from extra spec, not put something back to extra spec https://review.opendev.org/#/c/664420/42/nova/scheduler/request_filter.py@206
15:13:47 alex_xu that feel strange
15:14:25 aspiers alex_xu: did you see the eavesdrop discussions I linked to?
15:14:27 sean-k-mooney alex_xu the reques filter do modify the tempory flavor
15:14:39 efried yeah, it's an unfortunate temporary measure
15:14:42 alex_xu aspiers: sorry, I didn't get a chance yet
15:15:10 efried aspiers and I discussed at the PTG the longer road to restructuring the code so we wouldn't have to do that, we could just update the request spec (I think)
15:15:24 efried but afaik aspiers hasn't rewritten all of that for us yet :P
15:15:41 aspiers alex_xu: hw:mem_encryption=True is the extra spec visible to users/operators, but it needs to be translated into resources:MEM_ENCRYPTION_CONTEXT=1 behind the scenes just within the RequestSpec
15:16:07 aspiers users/ops shouldn't have to know about the resource class
15:16:13 efried ++ ^ this
15:16:22 alex_xu yea, I see
15:16:31 efried alex_xu: There's precedent in the other request filters, which I think aspiers pointed to.
15:16:36 sean-k-mooney we do the same thing with request filters that add required/forbiding tratis
15:16:43 alex_xu probably we can have request group obj in request spec, then it will be more beatiful
15:16:56 belmoreira melwitt thanks. I'm reading more about this change
15:16:57 efried yes alex_xu, that's the idea, but it's not simple to do.
15:17:11 efried aspiers and I talked about it for like 2h on Saturday at the PTG.
15:17:30 alex_xu ok, got it, seem that is already thing we agree on can do now
15:17:43 sean-k-mooney yep we were not planning to use the request spec in train right
15:17:46 aspiers I'd love the chance to improve things in the future, but it would be great if we don't have to block SEV for this
15:17:47 sean-k-mooney that was for U
15:17:59 melwitt belmoreira: k. let me know if you have questions. also, mnaser went through this same novnc "challenge" when he upgraded, so maybe he can help answer questions too
15:18:05 efried sean-k-mooney: we were tentatively thinking it could happen in T, but at this point, yeah, we should probably punt.
15:18:20 alex_xu I won't block on that
15:18:22 mnaser o/
15:18:39 melwitt belmoreira: from an operator perspective, I mean
15:18:47 aspiers efried: having said all that, alex_xu's suggestion to put the translation into scheduler/utils.py sounded interesting
15:19:01 aspiers but I don't know enough to understand how that relates to this discussion right now
15:19:17 sean-k-mooney efried: we have a few new request filters this cycle. i think it makes sense to prot them all in U
15:19:22 efried melwitt: Thank you for the fix around websockify. And stephenfin thanks for digging into the symbol alias business. I guess we're stuck supporting the dual-location business :(
15:19:35 efried aspiers: where was that part?
15:19:41 sean-k-mooney aspiers: we were moving away form that intentionally
15:19:46 aspiers efried: let me find it
15:19:53 belmoreira melwitt: workarounds/enable_consoleauth = True is only for the leftovers, right? becasue in stein the DB will be always used to store the token
15:20:09 sean-k-mooney e.g. we do some traslation in utils but we disucess not doing that and using request filster intead
15:20:27 melwitt mnaser: belmoreira just upgraded to stein and novnc started failing to work, console proxies are not yet deployed per cell. I'm thinking he needs to set [workarounds]enable_consoleauth = True during the live upgrade and then deploy proxies per cell and then after upgrade be able to turn [workarounds]enable_consoleauth = False
15:20:41 sean-k-mooney aspiers: efried that said we do translate vcpu/ram/disk in the scheulder/utils
15:20:47 alex_xu vpmem also need that, and stephenfin's standard cpu tracking need that also
15:20:54 aspiers efried: https://review.opendev.org/#/c/664420/42/nova/scheduler/request_filter.py@206
15:22:29 sean-k-mooney alex_xu ya so stephenfin's change will do it in the sculder utils replace this https://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L432
15:22:32 efried aspiers: So is the suggestion to put the translation from hw_mem_enc extra spec to MEM_ENC_CTX resource class into the sched/utils from_request_spec (or from_extra_specs) method instead of the request filter?
15:22:37 belmoreira melwitt, the main issue is because I'm running the novnc/console_auth globally (not per cell). And after the rocky->stein upgrade it can check tokens anymore (even when using [workarounds]enable_consoleauth=true)
15:22:43 sean-k-mooney but in general im not sure we shoudl keep doing that
15:22:48 melwitt belmoreira: right, for leftovers and also needed during the live upgrade to handle consoles while you're changing the deployment to deploy proxies per cell. once you have proxies per cell and upgrade complete, you can turn workarounds/enable_consoleauth = False and stop the nova-consoleauth service and stop having to deploy it
15:23:35 stephenfin melwitt: I think belmoreira is saying that it's still not working with 'workarounds/enable_consoleauth = True' though
15:23:48 aspiers efried: yes I think that's what alex_xu is suggesting
15:24:17 melwitt belmoreira, stephenfin: hm, ok. thinking...
15:24:18 sean-k-mooney aspiers: efried we could but i though we talked about this at the ptg and decied not to do that
15:24:28 alex_xu I'm ok with both, just want to figure out what is the plan for T release since we have serveal patch will do the same thing
15:24:37 sean-k-mooney i could be miss rememebring that
15:24:50 belmoreira stephenfin, melwitt it doesn't work with workarounds/enable_consoleauth = True (with everything running stein)
15:25:32 aspiers sean-k-mooney, alex_xu: what are the pros / cons?
15:25:39 melwitt belmoreira: I guess... that does make sense after the TTL has expired for already existing tokens, if you have not deployed proxies per cell yet
15:25:51 melwitt because new tokens are *not* stored in nova-consoleauth
15:26:03 melwitt *new token auths
15:26:16 aspiers both code paths run in the scheduler, right?
15:26:18 efried aspiers, alex_xu: Perhaps after we've merged stephenfin's patch to consolidate all that request spec mess, we could refactor the various request filters to do that. (I would still want it modular so it's not a monolithic mass of conversions.) But for now, that from_request_spec/from_extra_specs code is a bloody mess and it would be pretty hard to work the logic into the right places. So for now I say leave it in the request filte
15:26:36 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Get instances outside of update_available_resource lock https://review.opendev.org/677790
15:26:46 alex_xu efried: ok, got it
15:26:51 stephenfin efried: Are you referring to https://review.opendev.org/#/c/674894/ ?
15:26:57 aspiers efried: OK thanks
15:27:28 aspiers alex_xu: so that means https://review.opendev.org/#/c/664420/42 is ready for re-review
15:27:39 efried stephenfin: yes, that's the one.
15:27:42 belmoreira melwitt right... is there any easy way to enable tokens in nova-consoleauth again? (I'm in the middle of a crisis here)
15:27:52 alex_xu aspiers: yea
15:28:02 aspiers alex_xu: oops sorry I mean https://review.opendev.org/#/c/664420/ previous link was an old PS
15:28:08 stephenfin I think I could move that to the top of the list of approved patches if you want it now? I don't think it has a dependency on any of my other functional changes
15:28:12 efried stephenfin: I wonder, is there a way it could be moved out/earlier?... jinx
15:28:19 stephenfin coke!
15:28:41 aspiers alex_xu, stephenfin: also I fixed the tiny bug in https://review.opendev.org/#/c/638680
15:28:49 efried stephenfin: as discussed above, we're not blocking on it, we're going to continue to do the logic in the request filter for now. So don't worry about it unless it's easy.
15:29:04 stephenfin Sure. Let me try
15:32:18 melwitt belmoreira: hm, looking at the code, it should still work if you have workarounds/enable_consoleauth = True in stein. I see in nova/compute/api.py it still stores the token console auth in nova-consoleauth in addition to in the cell database. so I can't see why it wouldn't work yet
15:33:55 belmoreira melwitt from the consoleauth I only get: Checking Token: ***, False
15:34:50 belmoreira melwitt from nova-novncproxy: https://www.codepile.net/pile/O1P8ee1q
15:34:59 melwitt belmoreira: did you set workarounds/enable_consoleauth = True on hosts where nova-api and nova-novncproxy runs?
15:35:54 stephenfin belmoreira: Maybe it's just me, but I don't see anything in that codepile link
15:38:27 belmoreira melwitt workarounds/enable_consoleauth it's also for the nova-api? I'm only running it in the vnc nodes
15:38:55 melwitt belmoreira: you need it for nova-api to tell it to store token auth in nova-consoleauth service
15:39:21 melwitt and you need it for nonvcproxy to tell it to look in nova-consoleauth to find token auths when validating
15:39:24 melwitt so you need both
15:39:48 belmoreira that will be easy to change. Let me do it
15:41:04 openstackgerrit Merged openstack/nova master: Track libvirt host/domain capabilities for multiple machine types https://review.opendev.org/673151
15:41:13 openstackgerrit Merged openstack/nova master: Extract new base class for provider usage functional tests https://review.opendev.org/676964
15:53:12 efried I'm trying to hack up a patch to do that, but there's at least three things about it that I don't understand.
15:53:12 efried mriedem: clarkb suggested trying to repro the innodb thing with a post that copies database logs.
15:56:31 openstackgerrit Adam Spiers proposed openstack/nova master: Provide HW_CPU_X86_AMD_SEV trait when SEV is supported https://review.opendev.org/638680
15:56:42 openstackgerrit Stephen Finucane proposed openstack/nova master: claims: Remove useless caching https://review.opendev.org/671791

Earlier   Later