| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-22 | |||
| 15:19:01 | aspiers | but I don't know enough to understand how that relates to this discussion right now | |
| 15:19:17 | sean-k-mooney | efried: we have a few new request filters this cycle. i think it makes sense to prot them all in U | |
| 15:19:22 | efried | melwitt: Thank you for the fix around websockify. And stephenfin thanks for digging into the symbol alias business. I guess we're stuck supporting the dual-location business :( | |
| 15:19:35 | efried | aspiers: where was that part? | |
| 15:19:41 | sean-k-mooney | aspiers: we were moving away form that intentionally | |
| 15:19:46 | aspiers | efried: let me find it | |
| 15:19:53 | belmoreira | melwitt: workarounds/enable_consoleauth = True is only for the leftovers, right? becasue in stein the DB will be always used to store the token | |
| 15:20:09 | sean-k-mooney | e.g. we do some traslation in utils but we disucess not doing that and using request filster intead | |
| 15:20:27 | melwitt | mnaser: belmoreira just upgraded to stein and novnc started failing to work, console proxies are not yet deployed per cell. I'm thinking he needs to set [workarounds]enable_consoleauth = True during the live upgrade and then deploy proxies per cell and then after upgrade be able to turn [workarounds]enable_consoleauth = False | |
| 15:20:41 | sean-k-mooney | aspiers: efried that said we do translate vcpu/ram/disk in the scheulder/utils | |
| 15:20:47 | alex_xu | vpmem also need that, and stephenfin's standard cpu tracking need that also | |
| 15:20:54 | aspiers | efried: https://review.opendev.org/#/c/664420/42/nova/scheduler/request_filter.py@206 | |
| 15:22:29 | sean-k-mooney | alex_xu ya so stephenfin's change will do it in the sculder utils replace this https://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L432 | |
| 15:22:32 | efried | aspiers: So is the suggestion to put the translation from hw_mem_enc extra spec to MEM_ENC_CTX resource class into the sched/utils from_request_spec (or from_extra_specs) method instead of the request filter? | |
| 15:22:37 | belmoreira | melwitt, the main issue is because I'm running the novnc/console_auth globally (not per cell). And after the rocky->stein upgrade it can check tokens anymore (even when using [workarounds]enable_consoleauth=true) | |
| 15:22:43 | sean-k-mooney | but in general im not sure we shoudl keep doing that | |
| 15:22:48 | melwitt | belmoreira: right, for leftovers and also needed during the live upgrade to handle consoles while you're changing the deployment to deploy proxies per cell. once you have proxies per cell and upgrade complete, you can turn workarounds/enable_consoleauth = False and stop the nova-consoleauth service and stop having to deploy it | |
| 15:23:35 | stephenfin | melwitt: I think belmoreira is saying that it's still not working with 'workarounds/enable_consoleauth = True' though | |
| 15:23:48 | aspiers | efried: yes I think that's what alex_xu is suggesting | |
| 15:24:17 | melwitt | belmoreira, stephenfin: hm, ok. thinking... | |
| 15:24:18 | sean-k-mooney | aspiers: efried we could but i though we talked about this at the ptg and decied not to do that | |
| 15:24:28 | alex_xu | I'm ok with both, just want to figure out what is the plan for T release since we have serveal patch will do the same thing | |
| 15:24:37 | sean-k-mooney | i could be miss rememebring that | |
| 15:24:50 | belmoreira | stephenfin, melwitt it doesn't work with workarounds/enable_consoleauth = True (with everything running stein) | |
| 15:25:32 | aspiers | sean-k-mooney, alex_xu: what are the pros / cons? | |
| 15:25:39 | melwitt | belmoreira: I guess... that does make sense after the TTL has expired for already existing tokens, if you have not deployed proxies per cell yet | |
| 15:25:51 | melwitt | because new tokens are *not* stored in nova-consoleauth | |
| 15:26:03 | melwitt | *new token auths | |
| 15:26:16 | aspiers | both code paths run in the scheduler, right? | |
| 15:26:18 | efried | aspiers, alex_xu: Perhaps after we've merged stephenfin's patch to consolidate all that request spec mess, we could refactor the various request filters to do that. (I would still want it modular so it's not a monolithic mass of conversions.) But for now, that from_request_spec/from_extra_specs code is a bloody mess and it would be pretty hard to work the logic into the right places. So for now I say leave it in the request filte | |
| 15:26:36 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Get instances outside of update_available_resource lock https://review.opendev.org/677790 | |
| 15:26:46 | alex_xu | efried: ok, got it | |
| 15:26:51 | stephenfin | efried: Are you referring to https://review.opendev.org/#/c/674894/ ? | |
| 15:26:57 | aspiers | efried: OK thanks | |
| 15:27:28 | aspiers | alex_xu: so that means https://review.opendev.org/#/c/664420/42 is ready for re-review | |
| 15:27:39 | efried | stephenfin: yes, that's the one. | |
| 15:27:42 | belmoreira | melwitt right... is there any easy way to enable tokens in nova-consoleauth again? (I'm in the middle of a crisis here) | |
| 15:27:52 | alex_xu | aspiers: yea | |
| 15:28:02 | aspiers | alex_xu: oops sorry I mean https://review.opendev.org/#/c/664420/ previous link was an old PS | |
| 15:28:08 | stephenfin | I think I could move that to the top of the list of approved patches if you want it now? I don't think it has a dependency on any of my other functional changes | |
| 15:28:12 | efried | stephenfin: I wonder, is there a way it could be moved out/earlier?... jinx | |
| 15:28:19 | stephenfin | coke! | |
| 15:28:41 | aspiers | alex_xu, stephenfin: also I fixed the tiny bug in https://review.opendev.org/#/c/638680 | |
| 15:28:49 | efried | stephenfin: as discussed above, we're not blocking on it, we're going to continue to do the logic in the request filter for now. So don't worry about it unless it's easy. | |
| 15:29:04 | stephenfin | Sure. Let me try | |
| 15:32:18 | melwitt | belmoreira: hm, looking at the code, it should still work if you have workarounds/enable_consoleauth = True in stein. I see in nova/compute/api.py it still stores the token console auth in nova-consoleauth in addition to in the cell database. so I can't see why it wouldn't work yet | |
| 15:33:55 | belmoreira | melwitt from the consoleauth I only get: Checking Token: ***, False | |
| 15:34:50 | belmoreira | melwitt from nova-novncproxy: https://www.codepile.net/pile/O1P8ee1q | |
| 15:34:59 | melwitt | belmoreira: did you set workarounds/enable_consoleauth = True on hosts where nova-api and nova-novncproxy runs? | |
| 15:35:54 | stephenfin | belmoreira: Maybe it's just me, but I don't see anything in that codepile link | |
| 15:38:27 | belmoreira | melwitt workarounds/enable_consoleauth it's also for the nova-api? I'm only running it in the vnc nodes | |
| 15:38:55 | melwitt | belmoreira: you need it for nova-api to tell it to store token auth in nova-consoleauth service | |
| 15:39:21 | melwitt | and you need it for nonvcproxy to tell it to look in nova-consoleauth to find token auths when validating | |
| 15:39:24 | melwitt | so you need both | |
| 15:39:48 | belmoreira | that will be easy to change. Let me do it | |
| 15:41:04 | openstackgerrit | Merged openstack/nova master: Track libvirt host/domain capabilities for multiple machine types https://review.opendev.org/673151 | |
| 15:41:13 | openstackgerrit | Merged openstack/nova master: Extract new base class for provider usage functional tests https://review.opendev.org/676964 | |
| 15:53:12 | efried | mriedem: clarkb suggested trying to repro the innodb thing with a post that copies database logs. | |
| 15:53:12 | efried | I'm trying to hack up a patch to do that, but there's at least three things about it that I don't understand. | |
| 15:56:31 | openstackgerrit | Adam Spiers proposed openstack/nova master: Provide HW_CPU_X86_AMD_SEV trait when SEV is supported https://review.opendev.org/638680 | |
| 15:56:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: claims: Remove useless caching https://review.opendev.org/671791 | |
| 15:56:43 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Rename exception argument https://review.opendev.org/671795 | |
| 15:56:43 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.get_host_numa_usage_from_instance' https://review.opendev.org/671797 | |
| 15:56:44 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.host_topology_and_format_from_host' https://review.opendev.org/671798 | |
| 15:56:44 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.instance_topology_from_instance' https://review.opendev.org/671799 | |
| 15:56:45 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Rework 'hardware.numa_usage_from_instances' https://review.opendev.org/672565 | |
| 15:56:45 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Split NUMA object tests https://review.opendev.org/672336 | |
| 15:56:46 | openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props' https://review.opendev.org/674894 | |
| 15:56:46 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove legacy '_from_dict' functions https://review.opendev.org/537414 | |
| 15:56:47 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add '[compute] cpu_dedicated_set' option https://review.opendev.org/671792 | |
| 15:56:47 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793 | |
| 15:56:48 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available' https://review.opendev.org/672693 | |
| 15:56:48 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Rewrap definitions of 'NUMACell' https://review.opendev.org/674395 | |
| 15:56:49 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | |
| 15:56:49 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Rename 'fields' import to 'obj_fields' https://review.opendev.org/674103 | |
| 15:56:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571 | |
| 15:56:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples https://review.opendev.org/675338 | |
| 15:56:51 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 15:57:02 | stephenfin | efried: Yeah, trivial to rebase | |
| 15:57:07 | efried | nice, thanks stephenfin | |
| 15:57:23 | openstackgerrit | Adam Spiers proposed openstack/nova master: Extract SEV-specific bits on host detection https://review.opendev.org/636334 | |
| 15:57:57 | openstackgerrit | Adam Spiers proposed openstack/nova master: Add |
|
| 15:58:12 | openstackgerrit | Adam Spiers proposed openstack/nova master: Apply SEV-specific guest config when SEV is required https://review.opendev.org/644565 | |
| 15:58:26 | openstackgerrit | Adam Spiers proposed openstack/nova master: Enable booting of libvirt guests with AMD SEV memory encryption https://review.opendev.org/666616 | |
| 16:00:24 | efried | mriedem: Any idea what this [1] would look like if I wanted to copy "the database logs" ? | |
| 16:00:24 | efried | [1] https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/playbooks/infra-puppet-apply/post.yaml | |
| 16:06:00 | openstackgerrit | melanie witt proposed openstack/nova master: Handle websockify v0.9.0 in console proxy https://review.opendev.org/677856 | |
| 16:06:02 | mriedem | efried: i know devstack based jobs copy the mysqldb logs | |
| 16:06:37 | mriedem | well they used to anyway | |
| 16:08:20 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Try to repro bug 1823251 with mysql logs https://review.opendev.org/678051 | |
| 16:08:21 | openstack | bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed] https://launchpad.net/bugs/1823251 | |
| 16:08:29 | mriedem | efried: i'd guess it's a matter of adding a post-run play to collect the db logs https://github.com/openstack/nova/blob/master/playbooks/legacy/nova-lvm/post.yaml | |
| 16:08:31 | mriedem | like ^ | |
| 16:08:58 | mriedem | which i think runs after some other tempest/devstack post task copies things to /logs | |
| 16:09:17 | mriedem | where the sqlite lots are stored, idk - or does this only fail in mysql? | |
| 16:09:52 | mriedem | devstack-gate for legacy jobs would save off the mysql logs here https://github.com/openstack/devstack-gate/blob/9459f44cb5f604b84ca6b5642b89d7aeceae9f35/functions.sh#L667 | |
| 16:11:02 | mriedem | so we need a play that copies those to devstack_log_dir | |
| 16:11:05 | mriedem | $devstack_log_dir | |
| 16:12:12 | mriedem | ok i see something in devstack-base | |
| 16:12:32 | mriedem | https://github.com/openstack/devstack/blob/master/.zuul.yaml#L243 | |