| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-04-20 | |||
| 23:53:57 | sean-k-mooney | am it will be used both on the contoler and computes | |
| 23:54:13 | sean-k-mooney | so you are missing | |
| 23:54:16 | sean-k-mooney | [keystone_authtoken] | |
| 23:54:18 | sean-k-mooney | memcached_servers = localhost:11211 | |
| 23:54:25 | sean-k-mooney | or well your actull memcache servers | |
| 23:54:45 | abaindur | yeah, I dont see that config opt ever ok | |
| 23:54:49 | sean-k-mooney | this will be used to cach auth tokens for every api call we make | |
| 23:55:16 | abaindur | i dont see it ever set* | |
| 23:58:21 | sean-k-mooney | abaindur: so at least on the contoler side it has a significnat impact on the api | |
| 23:58:25 | openstack | Launchpad bug 1836642 in neutron "Metadata responses are very slow sometimes" [High,Incomplete] - Assigned to Slawek Kaplonski (slaweq) | |
| 23:58:25 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1836642 | |
| 23:58:53 | sean-k-mooney | we adress this porblem in the gate by truning on caching with https://github.com/openstack/devstack/commit/d33cdd01f83b891b010e0fd238f1816910f3fd77 | |
| 23:59:18 | abaindur | I dont see it on controller either | |
| 23:59:25 | sean-k-mooney | i am not sure if i tis use dby the compute node but i think it will be whenever it is calling cinder neutron or placment | |
| 23:59:35 | sean-k-mooney | abaindur: its optional | |
| 23:59:49 | sean-k-mooney | but it improves perfromacne alot | |
| #openstack-nova - 2020-04-21 | |||
| 00:00:01 | abaindur | so for pre-copy, you are suggesting to try memcache for keystone tokens? | |
| 00:00:29 | sean-k-mooney | ya but im not saying dont try enableing post-copy | |
| 00:00:48 | sean-k-mooney | im just sugging that memcache might impove the performace enough to not need it | |
| 00:00:54 | sean-k-mooney | i woudl test both | |
| 00:01:12 | sean-k-mooney | abaindur: how did you deploy openstack by the way | |
| 00:01:23 | abaindur | thanks, yea we'll probably try both options then to see which leads to least downtime | |
| 00:01:51 | sean-k-mooney | for both options is basically just a compute agent restart and cofig update | |
| 00:03:24 | openstack | Launchpad bug 1836642 in neutron "Metadata responses are very slow sometimes" [High,Incomplete] - Assigned to Slawek Kaplonski (slaweq) | |
| 00:03:24 | sean-k-mooney | abaindur: at least in the case of https://bugs.launchpad.net/nova/+bug/1836642 we were seeing edgcaces wher ethe metadta api respoce could take upwards of 12 - 16 seconds and that went away when we enabeld the cache | |
| 00:05:00 | sean-k-mooney | abaindur: although to be clear we enabel both the cache section and the memcahced_server in the keystone_authoken section | |
| 00:05:18 | sean-k-mooney | so all of the highlight section is how we adressed the isseu https://zuul.opendev.org/t/openstack/build/dcde79801a624c25b195a46ead7af562/log/controller/logs/etc/nova/nova-cpu_conf.txt#62-68 | |
| 00:09:27 | sean-k-mooney | abaindur: https://docs.openstack.org/keystonemiddleware/latest/middlewarearchitecture.html#improving-response-time might be of interst | |
| 00:11:30 | sean-k-mooney | i think that might only be relevent to the apis service however and might not apply to when we use the keystone auth client to talk to otehr apis but im not sure baout that. anyway its late here so im going to go o/ | |
| 00:11:58 | abaindur | sean-k-mooney: oh wait, we have memcached_servers set in keystone_authtoken | |
| 00:12:05 | abaindur | but we dont have a [cache] section with the other configs | |
| 00:12:37 | sean-k-mooney | ah the cache section is novas own internal cache config | |
| 00:13:07 | sean-k-mooney | so in the metadata api case it is what allows cache copies fo the metadata to be shareed between processes | |
| 00:14:00 | sean-k-mooney | i think your best path forward in that cae is likely your original one of enabling post copy. you might also want o consider auto convergence | |
| 00:27:50 | abaindur | ok thanks for the help! will update here later if i find out more | |
| 00:42:47 | openstackgerrit | Merged openstack/nova master: Allocate mdevs when resizing or reverting resize https://review.opendev.org/712741 | |
| 01:50:40 | openstackgerrit | Merged openstack/nova stable/train: Reject boot request for unsupported images https://review.opendev.org/708577 | |
| 06:38:58 | gibi | stephenfin, gmann, artom: I'm +2 on the policy code changes. | |
| 06:48:53 | gibi | I'm tempted to squash the doc follow up into the original doc patch for more readable diff but meh I wait for others to show up | |
| 06:55:19 | bauzas | gibi: gmann: fwiw, I'll leave the prelude section change pretty small for the policy changes, but you are free to amend it for clarifying what we finally got for policy changes | |
| 06:55:37 | bauzas | hopefully, will be posted later this morning | |
| 06:55:43 | gibi | bauzas: thanks | |
| 06:55:46 | bauzas | (and good morning Nova, sorry for my manners) | |
| 06:57:07 | bauzas | gibi: tbh, I'm mostly paraphrasing the cycle highlights with a couple of additions (the BFV instance rescue and the vGPU multiple types support) | |
| 06:57:18 | bauzas | gibi: worth mentioning others ? | |
| 06:58:00 | gibi | I think that is plenty | |
| 06:58:59 | bauzas | gibi: well, I'm not afraid, that will still be an order of magnitude less than the rest of the relnotes | |
| 06:59:15 | bauzas | https://blueprints.launchpad.net/nova/+spec/remove-xvpvncproxy and https://blueprints.launchpad.net/nova/+spec/remove-nova-network-ussuri should also be commented I tihnk | |
| 06:59:40 | bauzas | gibi: fwiw, this is how it looks at the moment without a prelude section https://docs.openstack.org/releasenotes/nova/unreleased.html | |
| 06:59:43 | bauzas | good luck with this. | |
| 07:00:17 | bauzas | anyway, late breakfast, kids are up | |
| 07:01:03 | gibi | hm, based on that I even highlight the fact that we dropped supporting python 2.7 | |
| 07:25:32 | openstack | Launchpad bug 1851545 in OpenStack Compute (nova) "Port update exception on nova unshelve for instance with PCI devices (part 2)" [Undecided,New] | |
| 07:25:32 | gibi | sean-k-mooney: I guess you have SRIOV env at hand. Could you check if the bug https://bugs.launchpad.net/nova/+bug/1851545 is valid? | |
| 07:26:01 | gibi | for me it would take a day or so to set one up | |
| 07:37:24 | bauzas | gibi: good point, okay, let's wrap this up and see what people think | |
| 07:38:27 | gibi | bauzas: yep, lets do the review on gerrit :) | |
| 07:59:58 | openstack | Launchpad bug 1788014 in OpenStack Compute (nova) rocky "when live migration fails due to a internal error rollback is not handeled correctly." [Medium,In progress] - Assigned to Elod Illes (elod-illes) | |
| 07:59:58 | elod | zigo melwitt : about https://bugs.launchpad.net/nova/+bug/1788014 , I realized that it needs maybe some refactoring... then I forgot about looking at it. :/ Will do today. I'll need some thorough review for that i guess. | |
| 08:00:26 | zigo | elod: Thanks ! | |
| 08:06:07 | bauzas | gosh, the tox target for relnotes takes a while recalculating all the relnotes... | |
| 08:38:49 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: Ussuri 21.0.0 prelude section https://review.opendev.org/721548 | |
| 08:38:59 | bauzas | wow, it took a while but I'm done | |
| 08:39:42 | openstackgerrit | Merged openstack/nova master: Add new default roles in quota class policies https://review.opendev.org/719100 | |
| 08:39:48 | bauzas | gibi: stephenfin: dansmith: melwitt: alex_xu: gmann: others, the prelude section patch is up for reviews https://review.opendev.org/721548 | |
| 08:40:34 | bauzas | lyarwood: too ^ | |
| 08:41:51 | bauzas | gibi: can I hold the bugs lock ? | |
| 08:42:32 | openstack | Launchpad bug 1831315 in OpenStack Compute (nova) "nova-manage cell_v2 discover_hosts fails for IPv6 - ValueError: invalid literal for int() with base 10 - db connection URI gets its brackets eaten" [Undecided,New] - Assigned to Harald Jensås (harald-jensas) | |
| 08:42:32 | gibi | bauzas: I'm looking at https://bugs.launchpad.net/nova/+bug/1831315 right now but after that I have to jump to your prelude patch so you can have the lock | |
| 08:42:32 | openstack | Launchpad bug 1831315 in OpenStack Compute (nova) "nova-manage cell_v2 discover_hosts fails for IPv6 - ValueError: invalid literal for int() with base 10 - db connection URI gets its brackets eaten" [Undecided,New] - Assigned to Harald Jensås (harald-jensas) | |
| 08:43:19 | bauzas | gibi: you don't order bugs by Age, right? | |
| 08:47:02 | gibi | bauzas: I skip those that I cannot do anyithing with | |
| 08:47:27 | bauzas | ok, I'll try to look at them and ask for more if needed | |
| 08:47:50 | gibi | thanks | |
| 09:00:33 | openstack | Launchpad bug 1860312 in OpenStack Compute (nova) "compute service failed to delete" [Undecided,New] | |
| 09:00:33 | bauzas | gibi: for example, https://bugs.launchpad.net/nova/+bug/1860312 looks unrelated to Nova | |
| 09:00:54 | gibi | bauzas: is it so? | |
| 09:01:18 | gibi | bauzas: for me it seems like we somehow manage to allow a db corruption | |
| 09:01:41 | gibi | "When I try to delete the old compute service(status down), it failed." | |
| 09:02:30 | gibi | sure I don't know how the deployment ended up in "ComputeHostNotFound: Compute host mgt-slave03 could not be found " state | |
| 09:02:52 | gibi | but I feel that nova should support deleting old unused compute services | |
| 09:05:37 | gibi | my problem is that I have no way to try to recreate the bug without building a kubenetes + ironic env | |
| 09:11:48 | kashyap | lyarwood: So ... one test failure seems legit here: https://review.opendev.org/#/c/708701/4 | |
| 09:12:04 | kashyap | lyarwood: You know the reason once you see the failing test: test_stable_device_rescue_cdrom_ide() + Q35 :) | |
| 09:16:25 | lyarwood | I've not had a look yet no | |
| 09:16:38 | lyarwood | I assume with that test that q35 doesn't have an IDE bus? | |
| 09:16:47 | lyarwood | iirc that's the case right | |
| 09:17:10 | lyarwood | I can add a configurable in tempest for that | |
| 09:17:25 | lyarwood | the config drive issue is still there as well btw | |
| 09:18:32 | openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Add stable device rescue docs https://review.opendev.org/700837 | |
| 09:18:33 | kashyap | lyarwood: Yeah, the issue is indeed 'q35' has no IDE bus | |
| 09:18:55 | kashyap | lyarwood: Sorry, which config drive issue? | |
| 09:19:35 | lyarwood | kashyap: see test_server_basic_ops etc, blkid -L config-2 -o device', exit status: 2 | |
| 09:19:46 | lyarwood | ah wait | |
| 09:19:57 | lyarwood | that's because that's a cdrom | |
| 09:20:06 | lyarwood | over SCSI? | |
| 09:20:29 | kashyap | lyarwood: You mean over IDE, instead of SATA (or 'virtio')? | |
| 09:21:11 | lyarwood | kashyap: we don't have IDE, I can't recall if we default to SCSI in that case or SATA | |
| 09:21:13 | kashyap | I'm not sure yet ... I'll see what my non-configurable one-liner yields: https://review.opendev.org/#/c/721555/ | |
| 09:21:43 | lyarwood | kashyap: nack, IDE + cdrom is still a valid test case prior to q35 IMHO | |