| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-08 | |||
| 12:51:55 | sean-k-mooney | looking at https://github.com/hhatto/autopep8#features there are seveeral that are fixing indentaiton or addign missing blank lines | |
| 12:58:14 | sean-k-mooney | it might be comeing form https://www.python.org/dev/peps/pep-0257/ | |
| 13:01:01 | sean-k-mooney | """Insert a blank line after all docstrings (one-line or multi-line) that document a class -- generally speaking, the class's methods are separated from each other by a single blank line, and the docstring needs to be offset from the first method by a blank line.""" | |
| 13:01:14 | sean-k-mooney | https://www.python.org/dev/peps/pep-0257/#multi-line-docstrings | |
| 13:01:23 | sean-k-mooney | lyarwood: ^ that is where that si cominng form | |
| 13:02:38 | sean-k-mooney | ah and its called out in the autopep8 | |
| 13:02:40 | sean-k-mooney | Put a blank line between a class docstring and its first method declaration. (Enabled with E301.) | |
| 13:03:27 | sean-k-mooney | E301 is inconsitent tabs and spaces | |
| 13:03:29 | sean-k-mooney | https://www.flake8rules.com/rules/E101.html | |
| 13:08:12 | lyarwood | and that's enabled so why isn't flake8 picking it up already? | |
| 13:08:29 | sean-k-mooney | its not part of pep8 | |
| 13:08:41 | lyarwood | ah | |
| 13:08:46 | sean-k-mooney | E301 does not specify the docstring behavior | |
| 13:09:00 | sean-k-mooney | auto pep8 impleted as part of the E301 fixing | |
| 13:09:04 | lyarwood | sorry I thought you said it did, I see what you're sayingnow | |
| 13:13:14 | sean-k-mooney | lyarwood: summerised it in the review https://review.opendev.org/c/openstack/nova/+/806182/3#message-f37506ed0411c71d9ce66de251196048da171b34 | |
| 13:15:31 | gibi | sean-k-mooney: I have one question in autopep8 https://review.opendev.org/c/openstack/nova/+/806182/comment/e15200f9_a50f0219/ | |
| 13:16:08 | sean-k-mooney | we can add it to test-requiremetns yes | |
| 13:16:21 | sean-k-mooney | i just did not to avoid installing it in enves that did not need it | |
| 13:16:36 | sean-k-mooney | but if you prefer that i can update it shortly | |
| 13:16:41 | gibi | hm, will we ever need to pin the version of autopep8? | |
| 13:17:07 | sean-k-mooney | this can be done with upperconstraitns | |
| 13:17:11 | sean-k-mooney | or inline | |
| 13:17:36 | sean-k-mooney | i dont think so but its a vaild question | |
| 13:17:50 | gibi | currently it can only be inline afaik. but anyhow we don't pin it now, and we can move it to test-requirements when we need to pin it | |
| 13:17:59 | gibi | so I'm +2 | |
| 13:19:04 | sean-k-mooney | for line 69 yes since that does not include the base test env dep for the pep8 env the upper constratis form the base env still apply | |
| 13:22:56 | lyarwood | okay LGTM | |
| 13:23:27 | lyarwood | oh wait, downstream meeting :| | |
| 13:23:41 | lyarwood | helps if I look at the correct day in my cal | |
| 13:26:13 | opendevreview | Merged openstack/nova master: db: Remove legacy placement models https://review.opendev.org/c/openstack/nova/+/812146 | |
| 13:26:23 | opendevreview | Merged openstack/nova master: objects: Stop querying the main DB for keypairs https://review.opendev.org/c/openstack/nova/+/812147 | |
| 13:50:21 | bauzas | gibi: +w'd your series | |
| 13:50:27 | gibi | bauzas: thanks | |
| 14:13:47 | gibi | melwitt: hi! When you are up, we are waiting for you +A in https://review.opendev.org/c/openstack/nova/+/815689/2 so if you have time please check | |
| 14:23:57 | opendevreview | Merged openstack/nova master: Revert "tox: Encode specific Python versions" https://review.opendev.org/c/openstack/nova/+/804168 | |
| 14:30:31 | EugenMayer | When doing a nova backup, it seems like ephemeral disks are entirely excluded. is this to be expected? | |
| 14:30:51 | sean-k-mooney | EugenMayer: yes | |
| 14:31:06 | sean-k-mooney | EugenMayer: snapshots and backups are only done of the root disk | |
| 14:31:17 | EugenMayer | wow, that is a huge bummer ;/ | |
| 14:31:51 | EugenMayer | Is there any way to workarround that? somehow using glance directly and creating an image per disk? | |
| 14:31:52 | sean-k-mooney | ephemeral is not intended to be used for storing data that cant be recreated or safely lost | |
| 14:32:12 | sean-k-mooney | no it was exluded by design and cant be worked around | |
| 14:32:18 | sean-k-mooney | without code changes | |
| 14:32:28 | EugenMayer | Well, the problem is, there is no way to have multi-disk environment without defining one as ephemeral, right? | |
| 14:32:43 | sean-k-mooney | not without cinder | |
| 14:33:01 | EugenMayer | yes, those are volumes then | |
| 14:33:31 | sean-k-mooney | with openstack and most cloud plathform the expected and recommended usecase is to have only 1 disk with any addtional storage provided via volumes | |
| 14:34:20 | kevko | Hi nova team :) , testing wallaby and when I'm creating heat stack ..from time to time I see this error in nova .. is it known bug ? | |
| 14:34:21 | kevko | https://paste.opendev.org/show/810848/ | |
| 14:34:51 | sean-k-mooney | not in nova that normally means you have an issue with neutron | |
| 14:34:56 | EugenMayer | sean-k-mooney i understand that since one scale that one disk easily, no need to add others. But those VMs are legacy and partially it is part of the application desing to have 2 disks (on premise things). But well, still that now is somewhat a huge showstopper is was not accounting for yet | |
| 14:35:12 | dansmith | kevko: yeah, means neutron failed to send the even to nova to indicate that networking is ready | |
| 14:35:20 | sean-k-mooney | kevko: if you create really large heat stacks and your neutron installation is slow then it can time out wireing up the ports | |
| 14:36:01 | sean-k-mooney | you can workaround it by extending the timeout but the real fix is to tune your neutron deployment to support the workload | |
| 14:37:00 | kevko | well, it's really small heat template | |
| 14:37:44 | sean-k-mooney | is this in the upstream ci or a production issue in your deployment by the way | |
| 14:38:01 | sean-k-mooney | ah you said you were testing it so i guess its in your lab | |
| 14:38:10 | sean-k-mooney | what network backend are you using? | |
| 14:38:11 | EugenMayer | sean-k-mooney thank you for givin the definit answer. Not sure what i make out of it in terms of options/solutions but at least good to know it is intended and road blocked | |
| 14:38:24 | kevko | well, it is production problem also on client's side .. | |
| 14:38:31 | kevko | so i'm trying to replicate issue on lab | |
| 14:38:42 | kevko | i can't see relevant errors in neutron | |
| 14:39:15 | sean-k-mooney | kevko: ack is this ml2/ovn? ml2/ovs? cisco aci perhaps? | |
| 14:39:59 | sean-k-mooney | kevko: on the neutron side you likely wont see an error more it wont have sent the event before the timeout exired | |
| 14:40:37 | sean-k-mooney | kevko: if its close but not quite longenough you might actully see the network-vif-plugged event in the nova log after the timeout exired as an unexpected event | |
| 14:41:03 | dansmith | sean-k-mooney: we should debug log the time we waited each time so that people can see if they're close to the deadline normally, or if it's normally 5 seconds, one 300 timeout is obviously an event that's never coming | |
| 14:41:09 | kevko | ml2/ovs | |
| 14:41:17 | sean-k-mooney | if the env is using ml2/ovs you would need to check the neutron l2 agent log to see if it processed the port. for the other two you would hae to check the neutron server log i belive | |
| 14:43:54 | sean-k-mooney | dansmith: ya that is not a bad idea, on a related note i was speaking to bogdando downstream about also potentially queueing the event in the compute agent if no handeler with a TTL that might help in case that the event come before we start waiting but there are downsides to that. | |
| 14:44:29 | dansmith | sean-k-mooney: the entire point of that infrastructure is to make it hard to allow such things :) | |
| 14:44:47 | dansmith | sean-k-mooney: we prepare the event before we do the thing that could cause it to be fired | |
| 14:44:58 | dansmith | otherwise we turn it into a statistical race condition problem | |
| 14:44:58 | sean-k-mooney | yep i know i mentioned that we did not do that for a reason and we discused it before and rejected it | |
| 14:45:40 | sean-k-mooney | am by the way i would like your in put on the healthcheck design at some point | |
| 14:45:55 | dansmith | is something up to review? | |
| 14:46:11 | kevko | sean-k-mooney: regarding above ... timeout for rpc response ? or which timeout ..sorry :/ | |
| 14:46:55 | sean-k-mooney | not yet ill likely start on the spec tomorrow or later today but the open question i have is should the datastucture be storge in a module gobal or passed via the context to each fucntion | |
| 14:47:12 | sean-k-mooney | kevko: ill get you the link to the docs but there is a vif_plug_timeout you should increase | |
| 14:47:26 | sean-k-mooney | kevko: that is really jsut a workaround however | |
| 14:47:49 | kevko | so, bump number of neutron workers ? | |
| 14:47:54 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.vif_plugging_timeout | |
| 14:48:05 | dansmith | kevko: in all likelihood, you're not just a couple seconds late for a *5* minute timeout.. much more likely that it's never coming and changing the timeout will just make things more painful | |
| 14:48:19 | sean-k-mooney | yep | |
| 14:48:32 | sean-k-mooney | kevko: what you really need to do is look at the neutron agent logs on the compute | |
| 14:48:43 | dansmith | kevko: you should see messages in the nova-compute log saying that the event actually showed up but was discarded - if you don't, then changing the timeout just makes your system worse | |
| 14:48:46 | sean-k-mooney | and see when the port was added and how far it got in wiring it up | |
| 14:52:59 | kevko | sean-k-mooney: same log + neutron-ovs-agent 10 minutes +/- ago https://paste.opendev.org/show/810850/ | |
| 14:54:04 | sean-k-mooney | kevko: are you using iptbals by the way or openvswtich firewall driver | |
| 14:54:13 | kevko | ovs | |
| 14:54:36 | sean-k-mooney | ok in that case in wallaby the port is plugged by libvirt as part of the instance creation | |
| 14:54:40 | sean-k-mooney | gibi: o/ | |
| 14:55:03 | opendevreview | Dan Smith proposed openstack/nova master: Log VIF event wait times https://review.opendev.org/c/openstack/nova/+/817030 | |
| 14:55:10 | dansmith | sean-k-mooney: ^ | |
| 14:55:15 | sean-k-mooney | kevko: at least presently since i have not backported the change to delegate taht to os-vif yet | |
| 14:56:05 | sean-k-mooney | oh you are using the Stopwatch functionality instead of time.now and subtracting ya that is nicer | |
| 14:56:36 | kevko | sean-k-mooney: what does it mean ? :/ | |
| 14:56:45 | kevko | checking bad logs ? | |
| 14:57:10 | sean-k-mooney | kevko: that was to dansmith he has already created a patch to log the elapsed time | |
| 14:57:21 | sean-k-mooney | kevko: https://review.opendev.org/c/openstack/nova/+/817030/1/nova/compute/manager.py#491 | |