| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-24 | |||
| 14:33:06 | dansmith | promotion time? | |
| 14:33:30 | mriedem | i'm not the person to ask probably | |
| 14:33:44 | dansmith | sean-k-mooney: lol | |
| 14:33:48 | mriedem | it won't block nova of course | |
| 14:33:57 | mriedem | we're just blocked by the rate limiting | |
| 14:34:23 | sean-k-mooney | mriedem: on the upside since neutron are not using there gate currently nova should speed up :P | |
| 14:36:55 | zigo | Is there an openstackclient equivalent to nova server-migration-show? | |
| 14:37:08 | amorin | hey all | |
| 14:38:21 | sean-k-mooney | zigo: not that i see at a glance | |
| 14:38:41 | zigo | sean-k-mooney: Ok, so I'm not completely bind then! :) | |
| 14:39:06 | stephenfin | zigo: I don't think so. That was one of the gaps we identified in Denver (summit) | |
| 14:39:09 | sean-k-mooney | you are not :) we dont have full parity | |
| 14:39:36 | sean-k-mooney | zigo: initally osc did not have admin commnds so that is the area we have the most gaps today | |
| 14:39:46 | zigo | Right. | |
| 14:40:46 | zigo | If I put arp_responder = True, will this speed-up MAC learning for the network equipment connected to my compute? | |
| 14:41:07 | zigo | It usually takes dozens of seconds for it to learn what the new floating IP location is... | |
| 14:41:09 | sean-k-mooney | no | |
| 14:41:21 | zigo | sean-k-mooney: Is there some tricks? :) | |
| 14:42:04 | sean-k-mooney | arp_resopnder = true need the l2_population ml2 driver and does arp suppression so that openflow rules rewrite the arp request to an arp responce | |
| 14:42:22 | sean-k-mooney | so if you set that and the vswich has the flow installed the arp will never get to the tor | |
| 14:42:22 | amorin | I am currently facing a race condition in my OpenStack deployment. If I delete a port while the instance was booting (before nova plug the interface), then nova plug the interface and left the interface like this, after a while I have a lot of interface staying on the host | |
| 14:42:37 | zigo | Right. | |
| 14:43:09 | sean-k-mooney | so it will have the opisite effect in that learnign will only happen when you get an non arp packet | |
| 14:43:28 | rouk | sean-k-mooney: https://bugs.launchpad.net/nova/+bug/1834048 let me know if more is needed. | |
| 14:43:29 | openstack | Launchpad bug 1834048 in OpenStack Compute (nova) "Nova waits indefinitely on ceph client hangs due to network problems" [Undecided,New] | |
| 14:43:52 | zigo | sean-k-mooney: Ok, thanks. | |
| 14:44:10 | mriedem | amorin: like this? https://review.opendev.org/#/c/663382/ | |
| 14:44:27 | sean-k-mooney | rouk: cool thanks for filing ^ lyarwood you might be able to determing if there is soemthing we can do to fix the ceph issue | |
| 14:44:58 | sean-k-mooney | mriedem: hehe i was looking for that but it might help | |
| 14:45:23 | sean-k-mooney | mriedem: that is specifcaly for the case wehre you delete the instance after teh vm has started running but before we have updated the db | |
| 14:45:33 | sean-k-mooney | i belive there are other case were this can happen too | |
| 14:45:34 | mriedem | dansmith: regarding promoting that revert, slaweq could request it since i wouldn't be surprised if it takes all day to run the nova change through and even then will likely fail on some gate bug | |
| 14:45:44 | sean-k-mooney | that was the one i know was definetly broken | |
| 14:45:45 | amorin | mriedem, sean-k-mooney almost like this | |
| 14:46:09 | dansmith | mriedem: yeah that's what I was thinking.. looking at how many other things have -1s on them, this could take a while | |
| 14:46:17 | dansmith | slaweq: mriedem: is that neutron job in neutron's gate? | |
| 14:46:33 | mriedem | it's voting for neutron so i expect it's in neutron's gate queue yeah | |
| 14:46:34 | sean-k-mooney | dansmith: yes it is | |
| 14:47:02 | dansmith | so anything that has a recent check on it will head to the gate but fail every time, making the gate queue super slow too yeah? | |
| 14:47:05 | amorin | mriedem: something like this: | |
| 14:47:07 | amorin | http://paste.openstack.org/show/753313/ | |
| 14:47:11 | sean-k-mooney | amorin: right in your case you are deleteing the neutorn port | |
| 14:47:19 | amorin | yes, while booting the instance | |
| 14:48:00 | sean-k-mooney | well you should not actully get stale port in that case in ovs | |
| 14:48:11 | mriedem | amorin: so nova plugs the vif, starts waiting for the network-vif-plugged event, the port is then deleted and we don't get the event so we timeout and fail the server build | |
| 14:48:17 | sean-k-mooney | in you logg you have the vif unplug which should clean it up | |
| 14:48:38 | mriedem | sean-k-mooney: pretty sure we don't handle vif unplug and actually unplug | |
| 14:48:44 | mriedem | from out of band network-vif-deleted events | |
| 14:48:57 | sean-k-mooney | mriedem: we dont but we do on instance delete | |
| 14:49:17 | sean-k-mooney | so when we role back the spawn we should call unplug and clean up | |
| 14:49:29 | amorin | I dont have unplug call | |
| 14:49:35 | mriedem | isn't that stephen's patch? | |
| 14:49:53 | sean-k-mooney | no his is for a specifc case were we get an exception form the db | |
| 14:50:02 | sean-k-mooney | in that case we dont call cleanup | |
| 14:50:10 | sean-k-mooney | for the timeout case we do | |
| 14:50:21 | mriedem | https://review.opendev.org/#/c/663382/3/nova/compute/manager.py@2023 | |
| 14:50:52 | mriedem | stephen's patch doesn't have any specific cases in it from what i can tell | |
| 14:51:07 | sean-k-mooney | the downstream bug did | |
| 14:51:14 | mriedem | "If an instance fails to build, which is possible for a variety of reasons, we may end up in a situation where we have remnants of a plugged VIF (typically files) left of the host. This is because we cleanup from the neutron perspective but don't attempt to unplug the VIF, a call which may have many side-effects depending on the VIF driver." | |
| 14:51:17 | amorin | my bad, I have unplug in my logs | |
| 14:51:18 | mriedem | oh well the downstream bug... | |
| 14:52:18 | sean-k-mooney | it should be linkend in the upstream one but its addressing the case of when UnexpectedDeletingTaskStateError is raised | |
| 14:52:52 | mriedem | the rh bz is linked into the lp bug, | |
| 14:53:09 | mriedem | i'm just annoyed that i need to follow the breadcrumb trail of bugs to know wtf is going on in a patch | |
| 14:53:34 | sean-k-mooney | ya it should have been copied into the upstream bug | |
| 14:53:57 | mriedem | amorin: if you have a reproducer of *your* scenario can you apply https://review.opendev.org/#/c/663382/ and see if that resolves it? | |
| 14:54:13 | sean-k-mooney | sorry about that. stephen pick that bug up from me because i didnt have time to start on it two weeks ago | |
| 14:54:20 | amorin | mriedem: yes | |
| 14:54:22 | amorin | will do | |
| 14:55:02 | mriedem | sean-k-mooney: no i'm wrong, the details are in the lp bug, it's fine | |
| 14:56:07 | sean-k-mooney | amorin: by the way stephenfin patch will just prevent new interface form leaking it will not clean up existing stale ports | |
| 14:56:55 | amorin | sean-k-mooney: yes of course :p | |
| 14:57:09 | sean-k-mooney | mriedem: as far as i can tell we could leak ports this way for years so its supper latent | |
| 15:22:52 | Nick_A | How do you set quota defaults for new projects? I did openstack quota set --instances 100 default, but new projects still show the instance limit at 10 | |
| 15:25:00 | mriedem | Nick_A: i suggest you read through https://docs.openstack.org/nova/latest/user/quotas.html and/or https://docs.openstack.org/nova/latest/admin/quotas.html | |
| 15:25:08 | mriedem | and if there is a problem in those docs, then bring it here | |
| 15:26:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: docs: Rewrite host aggregate, availability zone docs https://review.opendev.org/667133 | |
| 15:28:04 | Nick_A | I read through both before asking. Double checked my quota set command matches the docs. | |
| 15:28:34 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove orphaned comment from _get_group_details https://review.opendev.org/667135 | |
| 15:28:44 | mriedem | Nick_A: check if there are per-project quotas set on the project which would override the defaults | |
| 15:30:03 | openstackgerrit | Merged openstack/python-novaclient master: Add irrelevant files in dsvm job https://review.opendev.org/666217 | |
| 15:30:08 | mriedem | https://docs.openstack.org/nova/latest/user/quotas.html#checking-quota is the thing you want to check | |
| 15:31:58 | mriedem | stephenfin: pretty sure https://review.opendev.org/666217 is broken | |
| 15:33:01 | stephenfin | mriedem: Yup :( I'll submit a fixup | |
| 15:34:21 | openstackgerrit | Stephen Finucane proposed openstack/python-novaclient master: Only ignore unit test changes for functional jobs https://review.opendev.org/667147 | |
| 15:35:09 | Nick_A | http://paste.openstack.org/show/OOfHU4BMoigiHYmODPMm/ also set instance quota to 100 for "admin" before creating the new project | |
| 15:35:31 | openstackgerrit | Stephen Finucane proposed openstack/python-novaclient master: Only ignore unit test changes for functional jobs https://review.opendev.org/667147 | |
| 15:37:48 | mriedem | Nick_A: I think your usage is wrong here: "openstack quota set --instances 100 default" | |
| 15:37:59 | mriedem | that's setting instances quota to 100 for the "default" project, which doesn't exist | |
| 15:38:27 | mriedem | you should be using "--class default" https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/quota.html#quota-set | |
| 15:38:36 | Nick_A | That's what I thought but that seems to be what the docs indicate | |
| 15:38:58 | mriedem | the osc cli docs aren't clear about what a class is or that "default" is the only class supported by nova... | |
| 15:39:00 | Nick_A | https://docs.openstack.org/nova/latest/admin/quotas.html these specifically | |
| 15:39:22 | mriedem | you mean "2. Update a default value for a new project, for example:" | |
| 15:39:24 | mriedem | yeah that looks busted | |
| 15:39:31 | Nick_A | heh | |
| 15:39:57 | mriedem | "$ openstack quota set --QUOTA_NAME QUOTA_VALUE PROJECT_OR_CLASS" is also misleading | |
| 15:40:03 | mriedem | since if you use a class you need to specify --class | |
| 15:40:15 | mriedem | Nick_A: please hit the bug link at the top of the page and report a nova docs bug | |
| 15:42:03 | Nick_A | # openstack quota set --class --instances 100 default <-- worked thank you | |