| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-02-13 | |||
| 23:45:04 | sean-k-mooney | this is realy really old but if you have not seen it before its how neutron ovs networking used to work | |
| 23:45:06 | sean-k-mooney | https://www.rdoproject.org/networking/networking-in-too-much-detail/ | |
| 23:45:17 | sean-k-mooney | its now simpler | |
| 23:45:43 | sean-k-mooney | but it is a good thing to read over at least once even if its not how it works exactly today | |
| 23:48:18 | dking_desktop | Thank you. I'll check that out. It may help with some of the dark spots in my knowledge. | |
| #openstack-nova - 2020-02-14 | |||
| 00:59:21 | huaqiang | stephenfin: thank you and all for the review and patient. | |
| 01:11:25 | huaqiang | s/patient/patience/ :) | |
| 01:28:18 | brinzhang | efried: I think https://review.opendev.org/#/c/580336 all things are ready in SPEC, and also provide it's PoC codes https://review.opendev.org/#/c/693828/ agreement reached at Shanghai PTG https://etherpad.openstack.org/p/nova-shanghai-ptg(Line253) | |
| 01:31:29 | johnsom | Hi nova folks. FYI, I opened a bug that anti-affinity appears to have been broken since at least August. https://bugs.launchpad.net/nova/+bug/1863190 Let me know if there is more information I should collect while I have the instance up and running. | |
| 01:31:29 | openstack | Launchpad bug 1863190 in OpenStack Compute (nova) "Server group anti-affinity no longer works" [Undecided,New] | |
| 02:09:05 | melwitt | johnsom: did you boot the two instances at the same time or near the same time? did you use multi-create? | |
| 02:10:06 | johnsom | They were close in time, but individual calls to the nova api | |
| 02:10:32 | johnsom | Our normal active/standby boot sequence | |
| 02:11:21 | melwitt | ok. I'm wondering if it's a race basically | |
| 02:11:55 | melwitt | like if you wait for the first instance to become active and then do another request, would it fail | |
| 02:11:58 | johnsom | Yeah, probably. This used to work. | |
| 02:12:41 | johnsom | Yeah, I don’t know about waiting for active. That can take some time. | |
| 02:13:46 | johnsom | We have seen that up to five minutes, mostly scheduler time. I don’t think we want to hold up the boot process for the secondary that long | |
| 02:14:30 | melwitt | no sorry, I meant in your devstack | |
| 02:15:15 | melwitt | I can try it later too but not tonight. I was just asking in case you had that easy environment already up | |
| 02:15:21 | johnsom | Ah, as a test. I can try that. It will be tomorrow however, it is dinner time. Lol | |
| 02:15:27 | melwitt | same | |
| 02:15:52 | johnsom | If there is a list of thing you would like me to try, comment on the bug and I will run them tomorrow | |
| 02:15:57 | melwitt | there's a thing called the "late affinity check" in nova-compute that is on by default that should handle races | |
| 02:16:35 | melwitt | I don't know off the top of my head what could have regressed this. | |
| 02:18:03 | melwitt | I'll try out some things with a devstack tomorrow to start finding out what's going on | |
| 02:18:46 | melwitt | efried: potential regression alert ^ fyi https://bugs.launchpad.net/nova/+bug/1863190 | |
| 02:18:46 | openstack | Launchpad bug 1863190 in OpenStack Compute (nova) "Server group anti-affinity no longer works" [Undecided,New] | |
| 02:18:53 | johnsom | Ok, thanks. I will hold this instance through tomorrow in case we need it | |
| 02:21:14 | melwitt | k thanks | |
| 02:55:29 | openstackgerrit | Liang Fang proposed openstack/nova-specs master: Support volume local cache https://review.opendev.org/689070 | |
| 02:58:01 | openstackgerrit | Brin Zhang proposed openstack/nova master: Introduce scope_types in os-instance-action policy https://review.opendev.org/707751 | |
| 06:41:43 | openstackgerrit | Brin Zhang proposed openstack/nova master: Introduce scope_types in os-instance-action policy https://review.opendev.org/707751 | |
| 06:42:13 | openstackgerrit | Brin Zhang proposed openstack/nova master: Introduce scope_types in os-instance-action policy https://review.opendev.org/707751 | |
| 07:57:56 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add test coverage of existing os-instance-actions policies https://review.opendev.org/707777 | |
| 07:58:47 | openstackgerrit | Brin Zhang proposed openstack/nova master: Introduce scope_types in os-instance-action policy https://review.opendev.org/707751 | |
| 08:59:44 | openstackgerrit | Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.opendev.org/623120 | |
| 09:03:10 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add test coverage of existing os-instance-actions policies https://review.opendev.org/707777 | |
| 09:07:21 | openstackgerrit | Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.opendev.org/623120 | |
| 09:28:53 | openstackgerrit | Brin Zhang proposed openstack/nova master: Introduce scope_types in os-instance-action policy https://review.opendev.org/707751 | |
| 10:02:02 | openstackgerrit | Lee Yarwood proposed openstack/nova master: DNM - Test TEMPEST_EXTEND_ATTACHED_ENCRYPTED_VOLUME https://review.opendev.org/707593 | |
| 10:11:38 | openstackgerrit | Merged openstack/nova-specs master: Add nova-audit spec https://review.opendev.org/693226 | |
| 10:23:08 | openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: Proposes NUMA topology with RPs https://review.opendev.org/552924 | |
| 10:23:23 | bauzas | stephenfin: sean-k-mooney: efried: I made another iteration to clarify ^ | |
| 10:29:44 | openstackgerrit | Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735 | |
| 10:29:45 | openstackgerrit | Sundar Nadathur proposed openstack/nova master: Enable hard/soft reboot with accelerators. https://review.opendev.org/697940 | |
| 10:29:45 | openstackgerrit | Sundar Nadathur proposed openstack/nova master: Enable start/stop of instances with accelerators. https://review.opendev.org/699553 | |
| 10:29:46 | openstackgerrit | Sundar Nadathur proposed openstack/nova master: Enable and use COMPUTE_ACCELERATORS trait. https://review.opendev.org/699554 | |
| 10:29:46 | openstackgerrit | Sundar Nadathur proposed openstack/nova master: Bump compute rpcapi version and reduce Cyborg calls. https://review.opendev.org/704227 | |
| 10:29:47 | openstackgerrit | Sundar Nadathur proposed openstack/nova master: Add cyborg tempest job. https://review.opendev.org/670999 | |
| 10:47:01 | openstackgerrit | Lee Yarwood proposed openstack/nova master: api: Introduce microverion 2.82 allowing boot from volume rescue https://review.opendev.org/701430 | |
| 10:47:02 | openstackgerrit | Lee Yarwood proposed openstack/nova master: compute: Extract _get_bdm_image_metadata into nova.utils https://review.opendev.org/705212 | |
| 11:53:14 | openstackgerrit | Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.opendev.org/623120 | |
| 12:18:02 | openstackgerrit | Merged openstack/nova stable/rocky: Mask the token used to allow access to consoles https://review.opendev.org/704255 | |
| 12:41:43 | sean-k-mooney | bauzas: i think you missed some of efried's comments. like explainging the fallback query. | |
| 12:42:29 | sean-k-mooney | efried: also did you see my suggestion to have teh numa=false host be added to an aggreate so we can use forbiden aggreates for that case https://review.opendev.org/#/c/552924/21/specs/ussuri/approved/numa-topology-with-rps.rst@235 | |
| 12:46:33 | sean-k-mooney | over all however i think there is enough we agree on that we can proceed with this and adject with a fix up patch later based on implementaion discussions | |
| 12:47:43 | sean-k-mooney | i am off today so im not going to be around but i might see pings in the evening as my irc client will be connected | |
| 12:59:21 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.opendev.org/623120 | |
| 13:48:05 | bauzas | sean-k-mooney: again, I explained this | |
| 13:48:55 | bauzas | sean-k-mooney: oops my bad, I didn't passed my draft comments | |
| 13:48:59 | bauzas | sean-k-mooney: see them | |
| 13:49:31 | bauzas | stephenfin: can you please review it ? https://review.opendev.org/#/c/552924/ | |
| 14:19:25 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/queens: Mask the token used to allow access to consoles https://review.opendev.org/707845 | |
| 14:20:33 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.opendev.org/623120 | |
| 15:03:39 | bauzas | efried: stephenfin: the spec is waiting for your comments :) https://review.opendev.org/#/c/552924/ | |
| 15:03:58 | melwitt | johnthetubaguy, bauzas: thank you both for your review on the nova-audit spec ++ | |
| 15:08:04 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.opendev.org/623120 | |
| 15:10:57 | melwitt | alex_xu: hey, did you mean to hold off on +W on the unified limits spec? https://review.opendev.org/602201 | |
| 15:25:17 | openstackgerrit | Alexandre arents proposed openstack/nova master: Avoid allocation leak when deleting instance stuck in BUILD https://review.opendev.org/702368 | |
| 15:31:27 | alex_xu | melwitt: I'm ok with spec, just see bauzas whether want to look at that again | |
| 15:31:42 | bauzas | alex_xu: the unified limits one ? | |
| 15:31:44 | melwitt | alex_xu: cool, thanks | |
| 15:31:45 | bauzas | I can take a look | |
| 15:32:09 | alex_xu | np | |
| 15:42:31 | efried | alex_xu, johnthetubaguy: and also stephenfin: bauzas: gibi: Would you please have a look at https://review.opendev.org/#/c/580336 (delete on termination) and see if it's ready to +A today? | |
| 15:43:50 | bauzas | efried: /me opens a tab | |
| 15:44:18 | efried | sean-k-mooney, bauzas: I'm looking at the NUMA RP spec again right now. Based on comments I skimmed on my phone, I think we might have missed the boat on the `False` thing. But checking... | |
| 15:44:53 | bauzas | efried: when we discussed this with stephenfin, we agreed on splitting between non-NUMA and NUMA-aware nodes | |
| 15:45:13 | bauzas | so, if you say 'no' for a node, then you shouldn't get NUMA-aware instances | |
| 15:45:52 | bauzas | it's like 'no, I don't want to get a beer', but then the bar tender gives you one | |
| 15:46:08 | efried | bauzas: exactly, and there was a design hole for that in PS21. | |
| 15:46:49 | gibi | efried: ack | |
| 15:48:12 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Support re-configure deleted_on_termination in server https://review.opendev.org/580336 | |
| 15:50:13 | bauzas | efried: ok, look then at the new revision and tell me then if you see some hole | |
| 15:50:19 | efried | bauzas: now I think I understand that you got rid of the fallback query completely, and on purpose. But that doesn't work. The upgrade issue that gibi and stephenfin identified still exists. | |
| 15:50:42 | efried | bauzas: Specifically: without the 'fallback query', NUMA-aware flavors can't land on `None` hosts. | |
| 15:51:06 | efried | But *with* the fallback query, NUMA-aware flavors will *also* (incorrectly) land on `False` hosts. | |
| 15:51:08 | bauzas | efried: ah, possibly | |
| 15:51:14 | bauzas | shit | |
| 15:51:55 | efried | So we need a) the fallback query, and b) some other design element to fix that second problem. | |
| 15:52:08 | efried | sean-k-mooney's suggestion of using an aggregate... that will work, but it seems like a big hammer to me. | |
| 15:52:16 | bauzas | or using another trait then | |
| 15:52:22 | efried | I think I would prefer marking `False` hosts with a `NO_NUMA_HERE` trait... yes | |
| 15:52:28 | bauzas | like HW_NON_NUMA | |
| 15:52:40 | bauzas | ok, I think we can do this | |
| 15:52:58 | efried | It's not pretty. But it's also *temporary*. Once all computes are upgraded to V where the conf is mandatory, we can get rid of that, because we also get rid of the fallback query. | |
| 15:53:16 | efried | please be sure to mention ^ that. | |
| 15:53:22 | efried | sean-k-mooney: does that work for you? | |
| 15:53:32 | bauzas | a trait for which nodes then ? | |