Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-05-03
16:50:18 melwitt yeah exactly
16:50:51 sean-k-mooney excpetion objects nolonger keep the stack frame alive right via the traceback object
16:51:13 sean-k-mooney so we dont have to worry about it "leaking" memory like it did on python 2
16:51:34 dansmith https://stackoverflow.com/questions/44681681/what-could-prevent-a-traceback-from-being-garbage-collected
16:51:41 dansmith melwitt: see the last comment in there about py3
16:52:07 dansmith sean-k-mooney: yeah, that's what I was thinking... the old py2 stuff about things being special there
16:53:00 sean-k-mooney ya so no i think the traceback is generated at the time its raised with a weakref to the stack fram to avoid that issue on py3
16:53:00 melwitt oh interesting
16:53:28 sean-k-mooney melwitt: we had a case where we were returning raided excations if i rememebr correctly in nova that caused issue in the past
16:53:46 sean-k-mooney possible in relation to the scater gather implementions
16:53:49 sean-k-mooney its been a while
16:54:13 melwitt hrm... yeah that is where the code in question is
16:54:43 dansmith well, it's really anywhere we're persisting the exception outside the handler,
16:54:51 melwitt if we need it I definitely want to put a code comment there explaining why it's needed
16:54:53 dansmith so there might be a thing related to handling in rpc too, but yeah
16:55:10 sean-k-mooney dansmith: righ tbut in that case we wanted to do that to allow the other request ot proceed and deal with it at the end
16:55:25 dansmith in the scatter/gather case yeah
16:55:28 sean-k-mooney we normally dont have a good usecase to persist the object
16:56:03 dansmith right, in the rpc case we do end up raising out of there, you're right
16:56:20 dansmith I hedged with "might" so.. :P
16:56:42 sean-k-mooney :)
16:57:06 sean-k-mooney so back to https://review.opendev.org/c/openstack/nova/+/840260/2/nova/context.py
16:57:22 sean-k-mooney im not sure if a copy actuly breaks the reference ot the stack frame
16:57:43 sean-k-mooney oh its not a copy
16:57:55 sean-k-mooney we are constucting a new instace of the excpetion class
16:58:03 sean-k-mooney wiht the arges for the orgianl excption
16:58:10 sean-k-mooney so ya that would have broken the reference
16:58:23 melwitt yeah I used the word "copy" to mean a new object with the same stuff. sorry if that made it confusing
16:59:00 sean-k-mooney no your fine i just assumed you ment copy.deepcopy just reading the patch now
16:59:41 melwitt sean-k-mooney: so what does that mean, that constructing the new instance is actually bad? I wanted to remove it to make my unit test easier so I was trying to find the inverse, whether it is ok to not construct the new instance
16:59:56 sean-k-mooney i dont have a link to back this up but i belive we dont have to worry about this on python 3
17:00:06 bauzas looks like a contextmanager question
17:00:12 sean-k-mooney no constucting a new instance is fine
17:00:26 sean-k-mooney and should not by needed on python3 i belive
17:00:34 sean-k-mooney it was there only for py27
17:00:43 sean-k-mooney to break the internal refernce to the stack frame
17:01:28 melwitt ok I see
17:16:00 sean-k-mooney melwitt: so https://peps.python.org/pep-0344/#open-issue-garbage-collection i think is solved by https://peps.python.org/pep-3110/#semantic-changes as proposed in https://mail.python.org/pipermail/python-3000/2007-January/005363.html
17:19:01 sean-k-mooney i was hoping to find a "this is fix in python 3.x" statement somewhere but didnt see it it fixed before 3.5 as far as i know but not sure what version exactly
17:20:47 melwitt sean-k-mooney++ thanks for those docs, love seeing the details. I had been looking at this that was linked off the earlier stackoverflow question https://stackoverflow.com/questions/1658293/why-is-there-a-need-to-explicitly-delete-the-sys-exc-info-traceback and it mentions python 3.3 using lack of a documentation warning as a hint of it being fixed
17:23:36 sean-k-mooney that would algin with https://stackoverflow.com/questions/11414894/extract-traceback-info-from-an-exception-object/14564261#14564261 which say the traceback atribute was added in 3.2
17:23:50 sean-k-mooney .*3.2.3
17:24:45 melwitt ah
17:26:53 sean-k-mooney i really like c++ because i like knowing how the features work and the pros and cons to use them. im always a little sad the the same content is much less avaibale for how does python work under the hood
17:27:26 melwitt ++
17:28:09 sean-k-mooney there are some good talks form Raymond Hettinger like https://www.youtube.com/watch?v=npw4s1QTmPg
17:28:15 sean-k-mooney on python dicts
17:28:46 sean-k-mooney and datacless but python could do with more of those
17:56:32 melwitt thanks sean-k-mooney
18:10:15 opendevreview Rico Lin proposed openstack/nova-specs master: Add IOMMU device support for libvirt driver https://review.opendev.org/c/openstack/nova-specs/+/840310
21:05:22 opendevreview melanie witt proposed openstack/nova-specs master: Amend unified limits spec to explain "API limit" enforcement https://review.opendev.org/c/openstack/nova-specs/+/829413
21:10:18 sean-k-mooney melwitt: thanks for fixign the merge conflict i ment to ping you about that last week
21:11:05 melwitt sean-k-mooney: np, thanks for looking
21:12:12 sean-k-mooney i was not aware that ye had provided unified_limits_count_pcpu_as_vcpu to bridge that gap
21:13:37 sean-k-mooney melwitt: have you tought about giving a presentation on this and perhaps provider.yaml to show how unified limists can now be used to provide quota for arbitry resouces
21:14:17 sean-k-mooney you could also use network bandwith as a tie in to neutron qos as another example
21:14:27 sean-k-mooney to demonstrate the new capablity
21:15:22 sean-k-mooney from a downstrema docs poing of view i think we will want ot have several such examples but i think it would be nice to have upstream docs examples or a presentaiton on the topic at some point
21:15:37 melwitt sean-k-mooney: yes, I have thought it would be useful to present or otherwise put together a doc/slides about it
21:16:42 melwitt yeah +1
21:43:25 opendevreview sean mooney proposed openstack/nova master: enable blocked VDPA move operations https://review.opendev.org/c/openstack/nova/+/832330
23:15:42 gmann bauzas: do you want me to use the BP old name for this or is it ok to change the BP name in LP (which match with the spec file), I think I did update name in LP too but you reverted - https://review.opendev.org/c/openstack/nova-specs/+/833165/1/specs/zed/approved/server-boot-on-specific-hypervisor-with-new-rbac.rst#b10
#openstack-nova - 2022-05-04
00:45:02 opendevreview Merged openstack/nova stable/wallaby: Invalidate provider tree when compute node disappears https://review.opendev.org/c/openstack/nova/+/811807
07:59:32 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:09:01 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:10:05 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:13:06 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:23:58 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:57:13 opendevreview Balazs Gibizer proposed openstack/nova master: Allow claiming PCI PF if child VF is unavailable https://review.opendev.org/c/openstack/nova/+/838555
08:57:48 gibi melwitt, sean-k-mooney: ^^ thanks for noting the missing test coverage, I extended the patch now
09:55:51 EugenMayer Hello. Today an instance just shutdown. I checked the audit log and see that it has been stopped by 'nobody' means, the user UUID is '-' - what could that mean?
10:24:07 bauzas gmann: as you prefer, I just want to make sure that both the LP name, the spec name and the link to the LP BP in the spec are the same :)
10:24:39 bauzas gmann: if you modify the LP name tho, the link in the yoga spec won't work, but that's fine
10:56:32 opendevreview Merged openstack/nova-specs master: Amend unified limits spec to explain "API limit" enforcement https://review.opendev.org/c/openstack/nova-specs/+/829413
11:11:57 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
11:17:23 gibi fyi reported a gate failure https://bugs.launchpad.net/neutron/+bug/1971563 it is not super frequent (two hits in 14 days) but I needed a bug number to recheck ;)
11:24:56 sean-k-mooney ok so that in ml2/ovs with iptables
11:26:05 sean-k-mooney am both the dhcp agent and the l2 agent have to set porvisioning completed for the neutron server to send the event so maybe only one of those completed in time
11:27:13 gibi yeah I tagged neutron in the bug as I only see that the port state goes from ACTIVE -> ACTIVE on neutron side after nova started waiting for the vif-plug but I don't see the notification sending in the neutron server logs
11:27:35 sean-k-mooney rovisioning for port b6dc2b79-ed38-4907-86e2-bdff1c5a9b9f completed by entity L2.
11:27:54 sean-k-mooney so ya l2 agent completed wiring it up but not dhcp agent
11:29:50 sean-k-mooney May 03 16:47:04.331996 ubuntu-focal-ovh-bhs1-0029531414 neutron-dhcp-agent[90791]: INFO neutron.agent.dhcp.agent [None req-89d4fdba-e0f4-4778-b944-ed87f5102ff1 None None] DHCP configuration for ports {'b6dc2b79-ed38-4907-86e2-bdff1c5a9b9f'} is completed
11:29:59 sean-k-mooney https://zuul.opendev.org/t/openstack/build/518f8641b9a7419391b0f99f795f26bd/log/controller/logs/screen-q-dhcp.txt#4145
11:31:54 sean-k-mooney so ya the dhcp agent things it completed but we did not see the event on the neutron server marking provisioning complete by the dhcp agent so this looks like a bug in the dhcp agent
11:32:18 sean-k-mooney normally it compeltes first so perhaps there is a bug where it wont send the event if the port is already active or something like that
11:39:13 gibi sean-k-mooney: thanks for looking into this, could you please add these details to the bug
11:39:43 sean-k-mooney sure just responeing to a review comment and ill add them
11:40:03 gibi thank you
11:55:07 sean-k-mooney gibi: actully the dhcp agent has marked it as complete
11:55:17 sean-k-mooney in the neutron server log
12:02:46 sean-k-mooney gibi: neutron did send the event
12:02:48 sean-k-mooney INFO neutron.notifiers.nova [-] Nova event matching ['req-17e4fee0-ad06-4350-af09-1db0d331d6b5'] response: {'server_uuid': '3a81145d-d263-4e1d-8ec3-faf38fed34f2', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b6dc2b79-ed38-4907-86e2-bdff1c5a9b9f', 'code': 200}
12:05:18 gibi sean-k-mooney: timing doesn't add up
12:05:37 gibi nova plugged the vif at May 03 16:48:41.237538
12:05:51 sean-k-mooney we recvie it at 16:47:22
12:06:11 sean-k-mooney ya so this is proably because of the dhcp agent race
12:06:25 gibi neutron should not send the plugged event _before_ nova plugs the vif
12:06:28 sean-k-mooney i bet we dont have the config option set in neutron
12:06:56 sean-k-mooney gibi: there was a race in neutron where it would not wait for both the dhcp and l2 agent to finish
12:07:35 sean-k-mooney it was fixed by https://review.opendev.org/c/openstack/neutron/+/766277

Earlier   Later