Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-05-03
16:56:42 sean-k-mooney :)
16:57:06 sean-k-mooney so back to https://review.opendev.org/c/openstack/nova/+/840260/2/nova/context.py
16:57:22 sean-k-mooney im not sure if a copy actuly breaks the reference ot the stack frame
16:57:43 sean-k-mooney oh its not a copy
16:57:55 sean-k-mooney we are constucting a new instace of the excpetion class
16:58:03 sean-k-mooney wiht the arges for the orgianl excption
16:58:10 sean-k-mooney so ya that would have broken the reference
16:58:23 melwitt yeah I used the word "copy" to mean a new object with the same stuff. sorry if that made it confusing
16:59:00 sean-k-mooney no your fine i just assumed you ment copy.deepcopy just reading the patch now
16:59:41 melwitt sean-k-mooney: so what does that mean, that constructing the new instance is actually bad? I wanted to remove it to make my unit test easier so I was trying to find the inverse, whether it is ok to not construct the new instance
16:59:56 sean-k-mooney i dont have a link to back this up but i belive we dont have to worry about this on python 3
17:00:06 bauzas looks like a contextmanager question
17:00:12 sean-k-mooney no constucting a new instance is fine
17:00:26 sean-k-mooney and should not by needed on python3 i belive
17:00:34 sean-k-mooney it was there only for py27
17:00:43 sean-k-mooney to break the internal refernce to the stack frame
17:01:28 melwitt ok I see
17:16:00 sean-k-mooney melwitt: so https://peps.python.org/pep-0344/#open-issue-garbage-collection i think is solved by https://peps.python.org/pep-3110/#semantic-changes as proposed in https://mail.python.org/pipermail/python-3000/2007-January/005363.html
17:19:01 sean-k-mooney i was hoping to find a "this is fix in python 3.x" statement somewhere but didnt see it it fixed before 3.5 as far as i know but not sure what version exactly
17:20:47 melwitt sean-k-mooney++ thanks for those docs, love seeing the details. I had been looking at this that was linked off the earlier stackoverflow question https://stackoverflow.com/questions/1658293/why-is-there-a-need-to-explicitly-delete-the-sys-exc-info-traceback and it mentions python 3.3 using lack of a documentation warning as a hint of it being fixed
17:23:36 sean-k-mooney that would algin with https://stackoverflow.com/questions/11414894/extract-traceback-info-from-an-exception-object/14564261#14564261 which say the traceback atribute was added in 3.2
17:23:50 sean-k-mooney .*3.2.3
17:24:45 melwitt ah
17:26:53 sean-k-mooney i really like c++ because i like knowing how the features work and the pros and cons to use them. im always a little sad the the same content is much less avaibale for how does python work under the hood
17:27:26 melwitt ++
17:28:09 sean-k-mooney there are some good talks form Raymond Hettinger like https://www.youtube.com/watch?v=npw4s1QTmPg
17:28:15 sean-k-mooney on python dicts
17:28:46 sean-k-mooney and datacless but python could do with more of those
17:56:32 melwitt thanks sean-k-mooney
18:10:15 opendevreview Rico Lin proposed openstack/nova-specs master: Add IOMMU device support for libvirt driver https://review.opendev.org/c/openstack/nova-specs/+/840310
21:05:22 opendevreview melanie witt proposed openstack/nova-specs master: Amend unified limits spec to explain "API limit" enforcement https://review.opendev.org/c/openstack/nova-specs/+/829413
21:10:18 sean-k-mooney melwitt: thanks for fixign the merge conflict i ment to ping you about that last week
21:11:05 melwitt sean-k-mooney: np, thanks for looking
21:12:12 sean-k-mooney i was not aware that ye had provided unified_limits_count_pcpu_as_vcpu to bridge that gap
21:13:37 sean-k-mooney melwitt: have you tought about giving a presentation on this and perhaps provider.yaml to show how unified limists can now be used to provide quota for arbitry resouces
21:14:17 sean-k-mooney you could also use network bandwith as a tie in to neutron qos as another example
21:14:27 sean-k-mooney to demonstrate the new capablity
21:15:22 sean-k-mooney from a downstrema docs poing of view i think we will want ot have several such examples but i think it would be nice to have upstream docs examples or a presentaiton on the topic at some point
21:15:37 melwitt sean-k-mooney: yes, I have thought it would be useful to present or otherwise put together a doc/slides about it
21:16:42 melwitt yeah +1
21:43:25 opendevreview sean mooney proposed openstack/nova master: enable blocked VDPA move operations https://review.opendev.org/c/openstack/nova/+/832330
23:15:42 gmann bauzas: do you want me to use the BP old name for this or is it ok to change the BP name in LP (which match with the spec file), I think I did update name in LP too but you reverted - https://review.opendev.org/c/openstack/nova-specs/+/833165/1/specs/zed/approved/server-boot-on-specific-hypervisor-with-new-rbac.rst#b10
#openstack-nova - 2022-05-04
00:45:02 opendevreview Merged openstack/nova stable/wallaby: Invalidate provider tree when compute node disappears https://review.opendev.org/c/openstack/nova/+/811807
07:59:32 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:09:01 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:10:05 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:13:06 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:23:58 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
08:57:13 opendevreview Balazs Gibizer proposed openstack/nova master: Allow claiming PCI PF if child VF is unavailable https://review.opendev.org/c/openstack/nova/+/838555
08:57:48 gibi melwitt, sean-k-mooney: ^^ thanks for noting the missing test coverage, I extended the patch now
09:55:51 EugenMayer Hello. Today an instance just shutdown. I checked the audit log and see that it has been stopped by 'nobody' means, the user UUID is '-' - what could that mean?
10:24:07 bauzas gmann: as you prefer, I just want to make sure that both the LP name, the spec name and the link to the LP BP in the spec are the same :)
10:24:39 bauzas gmann: if you modify the LP name tho, the link in the yoga spec won't work, but that's fine
10:56:32 opendevreview Merged openstack/nova-specs master: Amend unified limits spec to explain "API limit" enforcement https://review.opendev.org/c/openstack/nova-specs/+/829413
11:11:57 opendevreview anguoming proposed openstack/nova master: Add catching InstanceNotFound exception when call live_migration_abort https://review.opendev.org/c/openstack/nova/+/840429
11:17:23 gibi fyi reported a gate failure https://bugs.launchpad.net/neutron/+bug/1971563 it is not super frequent (two hits in 14 days) but I needed a bug number to recheck ;)
11:24:56 sean-k-mooney ok so that in ml2/ovs with iptables
11:26:05 sean-k-mooney am both the dhcp agent and the l2 agent have to set porvisioning completed for the neutron server to send the event so maybe only one of those completed in time
11:27:13 gibi yeah I tagged neutron in the bug as I only see that the port state goes from ACTIVE -> ACTIVE on neutron side after nova started waiting for the vif-plug but I don't see the notification sending in the neutron server logs
11:27:35 sean-k-mooney rovisioning for port b6dc2b79-ed38-4907-86e2-bdff1c5a9b9f completed by entity L2.
11:27:54 sean-k-mooney so ya l2 agent completed wiring it up but not dhcp agent
11:29:50 sean-k-mooney May 03 16:47:04.331996 ubuntu-focal-ovh-bhs1-0029531414 neutron-dhcp-agent[90791]: INFO neutron.agent.dhcp.agent [None req-89d4fdba-e0f4-4778-b944-ed87f5102ff1 None None] DHCP configuration for ports {'b6dc2b79-ed38-4907-86e2-bdff1c5a9b9f'} is completed
11:29:59 sean-k-mooney https://zuul.opendev.org/t/openstack/build/518f8641b9a7419391b0f99f795f26bd/log/controller/logs/screen-q-dhcp.txt#4145
11:31:54 sean-k-mooney so ya the dhcp agent things it completed but we did not see the event on the neutron server marking provisioning complete by the dhcp agent so this looks like a bug in the dhcp agent
11:32:18 sean-k-mooney normally it compeltes first so perhaps there is a bug where it wont send the event if the port is already active or something like that
11:39:13 gibi sean-k-mooney: thanks for looking into this, could you please add these details to the bug
11:39:43 sean-k-mooney sure just responeing to a review comment and ill add them
11:40:03 gibi thank you
11:55:07 sean-k-mooney gibi: actully the dhcp agent has marked it as complete
11:55:17 sean-k-mooney in the neutron server log
12:02:46 sean-k-mooney gibi: neutron did send the event
12:02:48 sean-k-mooney INFO neutron.notifiers.nova [-] Nova event matching ['req-17e4fee0-ad06-4350-af09-1db0d331d6b5'] response: {'server_uuid': '3a81145d-d263-4e1d-8ec3-faf38fed34f2', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'b6dc2b79-ed38-4907-86e2-bdff1c5a9b9f', 'code': 200}
12:05:18 gibi sean-k-mooney: timing doesn't add up
12:05:37 gibi nova plugged the vif at May 03 16:48:41.237538
12:05:51 sean-k-mooney we recvie it at 16:47:22
12:06:11 sean-k-mooney ya so this is proably because of the dhcp agent race
12:06:25 gibi neutron should not send the plugged event _before_ nova plugs the vif
12:06:28 sean-k-mooney i bet we dont have the config option set in neutron
12:06:56 sean-k-mooney gibi: there was a race in neutron where it would not wait for both the dhcp and l2 agent to finish
12:07:35 sean-k-mooney it was fixed by https://review.opendev.org/c/openstack/neutron/+/766277
12:07:57 sean-k-mooney although hum
12:08:02 gibi this is not a live migration
12:08:02 sean-k-mooney that ws for live migration
12:08:07 gibi this is evacuate
12:08:16 sean-k-mooney the same could happen there
12:08:39 gibi and I still not get it. Can neutron send a vif-plugged event _before_ nova even plugs the vif via os-vif?
12:08:43 sean-k-mooney let me check if that is enabled or not
12:09:00 sean-k-mooney gibi: its simple
12:09:20 sean-k-mooney the port was active on the host we are evacuating form
12:09:36 sean-k-mooney so it thinks the l2 agent is finished doign its work
12:09:47 sean-k-mooney so when the dhcp agent responds it sends the event
12:09:54 sean-k-mooney that is proably what is happening here
12:10:06 sean-k-mooney that is what happend for live migration
12:10:15 sean-k-mooney i expect the same behavior for evacuate
12:11:44 sean-k-mooney gibi: https://zuul.opendev.org/t/openstack/build/518f8641b9a7419391b0f99f795f26bd/log/controller/logs/etc/neutron/neutron_conf.txt#1322-1334
12:11:47 sean-k-mooney its disabled
12:11:57 sean-k-mooney we should try enabling that and see if it fixes the problem
12:12:59 sean-k-mooney gibi: the fix in neutron is based on the presence of migratin_to in the port profile
12:13:22 gibi I thought the expected sequence would be: 1) nova bounds the port to the target host 2) nova plugs the vif on the target host 3) neutron agents plugs the other end of the vif on the target host 4) neutron sends the vif-plugged event to nova
12:14:32 gibi but based on what you said 3) and 4) happens before 2)

Earlier   Later