Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-12-07
07:05:39 openstackgerrit MIKI Nobuhiro proposed openstack/nova master: libvirt: add IP address to libvirt guest metadata https://review.opendev.org/c/openstack/nova/+/750552
07:47:48 openstackgerrit Brin Zhang proposed openstack/nova master: Replace all_tenants with all_projects in List Server APIs https://review.opendev.org/c/openstack/nova/+/765311
07:53:27 openstackgerrit Brin Zhang proposed openstack/nova master: Replace tenants* with projects* of policies https://review.opendev.org/c/openstack/nova/+/765315
08:42:09 openstackgerrit Balazs Gibizer proposed openstack/nova master: Make remove allocation symmetric with add allocation https://review.opendev.org/c/openstack/nova/+/757110
08:43:37 openstackgerrit Balazs Gibizer proposed openstack/nova master: Factor out port resource deallocation https://review.opendev.org/c/openstack/nova/+/757111
08:45:10 openstackgerrit Balazs Gibizer proposed openstack/nova master: Refactor update_pci_request_spec_with_allocated_interface_name https://review.opendev.org/c/openstack/nova/+/757614
08:46:50 openstackgerrit Balazs Gibizer proposed openstack/nova master: Support interface attach with qos ports https://review.opendev.org/c/openstack/nova/+/756530
09:30:32 openstackgerrit Wenping Song proposed openstack/nova-specs master: libvirt supports composing cyborg owned vGPU accelerator into domain XML https://review.opendev.org/c/openstack/nova-specs/+/750116
09:35:32 lyarwood \o morning all
09:35:32 kashyap chengsheng: Do you mean the unit test failure is unrelated to your patch? If so, sometimes the CI hits issues that are not your fault
09:36:47 kashyap chengsheng: If you can post the failure here, others might recognize it. Alternatively, there might be intermittent failures; so you try to type "recheck" (without quotes) in a comment in your change, and see if that helps.
09:38:10 openstackgerrit Lee Yarwood proposed openstack/nova stable/victoria: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765769
09:38:47 openstackgerrit Lee Yarwood proposed openstack/nova stable/ussuri: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765770
09:39:15 openstackgerrit Lee Yarwood proposed openstack/nova stable/train: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765771
09:40:07 openstackgerrit Lee Yarwood proposed openstack/nova stable/stein: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765772
09:40:32 openstackgerrit Lee Yarwood proposed openstack/nova stable/rocky: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765773
09:41:07 openstackgerrit Lee Yarwood proposed openstack/nova stable/queens: libvirt: Skip encryption metadata lookups if secret already exists on host https://review.opendev.org/c/openstack/nova/+/765774
09:43:22 openstack bug 1899649 in OpenStack Compute (nova) "Volume marked as available after a failure to build" [Undecided,In progress] https://launchpad.net/bugs/1899649 - Assigned to Lee Yarwood (lyarwood)
09:43:22 openstackgerrit Lee Yarwood proposed openstack/nova stable/victoria: Add regression test for bug #1899649 https://review.opendev.org/c/openstack/nova/+/764611
09:43:24 openstackgerrit Lee Yarwood proposed openstack/nova stable/victoria: compute: Don't detach volumes when RescheduledException raised without retry https://review.opendev.org/c/openstack/nova/+/764612
09:50:19 openstackgerrit chengsheng proposed openstack/nova master: CPU selection with hypervisor consideration https://review.opendev.org/c/openstack/nova/+/762330
09:54:27 gibi aarents: I have a question in https://review.opendev.org/c/openstack/nova/+/765561
09:55:06 gibi lyarwood: welcome back o/
10:00:06 openstackgerrit Aleksey Ulyanov proposed openstack/nova master: Fix graceful shutdown timeout https://review.opendev.org/c/openstack/nova/+/764927
10:22:08 aarents Hi gibi lyarwood
10:22:13 aarents gibi: I just replied
10:22:17 gibi looking
10:26:13 gibi aarents: so resize depends on image_base_image_ref being present in metadata and some instances doesn't have it as the previous fix only fixed it for new instances
10:26:17 gibi Am I correct?
10:26:42 aarents gibi: correct
10:27:44 gibi then I think only the comment need a bit of rewording to state this
10:27:52 gibi as from the bug link it was not totally clear
10:28:02 gibi but your review answer cleared it
10:29:54 gibi aarents: replied in the review, thanks for the clarification
10:33:23 aarents gibi: oky
10:33:39 aarents lyarwood: thnsk I will address the nit
12:03:43 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Update modernize-os-hypervisors-api spec, pt. 2 https://review.opendev.org/c/openstack/nova-specs/+/765796
12:03:44 openstackgerrit Stephen Finucane proposed openstack/nova-specs master: Update modernize-os-hypervisors-api spec, pt. 3 https://review.opendev.org/c/openstack/nova-specs/+/765797
12:11:02 openstack Launchpad bug 1907084 in tempest "test_attach_scsi_disk_with_config_drive intermittently fails at detaching volume" [Undecided,New]
12:11:02 gibi fyi I see this happening quite a lot recently on the gate https://bugs.launchpad.net/tempest/+bug/1907084
12:11:40 gibi I think it is just a race condition in the test. So I will try to propose a re-query of the volume attachemnts in tempest
12:14:27 sean-k-mooney gibi: well that or its realted to the race lyarwood was working around with the retry interval
12:15:27 gibi sean-k-mooney: do you have a link for that?
12:16:56 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/755799 was the workaround
12:17:10 sean-k-mooney the fix is to use libvirt events
12:18:17 gibi sean-k-mooney: ahh, could be related to the fact that now the race happens more than before as I guess we delayed the detach on the libvirt side
12:18:30 gibi making the bdm.destroy even more delayed
12:18:39 openstackgerrit Stephen Finucane proposed openstack/nova master: WIP: Change default policy for '/os-hypervisors' https://review.opendev.org/c/openstack/nova/+/765798
12:18:39 openstackgerrit Stephen Finucane proposed openstack/nova master: api: Drop statistics-style fields from os-hypervisors https://review.opendev.org/c/openstack/nova/+/764040
12:18:39 openstackgerrit Stephen Finucane proposed openstack/nova master: tests: Merge 'test_hypervisor_status' into 'test_hypervisors' https://review.opendev.org/c/openstack/nova/+/764039
12:18:57 sean-k-mooney well its load related, so 10 seconds was just a guess based on what we were seeing
12:19:10 sean-k-mooney but a slighly slower node could still nit it
12:19:37 sean-k-mooney it may be something else but it sounds related
12:21:35 lyarwood gmann: ^ btw, smells like an issue with https://review.opendev.org/c/openstack/tempest/+/764407
12:22:02 lyarwood I don't think this is an issue with https://review.opendev.org/c/openstack/nova/+/755799 fwiw sean-k-mooney, at least at first glance
12:22:36 lyarwood gmann: ignore that ping, the assertion is in the test itself, not cleanup.
12:23:24 sean-k-mooney that test chagne merged 9 days ago, gibi are you still seeing it after that?
12:23:51 gibi sean-k-mooney: yepp it is actively happening
12:24:00 gibi logstash link is in the bugreport
12:24:29 gibi but I now 70% confident that the cinder volume becomes available _befor_ nova deletes the BDM
12:24:35 gibi so the tempest test is simply racing on this
12:24:43 lyarwood that shouldn't be possible
12:25:25 gibi lyarwood: I think this is the relevant code path https://github.com/openstack/nova/blob/97729aabaa7768e1af2fa4a0bdda0df7ad598426/nova/compute/manager.py#L7071-L7086
12:25:49 gibi the bdm.destory() is the last thing nova does during volume detach
12:26:50 lyarwood gibi: right and that point we've already detached from the instance in libvirt and detached the volume from the instance via c-api
12:26:59 gibi yepp
12:27:21 gibi so there is a timewindow where the volume is available in cinder, but still has a BDM in nova
12:27:36 gibi hence the nova volume attachment list will show the attachment
12:27:44 lyarwood ah right
12:27:48 lyarwood it's using servers_client
12:27:53 lyarwood instead of volumes_client
12:28:07 lyarwood so we could race the bdm.destroy in the db
12:28:26 sean-k-mooney for what its worth that might also be an issue with shift on stack
12:28:52 sean-k-mooney they were reporting similar behavior where the volumes were atached in nova but not in cinder
12:29:32 sean-k-mooney although in there case i think that was a persitent state
12:29:40 sean-k-mooney not just an interval where it was racy
12:30:30 gibi sean-k-mooney: we will see if this is a persistent bug after I propose a re-query in the tempest test
12:30:44 gibi if the re-query fails then we know we have a problem where the bdm is not deleted
12:31:00 lyarwood gibi: the way to fix this is to add a waiter for volume attachments to the servers_clienty
12:31:02 lyarwood client*
12:31:18 gibi lyarwood: yes
12:31:20 sean-k-mooney i prefer clienty
12:31:21 gibi that is my plan
12:31:44 lyarwood cool, let me know when you've posted a review and I'll happily review
12:31:45 sean-k-mooney espcially the clinety thingy
12:31:51 gibi sean-k-mooney: :)
12:31:55 gibi lyarwood: ack
12:33:52 lyarwood urgh
12:33:54 lyarwood I see
12:54:06 openstackgerrit Lee Yarwood proposed openstack/nova master: tools: Allow check-cherry-picks.sh to be disabled by an env var https://review.opendev.org/c/openstack/nova/+/765802
12:54:36 lyarwood ^ workaround for our downstream CI so we can use jobs and tox envs directly from upstream for a change
13:02:56 sean-k-mooney instead of just using sed to remove it
13:03:11 sean-k-mooney that is cleaner then our current hack i guess
13:14:40 openstack Launchpad bug 1751784 in OpenStack Compute (nova) "Add more LOG.debug in nova.virt.hardware" [Wishlist,Confirmed] - Assigned to Matteo Sposato (teoobo)
13:14:40 teoobo_ gibi: I am working on this bug (https://bugs.launchpad.net/nova/+bug/1751784) but I cannot reproduce it in my devstack
13:14:50 teoobo_ gibi: I have already filled out the form to request a mentor, but in the meantime I would like to continue working on it
13:21:12 openstackgerrit Lee Yarwood proposed openstack/nova master: [WIP] Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/c/openstack/nova/+/742056
13:28:56 gibi teoobo_: on a call, I will get back to you in 30 minutes
13:42:39 lyarwood `ERROR: No matching distribution found for packaging>=20.4` fun
13:50:28 gibi lyarwood: hm, pypi has packaging-20.4-py2.py3-none-any.whl

Earlier   Later