Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-09-08
07:14:09 yingjisun apache logs are in apache2 folder.
07:17:00 gibi yingjisun: thanks
07:32:29 openstackgerrit Brin Zhang proposed openstack/nova master: Cyborg shelve/unshelve support https://review.opendev.org/729563
07:44:32 lyarwood kashyap: http://paste.openstack.org/show/797563/ - had another hit of the phantom DEVICE_DELETED bug in CI last night, hopefully these logs are easier to parse today
07:44:43 lyarwood kashyap: I think we should raise a bug about this today if possible
07:46:23 bauzas good morning Nova
07:46:26 bauzas review day for me
07:47:33 openstackgerrit Balazs Gibizer proposed openstack/nova master: Follow up for I67504a37b0fe2ae5da3cba2f3122d9d0e18b9481 https://review.opendev.org/750184
07:49:14 gibi bauzas: good morning! if you can hit the sriov attach series that would be terrific.
07:50:21 bauzas gibi: my first one :p
07:50:26 gibi awesome
07:50:27 bauzas you got the first ticket
07:50:57 bauzas hopefully, I know a bit better about networking :)
07:59:55 gibi bauzas: actually the patch is not that heavy about networking it is compute manager and a bit of libvirt
08:00:28 bauzas yeah but you need to know a bit about VIFs attach/detach mechanims
08:00:36 bauzas mechanisms*
08:00:59 gibi true
08:01:06 bauzas a whole new territory to discover for me
08:01:25 bauzas but I'm on it
08:03:32 bauzas gibi: question on https://review.opendev.org/#/c/741436/4
08:04:43 gibi sure, listening
08:05:52 gibi or did you forget to push the post button?!
08:06:51 bauzas meh
08:06:55 bauzas sorry was disturbed
08:06:59 gibi no worries
08:07:43 bauzas gibi: I guess deleting the metadata is harmless when you deallocate the port, since you need to rebuild it when you reattach it, right?
08:08:38 gibi bauzas: right
08:08:45 gibi it is a missed cleanup
08:08:59 bauzas that's what I read, but I wondered any side effect
08:09:12 bauzas because you update the instance
08:09:58 gibi it only deletes the metadata of the detached instance
08:10:04 gibi detached port
08:10:24 gibi before the patch this was deleted only when the instance is deleted
08:10:33 bauzas I see, hence a race
08:10:45 bauzas well, not a race, but a corner case when you had stale data
08:10:53 bauzas kk
08:11:35 gibi yeah it was stale, it did not make anything break though, maybe the guest saw the old metadata in the metadata response
08:11:58 gibi I just found it by looking at the db to see that detach cleaned up or not every relevant data
08:13:20 bauzas ack
08:13:54 kashyap lyarwood: Just connected to Freenode; sure. As discussed elsewhere, a Launchpad QEMU bug?
08:14:06 openstackgerrit Hao Sun proposed openstack/nova stable/stein: Add checks for volume status when rebuilding https://review.opendev.org/748559
08:24:37 openstackgerrit Merged openstack/nova master: functional: Don't inherit from 'ProviderUsageBaseTestCase' https://review.opendev.org/748271
08:29:22 lyarwood kashyap: https://bugs.launchpad.net/qemu/+bug/1894804
08:29:22 openstack Launchpad bug 1894804 in QEMU "Second DEVICE_DELETED event missing during virtio-blk disk device detach" [Undecided,New]
08:29:55 kashyap lyarwood: If I don't see a response, I'll nudge folks on #qemu on OFTC in a day or this evening
08:30:02 kashyap lyarwood: Given that this is somewhat more urgent
08:38:41 stephenfin bauzas: Seeing as you're doing reviews today, https://review.opendev.org/#/c/749055/ and https://review.opendev.org/#/c/749056/ are both ready to go :)
08:39:02 bauzas stephenfin: awesome, you're next in the queue
08:42:10 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Enable real policy fixture by default https://review.opendev.org/745906
08:42:11 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add and use 'GlanceFixture' https://review.opendev.org/745907
08:42:11 openstackgerrit Stephen Finucane proposed openstack/nova master: tests: Remove '_FakeImageService' https://review.opendev.org/745908
08:58:51 openstackgerrit Stephen Finucane proposed openstack/nova master: Expand generic reproducer for bug #1879878 https://review.opendev.org/750186
08:58:51 openstack bug 1879878 in OpenStack Compute (nova) "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Medium,In progress] https://launchpad.net/bugs/1879878 - Assigned to Stephen Finucane (stephenfinucane)
08:58:51 openstackgerrit Stephen Finucane proposed openstack/nova master: Set 'old_flavor', 'new_flavor' on source before resize https://review.opendev.org/750187
08:58:51 openstackgerrit Stephen Finucane proposed openstack/nova master: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934
09:26:28 yingjisun jhui: hello
09:45:59 openstackgerrit Merged openstack/nova master: Delete non tag related device metadata during detach https://review.opendev.org/741436
09:51:23 openstackgerrit Stephen Finucane proposed openstack/nova master: Introduce a new driver interface 'cleanup_instance' https://review.opendev.org/714653
09:56:38 openstackgerrit Brin Zhang proposed openstack/nova master: Cyborg shelve/unshelve support https://review.opendev.org/729563
09:56:58 kashyap lyarwood: We have air-time for the DEVICE_DELETED bug on #qemu, OFTC.
09:57:07 kashyap lyarwood: The storage backend for the guest is Ceph, yeah?
10:00:39 openstackgerrit Liang Fang proposed openstack/nova master: Add volume local cache support https://review.opendev.org/663542
10:03:48 kashyap lyarwood: If you want to read through, some chat snippet here: http://paste.openstack.org/show/797568/
10:15:16 lyarwood kashyap: yeah but this reproduces with an iSCSI block device
10:15:29 lyarwood kashyap: we've just blacklisted the tests in that job
10:16:41 kashyap lyarwood: Yeah, noted. But I think folks on #qemu will have something to run with, if we can narrow down a non-Tempest reproducer.
10:17:00 kashyap (But I've told that you were able to reproduce it outside the CI, but with Tempest)
10:18:19 lyarwood kashyap: yeah I've not been able to reproduce it outside of tempest that's a pain
10:19:15 kashyap lyarwood: I hear ya. So, you're able to reproduce with 'tox -e full'
10:19:47 kashyap lyarwood: Maybe we can go the hard route of tracing? If it's super urgent, we can set up a machine with reproducer, install QEMU w/ debuginfo
10:20:03 kashyap ... and then let the QEMU devs do use the tracing infra to probe/debug
10:22:21 lyarwood kashyap: I'm not sure how they can trace the issue given the instances are being created by tempest
10:23:28 lyarwood kashyap: would you be able to create the shared reproducer env somewhere for them?
10:23:37 kashyap lyarwood: Ah, you mean Tempest is making the instances on the fly
10:23:42 lyarwood kashyap: yeah
10:24:45 openstackgerrit Merged openstack/nova master: Remove unused vpn param from allocate_for_instance https://review.opendev.org/744933
10:24:48 kashyap lyarwood: I can try setting it up on my test machine near me. But I'm not good w/ Ubuntu; so if you have a reproducer disk image, post it somewhere :)
10:26:02 lyarwood kashyap: my previous reproducer is in sean-k-mooney's personal cloud
10:26:11 lyarwood kashyap: we could build another there I guess
10:26:20 lyarwood sean-k-mooney: ^ would that be okay?
10:26:36 lyarwood sean-k-mooney: a few QEMU devs would need access but that should be easy enough to organise
10:26:44 kashyap lyarwood: I see. Yeah; at least one
10:28:26 manuvakery hi, even though i have tls enabled on between client -> haproxy and novncproxy -> vnc server(instance). horizon console is not showing the connection as encrypted. am i misssing something here
10:32:59 lyarwood manuvakery: https://docs.openstack.org/nova/latest/admin/remote-console-access.html#vnc-proxy-security - and libvirtd has the required certs?
10:34:48 manuvakery lyarwood: yes the certificates are there .. for now am using self signed certificates
10:34:55 manuvakery 2020-09-08 10:32:23.138 12800 INFO nova.console.rfb.authvencrypt [req-7350e57c-477c-41e2-b478-f239d6df6dcf - - - - -] VeNCrypt security handshake accepted
10:34:55 manuvakery 2020-09-08 10:32:23.138 12800 INFO nova.console.securityproxy.rfb [req-7350e57c-477c-41e2-b478-f239d6df6dcf - - - - -] Finished security handshake, resuming normal proxy mode using secured socket
10:35:44 lyarwood hmmm I'm not sure then tbh
10:39:59 manuvakery ok thanks for ur reply .. the conenction between haproxy-> novncproxy is not encrypted, can that be the problem?
10:41:16 lyarwood I wouldn't think so but you should be able to prove that pretty easily by connecting directly to the proxy right?
10:44:32 manuvakery lyarwood: yeah even i don't think thats the issue. still try to connect to the proxy directly
10:49:50 openstackgerrit sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432
10:55:39 noonedeadpunk ok, I found what makes nova-compute to fail on the startup with http://paste.openstack.org/show/797571/
10:55:48 noonedeadpunk it's libvirt-python 6.7.0
10:56:05 noonedeadpunk luckily it's bumped in u-c, but I was not using that....
11:20:13 brinzhang gibi, stephenfin, sean-k-mooney: the cyborg shelve/unshelve support feature was updated, hope you can review recently, may we can merge it before feature freeze \o
11:20:24 brinzhang gibi, stephenfin, sean-k-mooney: https://review.opendev.org/#/c/729563/
11:20:48 gibi brinzhang: try to get to it today
11:21:00 brinzhang gibi: ack, thanks
11:24:32 brinzhang gibi: I was update the runway slots just now

Earlier   Later