Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-04-02
22:21:56 mriedem NewBruce: while you're at it, dump the libvirtd and qemu-kvm package versions for both rdo and osa rocky nodes in the bu
22:21:58 mriedem *bug
22:22:17 NewBruce (i’ve just added to the ml2_port_binding trace to the launchpad as well as a debug version of the logs)
22:25:37 openstackgerrit Chris Friesen proposed openstack/nova-specs master: Re-propose emulated virtual TPM spec to train https://review.openstack.org/649463
22:28:06 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: Test theory about bug 1822884 https://review.openstack.org/649464
22:28:07 mriedem mnaser: NewBruce: sean-k-mooney: ^ we'll find out
22:28:07 openstack bug 1822884 in OpenStack Compute (nova) "live migration fails due to port binding duplicate key entry in post_live_migrate" [Undecided,New] https://launchpad.net/bugs/1822884
22:29:06 mnaser Let’s wait and see
22:29:41 NewBruce mriedem posted version info
22:30:32 sean-k-mooney i might be able to reporduce somethign on the neutron side too if i extend https://github.com/openstack/neutron/blob/master/neutron/tests/fullstack/test_ports_rebind.py but honestly im not sure that neutorn has mulit service functional test like nova has where we use a fake message bus and run everything in the one process
22:30:46 NewBruce mriedem will test it out
22:30:56 mriedem NewBruce: our ci system will test it
22:31:05 mriedem if that's the bug, the nova-live-migration job should explode
22:31:32 mriedem NewBruce: and it does look like your libvirt/qemu versions are different between rdo and osa so i wonder if that has something to do with the event getting sent or not
22:31:44 mnaser I’ll head off for a lil bit to doze and catch up on this buffer
22:32:44 mriedem reminds me that i still have https://review.openstack.org/#/c/594527/
22:33:01 mriedem and https://review.openstack.org/#/c/594139/1
22:33:31 NewBruce yep, id better get some shut eye soon too and check back in the morning. mnaser do you still want the debug from sean-k-mooney post earlier?
22:34:13 NewBruce mriedem any value in testing the service values? / disabling binding-extended
22:34:24 NewBruce ?
22:35:29 mriedem hmm, well we have a nova-grenade-live-migration job but would need to have that running on a stable/rocky change because then one compute would be queens and one would be rocky, or we could just pin rpc to queens in a job on master...
22:35:30 sean-k-mooney i dont think neutron provides a way to disabel binding extended via config so it would need a code chagne
22:35:39 mriedem sean-k-mooney: there is no api for that?
22:35:55 mriedem i guess not https://developer.openstack.org/api-ref/network/v2/index.html#id5
22:35:59 sean-k-mooney i have looked before ill check but i dont think so
22:38:21 NewBruce i think we can run and upgrade accross all the compute nodes anyway; the issue we had with libevent seems to be isolated and easily tested for …. ok, lets see how that job comes out anyway
22:38:25 NewBruce cheers
22:38:55 mriedem posted https://review.openstack.org/649470 for the upgrade_levels/compute=queens pin simulation
22:39:14 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Update tests from fake_libvirt_util mocks https://review.openstack.org/649471
22:41:30 sean-k-mooney mriedem: we expect this to show up in both the nova-live-migration and the grenade version right?
22:42:34 sean-k-mooney actully ill check it in the morning or ill fall a sleep watching devstack
22:42:37 sean-k-mooney night o/
22:45:37 openstackgerrit Merged openstack/nova-specs master: Re-propose emulated virtual TPM spec to train https://review.openstack.org/649463
23:19:04 mriedem ooooo hot dog i've got a devstack single node env with 2 non-cell0 cells, 2 computes on cell1 and 1 on cell2
23:19:07 mriedem and it's pretty easy
23:41:53 openstackgerrit Merged openstack/nova master: libvirt: vzstorage: Use 'writeback' QEMU cache mode https://review.openstack.org/643376
23:42:04 openstackgerrit Merged openstack/nova master: libvirt: smbfs: Use 'writeback' QEMU cache mode https://review.openstack.org/643377
23:42:13 openstackgerrit Merged openstack/nova master: Fix comment in test_attach_with_multiattach_fails_not_available https://review.openstack.org/649440
23:47:48 mriedem heh just ran into bug 1781286 again
23:47:49 openstack bug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during reschedule - get_host_availability_zone up-call" [Medium,Triaged] https://launchpad.net/bugs/1781286
23:47:54 mriedem we should maybe think about fixing that...
23:48:21 mriedem also, if things fail during server create rescheduling in conductor, chances are pretty good we don't set the instance to error status and it's stuck in build status
#openstack-nova - 2019-04-03
00:16:52 openstackgerrit Merged openstack/nova master: Fix bug preventing forbidden traits from working https://review.openstack.org/648653
00:17:31 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add a live migration regression test https://review.openstack.org/641200
00:18:41 mriedem oh wow this is pretty bad https://review.openstack.org/#/c/648653/
00:19:09 mriedem efried: so forbidden traits just never worked in nova? ^
01:13:21 openstackgerrit Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765
01:13:21 openstackgerrit Yongli He proposed openstack/nova master: Clean up orphan instances virt driver https://review.openstack.org/648912
03:04:58 openstackgerrit Yongli He proposed openstack/nova master: Clean up orphan instances virt driver https://review.openstack.org/648912
03:04:59 openstackgerrit Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765
06:39:48 openstackgerrit Merged openstack/nova master: Remove flavor id and name validation code https://review.openstack.org/638150
07:44:10 awalende I want to set the numatune memory mode to "preferred" in nova, how do I do that in Queens?
08:06:00 openstackgerrit Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765
08:11:02 openstackgerrit Kashyap Chamarthy proposed openstack/nova-specs master: Re-propose the spec to allow specifying a list of CPU models https://review.openstack.org/642030
08:16:30 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: Add node_uuid field to Destination object https://review.openstack.org/649532
08:16:31 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: Pass node uuid to new Destination.node_uuid https://review.openstack.org/649533
08:16:32 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: node_uuid from RequestSpec to ResourceRequest https://review.openstack.org/649535
08:16:32 openstackgerrit Tetsuro Nakamura proposed openstack/nova master: Add in_tree field to RequestGroup object https://review.openstack.org/649534
08:16:52 openstackgerrit Lee Yarwood proposed openstack/nova master: Block swap volume on volumes with >1 rw attachment https://review.openstack.org/572790
09:36:24 openstackgerrit Merged openstack/nova master: De-cruft compute manager live migration https://review.openstack.org/641449
10:19:06 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove unreachable codepaths https://review.openstack.org/649559
10:27:28 openstackgerrit Stephen Finucane proposed openstack/nova master: doc: Trivial fixes to API version history https://review.openstack.org/649561
10:27:28 openstackgerrit Stephen Finucane proposed openstack/nova master: doc: Group API versions by release https://review.openstack.org/649560
10:27:56 openstackgerrit Stephen Finucane proposed openstack/nova master: hacking: Fix dodgy check https://review.openstack.org/649563
10:27:56 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead code https://review.openstack.org/649562
10:27:57 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead 'ALIAS' constant https://review.openstack.org/649565
10:27:57 openstackgerrit Stephen Finucane proposed openstack/nova master: zvm: Remove dead code https://review.openstack.org/649564
10:27:58 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove unused constants, functions https://review.openstack.org/649567
10:27:58 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead placement API functions https://review.openstack.org/649566
10:27:59 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead resource tracker code https://review.openstack.org/649569
10:27:59 openstackgerrit Stephen Finucane proposed openstack/nova master: Use '_' for unused variables https://review.openstack.org/649568
10:28:00 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead nova.db functions https://review.openstack.org/649570
10:28:15 stephenfin awalende: You can't. We don't expose that
10:28:31 stephenfin awalende: It was planned but never implemented
10:39:21 openstackgerrit Kashyap Chamarthy proposed openstack/nova-specs master: Add Secure Boot support for KVM- and QEMU-based guests https://review.openstack.org/506720
10:40:56 sean-k-mooney yes just remove teh hypens
10:41:08 sean-k-mooney for kvm and qemu based guests
10:41:26 sean-k-mooney you can also remove based
10:41:49 NewBruce howdy sean-k-mooney
10:42:07 sean-k-mooney just looking at your proposea i dont se any reference to the use of the flavor extraspecs for contoling secure boot
10:42:14 sean-k-mooney NewBruce: hi
10:42:15 NewBruce was a bit like christmas last night, waiting to see how the tests would run :)
10:42:29 NewBruce shame it didn’t blow up, huh?! ;)
10:42:50 sean-k-mooney ya i had the tab open for a while but needed to go sleep
10:43:20 sean-k-mooney there are some other thing we could try like forcing different conpute level on either node
10:43:35 NewBruce yep, same - we’re gonna roll out rocky accross the full compute today and see what that does
10:44:00 kashyap sean-k-mooney: But the sentence is syntactically correct. (I read about dangling hyphens enough that the above is correct, but ugly :D)
10:44:00 NewBruce …. well, I was going to, until a net node decided to fall over due to excessive CPU usage in netlink
10:44:15 sean-k-mooney the grenade job should have been testing queens to rocky migration i think
10:44:19 NewBruce ever seen this?
10:44:20 NewBruce 2019-04-03T10:30:28.283Z|2158411|poll_loop(handler44)|INFO|wakeup due to [POLLIN] on fd 33 (unknown anon_inode:[eventpoll]) at ../lib/dpif-netlink.c:2786 (99% CPU usage)
10:44:20 NewBruce 2019-04-03T10:30:28.283Z|2158410|poll_loop(handler44)|INFO|Dropped 3426304 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
10:44:23 sean-k-mooney actuly no it would have been rocky to master
10:44:54 sean-k-mooney is that form ovs
10:45:00 NewBruce yup
10:45:11 NewBruce Absolutely nothing on the machine; freshly rebooted
10:45:18 NewBruce 1979 root 10 -10 1000172 135296 10912 S 799.3 0.2 1136:19 ovs-vswitchd
10:45:18 NewBruce PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10:45:38 kashyap sean-k-mooney: Hmm, I mentioned the metadata property (`os_secure_boot`), but didn't write anything about flavor extra_specs
10:45:39 NewBruce … ouch; seems like a FD leak perhaps

Earlier   Later