Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-12-04
16:03:25 stephenfin nope, they're dead
16:03:38 stephenfin all DB model driven
16:05:36 mriedem yeah, i mean the -1 is a lie because we're not enforcing those. we did deprecate them from the apis in 2.35 and 2.49 and they were a lie before if using neutron anyway - you could change security group rules quota to 500 in nova and it wouldn't apply to neutron
16:06:15 melwitt re: the noop quota driver, I had thought that was the way for a deployment that doesn't care about quotas to bypass all the calculating (perf impact). so I was thinking not to remove it because of that. am I missing something?
16:06:45 mriedem how many deployments besides nfv single tenant don't care about quota?
16:08:19 stephenfin If -1 is a lie, what do I return instead? Just 0?
16:08:35 stephenfin I have to return _something_
16:10:45 mriedem it's all a lie is my point i think, so -1 is probably good enough
16:11:11 mriedem i'm just not sure how clearly i'm thinking about this atm, or what alternatives there are
16:11:31 mriedem i defer to dansmith
16:11:36 mriedem o-)
16:12:05 dansmith I don't know about quota things
16:12:09 dansmith I defer to melwitt
16:15:08 melwitt I'll take a look at it
16:15:52 salmankhan Anyone??? HI guys, I am facing an issue if someone can help. I upgraded the nova queens api from xenial to bionic and I am not able to see network info in "nova show" or "openstack server show" commands. However I can see the network details on dashboard or "nova interface-list" or "openstack port show" commands. Doesn't look like a database issue, but may be. If I reboot a instance there is no interface attached as well. Wha
16:15:52 salmankhan fine.
16:17:35 mriedem the instance network info cache is probably missing
16:17:58 salmankhan how can it be fixed
16:18:02 mriedem check the instance_info_caches.network_info column for that instance
16:18:26 mriedem on queens....i'm not sure
16:18:32 mriedem there is a periodic that would heal it starting in i think stein
16:18:49 mriedem see https://review.opendev.org/#/c/679274/
16:20:26 mriedem you could also take a look at this https://review.opendev.org/#/c/640516/
16:20:55 mriedem "If someone comes forward saying "yes this fixes a thing I'm hitting" then I'd be willing to restore it."
16:21:38 mriedem that second patch won't help you on queens since it relies on the force_refresh kwarg from the previous patch
16:21:53 sean-k-mooney mriedem: for what its worth we have backported only the periodic heal downstream
16:22:06 mriedem super
16:22:09 sean-k-mooney we did not backport the data migration
16:22:18 sean-k-mooney because that caused other issues
16:22:19 mriedem and canonical is going to as well it looks like
16:23:02 sean-k-mooney we have it all the way back to newton at this point
16:23:31 sean-k-mooney although we only did newton in the last month
16:23:34 salmankhan So you saying apply this 679274, that could fix on queens?
16:24:18 sean-k-mooney only if you have fixed teh underlying network issue
16:24:51 sean-k-mooney this only happens if there was an external failure that casued the cache to be curropted
16:25:43 salmankhan I messed up with neutron ports policy and that probably corrupted the cache, I think.
16:25:57 sean-k-mooney yep that is one of the know ways to cause it
16:25:57 salmankhan though I have fixed that policy
16:26:27 sean-k-mooney if the neutron policy.json is miss configured an empty list will be returned form neutron
16:26:33 sean-k-mooney and then we update the caceh with that
16:26:50 sean-k-mooney if that has been fixed the back can fix that
16:27:11 salmankhan ok, :finger_crossed: going to apply 679274 and see.
16:32:55 stephenfin dansmith: Any idea what could cause this test to fail? https://zuul.opendev.org/t/openstack/build/c598788184854780b2494a45b42a9718/console
16:33:07 stephenfin (from https://review.opendev.org/697156)
16:33:58 dansmith mucking around with the registry
16:34:01 dansmith oh
16:34:14 stephenfin I thought maybe I'd removed an object that was referenced from another object, but it doesn't seem like I have
16:34:22 dansmith stephenfin: did you maybe remove or not register an object?
16:34:39 stephenfin I removed objects, yes
16:34:47 stephenfin But I've removed all references to those objects, fwict
16:34:48 dansmith did you remove one that you didn't remove from the test_objects list of hashes?
16:35:44 dansmith unrelated to your failure, did we talk about removing objects already?
16:35:48 stephenfin grep doesn't bring up any matches for the removed objects in 'nova/tests/unit/objects/'
16:36:18 stephenfin I think so. You've to keep the DB models if it's persistent for a cycle or two, but the objects can go
16:36:21 stephenfin I think...
16:37:53 dansmith so, removing these objects will mean that if anything sends us one over the wire, we will completely fail,
16:38:08 dansmith and obviously things stop being able to run remotable methods if they're not upgraded
16:38:55 stephenfin Then I will triple check, but I think the only thing that could have sent these over the wire was nova-network, and that has been non-functional for at least a cycle now
16:39:13 dansmith stephenfin: no, I think api would send them in some cases wouldn't it?
16:39:44 dansmith note I'm not saying we should keep them necessarily, just trying to think aloud
16:40:02 stephenfin yup, I get ya. Checking now myself
16:41:41 stephenfin Yeah, leave it with me. I'll check my queue of patches and see if any of those objects could get through to the APIs. I'll state whatever I find in the commit message
16:56:45 openstackgerrit Matt Riedemann proposed openstack/nova master: ironic: report a custom trait for the node owner https://review.opendev.org/697184
16:56:45 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add node owner pre-filter https://review.opendev.org/697187
16:56:47 mriedem efried: i wrote a functional test for ^ and it shows what i suspected - traits aren't going to work
16:56:49 mriedem not enough context
16:57:11 efried mriedem: yeah, I was noodling that yesterday and I concur it would be pretty tough to do.
16:57:50 efried mriedem: If you had a I_AM_IRONIC trait, and Placement had a ?required_if=CUSTOM_OWNER_XXX:I_AM_IRONIC syntax....
16:58:19 mriedem and bingo bango you have metadata in placement :)
17:05:41 sean-k-mooney im pretty sure when i raised the idea of key:value traits in the past it was shot down pretty hard
17:10:59 dansmith please god no
17:11:47 sean-k-mooney yep that was the reaction back then too
17:18:57 efried I was not seriously proposing that.
17:19:13 efried forgot my /s
17:26:49 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add NodeOwnerFilter https://review.opendev.org/697331
18:51:16 salmankhan sean-k-mooney: Thanks mate! This fix worked for me https://review.opendev.org/#/c/679274/
18:58:35 eandersson Off topic but who do you poke to get Debian packages updated?
18:59:10 frickler eandersson: zigo
18:59:21 eandersson Thanks frickler
19:13:48 openstackgerrit Eric Fried proposed openstack/nova-specs master: Spec: Ussuri: Encrypted Emulated Virtual TPM https://review.opendev.org/686804
19:14:12 efried gibi, stephenfin, sean-k-mooney: Thanks for those reviews. Hopefully addressed everything ^
19:19:54 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add test coverage of existing admin_actions policies https://review.opendev.org/657698
19:31:51 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Add NodeOwnerFilter https://review.opendev.org/697331
19:35:02 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Introduce scope_types in Admin Actions https://review.opendev.org/657823
21:04:04 openstackgerrit Merged openstack/nova stable/pike: Hook resource_tracker to remove stale node information https://review.opendev.org/676461
#openstack-nova - 2019-12-05
00:11:49 openstackgerrit Merged openstack/nova master: vmware: Remove vestigial nova-network support https://review.opendev.org/696503
00:21:35 openstackgerrit sean mooney proposed openstack/nova master: Block rebuild when NUMA topology changed https://review.opendev.org/687957
00:21:35 openstackgerrit sean mooney proposed openstack/nova master: Disable NUMATopologyFilter on rebuild https://review.opendev.org/689861
00:30:01 openstackgerrit Merged openstack/nova master: zvm: Remove vestigial nova-network support https://review.opendev.org/696504
01:24:31 openstackgerrit Merged openstack/nova master: Cache security group driver https://review.opendev.org/697122
01:36:43 openstackgerrit jichenjc proposed openstack/nova master: libvirt: avoid cpu check at s390x arch https://review.opendev.org/696228
01:51:03 openstackgerrit melanie witt proposed openstack/nova stable/queens: delete consumers which no longer have allocations https://review.opendev.org/697398
04:19:58 eandersson mnaser what version of RabbitMQ are you running now?
06:47:23 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.opendev.org/694717
07:07:49 eandersson I think I figured out the main issue we were having with instances getting stuck in BUILD. Many of our RabbitMQ Clusters got stuck into a bad state and because of how our scheduling works instances kept getting scheduled to the same computes. These computes had bad queues, but because it's almost impossible to know which compute(s) are affected.
07:08:40 eandersson Because the database is not yet updated to reflect the compute hosting the instance (or at least it's not easy, or obvious to find the compute without digging deep into the database)
07:09:16 eandersson I am still baffled on why so many RabbitMQ Clusters broke within a short period of time. Seems to be caused by some sort of memory leak on one of the RabbitMQ servers.
07:10:44 eandersson But we also saw it happen on one cluster after a network partition.
07:12:46 eandersson The only thing we could really find on RabbitMQ was messages stuck un'acked (even without consumers), but unsure if that was just a display issue.
08:11:14 openstackgerrit Luyao Zhong proposed openstack/nova-specs master: support live migration with virtual persistent memory https://review.opendev.org/695863

Earlier   Later