Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-12-08
18:01:44 superdan yeah
18:01:49 superdan I don't see where the cells job ran on that
18:02:00 mriedem https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2594
18:02:09 mriedem the latest patch timed out in on the cells job
18:02:23 superdan okay
18:02:31 superdan would have thought that would be reported
18:02:40 mriedem zuul in the last 48 hours...
18:02:55 superdan oh I see, it is i the comment
18:02:57 superdan but not the summary
18:03:45 alaski stepped away, but it's exactly as superdan describes
18:03:54 superdan \o/
18:04:23 mriedem so i wonder if it has something to do with this comment from comstud https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2594
18:04:55 mriedem but i think that is just about setting the task_state
18:04:59 superdan mriedem: that's a remote call, so maybe it's calling to conductor which is looking in the wrong db?
18:05:15 superdan er, a remote exception I mean
18:06:02 mriedem well,
18:06:13 superdan not sure that makes sense actually
18:06:13 mriedem i don't know why it would just fail on that one lookup and not all of the others
18:06:19 mriedem that's what i don't understand about this
18:06:57 mriedem fried_rolls: have you ever mocked something that can't be imported?
18:07:01 superdan oh Iknow why
18:07:39 superdan mriedem: backup_instance doesn't get replayed in the cell for some reason, unlike everything else, which means it calls straight to the compute I think: https://github.com/openstack/nova/blob/master/nova/cells/messaging.py#L915-L925
18:07:45 superdan mriedem: compare that to rebuild right below
18:08:01 mriedem ok yeah that's what i was looking at
18:08:27 mriedem https://github.com/openstack/nova/commit/2be50f0f3f02830798c25442dc54e92faa18106b
18:09:43 superdan I'd be extremely concerned about changing _any_ of that stuff at this point
18:09:50 mriedem weird, ok, so i guess the note about "any changes made here have to be made there" is what that's about
18:10:06 mriedem including creating the action record
18:10:14 mriedem well i see 2 options
18:10:29 mriedem 1. do the action create for backup and snapshot in the cells messaging code like it says,
18:10:46 mriedem 2. do something in the EventReporter where if cells is enabled, we blacklist some actions
18:10:55 mriedem we could remove #2 when we remove cellsv1
18:11:29 superdan I don't love either of those
18:11:40 peter-hamilton mriedem: thanks!
18:11:49 mriedem superdan: me neither
18:11:57 peter-hamilton mnaser: bpoulos is who you want to talk to; I'll ping her for you
18:12:09 mriedem superdan: but i'm not sure what else to do at this point
18:12:15 mnaser peter-hamilton: cool, thanks!
18:12:16 mriedem so, i'll go to lunch and not think about this
18:17:54 openstackgerrit Matt Riedemann proposed openstack/nova master: Don't launch guestfs in a thread pool if guestfs.debug is enabled https://review.openstack.org/526748
18:17:58 mriedem dmsimard: clarkb: ^
18:21:28 dmsimard mriedem: so we toggled the bug by setting it to debug ? :/
18:21:40 dmsimard mriedem: the force_tcg patch would work without debug ?
18:25:55 bpoulos mnaser: only nova-compute needs the verify_glance_signatures option set in order to enable glance image signature verification
18:26:27 mnaser bpoulos: awesome, thank you so much (fyi, asking in context of https://review.openstack.org/#/c/526514/ )
18:27:38 bpoulos mnaser: no problem, happy to help
18:56:42 openstackgerrit Jay Pipes proposed openstack/nova master: allow compute nodes to be associated with host agg https://review.openstack.org/526753
19:10:05 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/524795
19:16:35 openstackgerrit Merged openstack/nova master: Updated from global requirements https://review.openstack.org/526386
19:27:26 openstackgerrit Merged openstack/nova master: Fix 'force' parameter in os-quota-sets PUT schema https://review.openstack.org/523194
19:47:12 mriedem dmsimard: that's what i hope to find out
19:50:24 fried_rolls mriedem No, I don't think I have ever (successfully) mocked something that can't be imported. It will fail at "compile" time.
19:50:44 fried_rolls mriedem You can cheat by mocking it after it's imported, though.
19:52:07 fried_rolls I assume this is in a situation where the thing imports correctly on the prod side but for some reason not on the test side? Which... why?
19:53:39 fried_rice cdent How's now?
19:53:48 cdent one moment caller
19:54:34 cdent blargh, timeouts in migration tests: http://logs.openstack.org/39/521639/8/gate/openstack-tox-functional/f6880ac/testr_results.html.gz
20:02:55 cdent fried_rice: okay, sorry, had to feed that cat and put dinner dishes aweay
20:03:09 fried_rice How domestic.
20:03:40 fried_rice Should we do this here, or maybe snag an empty meeting channel so we don't consume this one?
20:04:04 figleaf stay here - I'm eavesdropping
20:04:09 fried_rice ight
20:04:19 cdent agree with figleaf, probably best to spread the word as much as possible
20:05:14 fried_rice I'll lead by saying I've got several balls in the air, including the 'granular' effort and a non-placement-related bp for ksa adapter stuff; but I'll be happy if just the nrp series gets shepherded while I'm away.
20:05:40 fried_rice That's the series starting at the hopefully-just-about-to-merge https://review.openstack.org/#/c/385693/91
20:06:18 fried_rice The culmination of this series will be the ability for the compute driver to tell the scheduler (almost) everything about how it wants to perceive its world, placement-wise.
20:07:21 cdent “the scheduler”? or “placment” or both?
20:07:33 fried_rice As currently conceived in the top patch in the series (https://review.openstack.org/#/c/520246/12/nova/compute/resource_tracker.py) the resource tracker will hand off the ProviderTree to the compute driver; the compute driver will mangle it as it sees fit; and the resource tracker will flush any changes back to placement.
20:08:12 fried_rice cdent I should have said "resource tracker" instead of "scheduler". The main point there is that the compute driver does not directly tell placement anything; that is brokered by resource tracker via ProviderTree.
20:08:20 fried_rice ...via report client.
20:08:53 cdent okay, so the virt driver tells the compute manager and its resource tracker about what it knows about its world
20:09:07 fried_rice Yup.
20:09:15 cdent ✔
20:09:58 fried_rice I think there are reasons we don't want virt talking to placement directly. leakypipes and superdan have those in their heads. But I think it has to do with narrowing the points of control.
20:10:25 leakypipes fried_rice: ya
20:10:50 leakypipes fried_rice: the scheduler reporting client is a good bright-line interface for nova-compute with the placement API.
20:11:02 fried_rice Nod.
20:11:41 cdent yeah, my concern was not so much about violating that line, rather trying to clarify the use of the term scheduler, which is ambiguous, but it’s clear now
20:11:47 fried_rice As currently written in the series, the first time around, resource tracker => report client will do get_provider_tree to bootstrap the ProviderTree, which will _ensure_resource_provider with the compute nodename & uuid. That will create the entry in placement for the compute host RP and populate the local ProviderTree with that.
20:12:10 fried_rice And then pass that ProviderTree to virt, via ComputeDriver.update_provider_tree.
20:12:35 fried_rice So it'll have just the compute RP as a root (no children), no inventory, no traits, no aggs, no sharing RPs.
20:13:26 cdent ✔
20:14:06 fried_rice virt does whatever it wants, as long as it's possible via the public ProviderTree interfaces. This includes creating new root RPs (which will probably typically be sharing RPs, though we're not enforcing that) or child RPs (usually descendants of the compute RP, though we're not enforcing that).
20:14:19 fried_rice It also includes setting traits and inventories on any of those guys.
20:14:30 fried_rice And assigning aggregate UUIDs.
20:15:16 fried_rice virt is responsible for returning True or False from that method to let us know whether it changed anything. (TBH, I'm not sure that's necessary, cause it's pretty easy for us to figure it out, and that might be better than trusting virt to get it right.)
20:16:27 fried_rice Anyway, resource tracker gets that provider tree back and diffs it with what it passed in originally (which, I should have mentioned, is a copy of what it had built up); and then makes report client calls to push those changes back to the placement db.
20:16:33 cdent can I pause you for a moment?
20:16:36 fried_rice sho
20:18:26 cdent this is making sense, mostly, so far, except for one thing. Can you give me a scenario or two in which the provider tree will have more than one root? Is this for something like ironic, of clustering hypervisor or something else or something in addition to that?
20:19:57 fried_rice I can't say what'll happen for ironic. It's possible they'll have their root be their (I'm going to get the terminology wrong, but) compute "host" with children being the compute "nodes". Or maybe the compute "nodes" will all be roots. I dunno.
20:20:02 fried_rice A real scenario for multiple roots is sharing RPs.
20:20:32 fried_rice So if I have a shared disk provider that my virt driver knows about, the compute manager won't know about it yet.
20:20:59 openstackgerrit Matt Riedemann proposed openstack/nova master: Add instance action record for backup instances https://review.openstack.org/523676
20:21:12 fried_rice virt will populate the provider tree with that shared disk provider as a new root with the MISC_SHARES_VIA_AGGREGATE trait and tag both it and the compute RP with the aggregate UUID.
20:21:33 cdent let me know when you are EOF, so I can ask my followup, but I don’t want to do it if you are stil typing away
20:21:44 fried_rice And the rt will have to handle the case where we go to create that guy in placement and find it already exists, because some other compute node that *also* shares with it already created it.
20:22:29 fried_rice By the same token, the various virts sharing a given sharing RP will have to make sure they use the same name and/or UUID (and we should probably document some rules for them doing that) so that we can in fact do that coordination.
20:22:58 fried_rice We've been saying we won't support sharing in Q, but we at least need to consider this stuff so we don't paint ourselves into a corner.
20:22:59 fried_rice EOF

Earlier   Later