Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-16
16:05:25 sean-k-mooney melwitt: dansmith. i have found an issue in my livemigration. specifcally there is a case where livemigration fails, the instance is still running on the soruce but the vif_type is none beacuse the host_id is not set.
16:05:52 sean-k-mooney i need to do more digging to localise the actull issue
16:06:06 dansmith because we update the instance info in the db from the failure yeah?
16:06:20 dansmith I think mriedem had a patch for that recently, IIRC
16:07:19 dansmith sean-k-mooney: does the healer periodic fix it after it runs?
16:07:39 sean-k-mooney im not sure. i think we may have activated the port binding for the dest and not reactivated for source.
16:08:28 sean-k-mooney dansmith: ill get the logs and file a propper bug
16:08:42 dansmith okay there was a situation where we saved the failed binding info cache recently that matt was working on, so might be related
16:08:46 dansmith but yeah
16:09:37 sean-k-mooney dansmith: i dont think its related as teh vif_type is none in neutron not binding_failed.
16:09:46 dansmith ah okay
16:23:35 kosamara efried: On the namespacing of driver-generated traits: if we demand that all driver-generated traits exist within a namespace, we don't allow the drivers to generate standard traits.
16:28:38 kosamara I propose to allow the driver to generate either namespaced or standard traits.
17:34:27 openstackgerrit melanie witt proposed openstack/nova-specs master: Move rocky implemented specs https://review.openstack.org/592622
17:35:48 openstackgerrit Chris Dent proposed openstack/nova master: Add explanatory prefix to post_test_perf output https://review.openstack.org/591850
17:35:49 openstackgerrit Chris Dent proposed openstack/nova master: Add trait query to placement perf check https://review.openstack.org/592624
17:42:10 openstackgerrit melanie witt proposed openstack/nova-specs master: Dynamically find releases for move-implemented-specs https://review.openstack.org/592628
17:54:38 Sundar Hi all, could somebody tell me the status of nested resource providers, including virt driver changes (to avoid stomping on custom resources)? If there is a bp to track it, please point me to that. Thanks!
17:55:57 melwitt I think the placement side is considered done, but none of the nova integration side has been started yet. I'm not aware of a bp tracking it yet
17:58:55 Sundar melwitt: Thanks. So, in particular, the work to prevent virt drivers from overwriting custom resources has not started yet, right? If so, what is the blocker -- is it still upgrade considerations?
17:59:54 dansmith Sundar: resources or traits?
18:00:38 Sundar dansmith: I think both. When last checked, the virt drivers just overwrite the inventory of the compute node.
18:00:54 dansmith well, nova-compute does that with the help of the virt driver,
18:01:00 openstackgerrit melanie witt proposed openstack/nova-specs master: Add a script for counting blueprints https://review.openstack.org/581914
18:01:09 dansmith but inventory of a provider should only be owned by a single entity,
18:01:27 sean-k-mooney Sundar: in dublin we said external service were not allowed to create inventoei in resouce providers they did not create
18:01:30 dansmith and since there is only one provider for a compute node right now, overwriting inventory entirely is kinda expected
18:01:38 dansmith right, what sean-k-mooney said
18:01:50 dansmith so until the providers start reporting things as nested,
18:02:07 dansmith there's no real opportunity for the coexistence of other nested providers with their own inventory they manage
18:02:16 efried kosamara: Yes, tricky question as to who "owns" a standard trait.
18:02:19 dansmith and that is dependent on the upgrade bit getting worked out
18:02:37 Sundar Yes. Even with nested RPs, we had the issue that the libvirt driver would overwrite the nested RP config. This was about a month or so ago.
18:03:31 dansmith Sundar: because it's not nested-aware, AFAIK
18:03:42 efried true ^
18:04:02 efried dansmith: The upgrade (reshape) series is ready for review btw.
18:04:37 dansmith yeah I know
18:04:46 efried :P
18:05:24 sean-k-mooney Sundar: cyborg will have to create a seperate resouce provider tree under the compute node. it should be possible to do that today but im not sure if the virt driver will remove them or not
18:05:29 efried kosamara: Do you understand the concern about trait "ownership" in general?
18:05:47 efried sean-k-mooney: The virt driver won't remove them; but we also won't schedule to them either.
18:06:05 sean-k-mooney efried: can you allocate from them
18:06:32 sean-k-mooney the intent would to still schduler to the compute node but consume the cyborg resouce from the cyborg subtrees
18:06:42 efried sean-k-mooney: basically, that whole code path has some bits that will work, some bits that definitely won't, and some that are still a mystery. This is work that needs to be done in Stein.
18:07:08 efried sean-k-mooney: I think the scheduler *should* consume the cyborg resources.
18:07:20 efried It's just not allowed to muck with the cyborg providers' inventories or traits.
18:07:23 Sundar sean-k-mooney: Yes, cyborg is trying to do nRPs. We hit the virt driver overwrite issue and developers decided to wait for that to be resolved.
18:07:54 sean-k-mooney efried: right. we may also have to revisit the idea of have a seperate root node just for the phyical server and then have per projec subtrees
18:08:31 sean-k-mooney Sundar: again if the overriete issue is you tried to create invtories on the comptue node resouce provider that will never be supported
18:08:33 efried revisit? I don't remember visiting that in the first place, ever. Per-project subtrees?
18:09:14 sean-k-mooney efried: it came up in dublin on the friday when we were all tired
18:09:38 Sundar sean-k-mooney: no, we created nested RPs within the compute node.
18:09:42 efried I think Sundar is talking about creating child providers of the compute node provider but having them ignored by the scheduler and/or deleted by the resource tracker. The latter may have happened in libvirt when it was still using get_inventory.
18:09:56 Sundar Yes ^
18:09:58 efried I'm guessing you haven't tried it since we implemented libvirt's update_provider_tree method.
18:10:14 Sundar efried: when was that done?
18:10:25 efried Sundar: So if you have the resources and inclination, you could try that again. I guarantee it will still break, but it may get slightly further than it did before.
18:10:29 efried Sundar: Let me find the patch...
18:11:01 sean-k-mooney efried: the idea being that nova would nolonger own the root node of the tree. the root would have no invetores and just represent teh server then each service on the serve would create there own nested resouce provider beneate that root node
18:11:18 Sundar efried: "I guarantee it will still break" -- that won't help me persuade other Cyborg developers to start moving fwd :)
18:11:26 efried Sundar: https://review.openstack.org/#/c/560444/ merged 5/25
18:11:53 sean-k-mooney efried: it was a fallback incase we did not figure out how to have multple service own different nodes under the current tree create by nova
18:11:54 efried sean-k-mooney: Nono, nova still owns the root provider. But nova does *not* own the device providers - cyborg owns those.
18:12:04 efried oh, sorry, you're talking about the project-based thing.
18:12:27 sean-k-mooney efried: ya sorry i was
18:12:58 efried sean-k-mooney: Without further placement API work, we would probably have to do that by lassoing the project subtree in an aggregate corresponding to the project ID.
18:13:26 sean-k-mooney efried: by project id you mean service id?
18:13:39 efried But regardless, that would still rely on scheduler and resource tracker implementation that we have not started yet. Same as the stuff Sundar is blocked behind.
18:13:43 sean-k-mooney when i said per-procject i ment nova, neutron, cinder not tenants
18:14:09 efried um, oh, then forget what I'm saying.
18:14:23 efried But yes, all of that needs to be thought through.
18:14:39 efried sean-k-mooney: I would think that nova would still own the root node.
18:15:12 efried Sundar: Define "moving fwd"? You can't integrate with nova yet, for sure. But I assume there's work you can do while you're waiting for enablement.
18:15:56 sean-k-mooney efried: perhaps, the idea of spliting it out was in the event that nova is not the first service deployed on the node, we won know what uuid it will get so we cant create the provider tree
18:16:35 efried somebody would have to reparent
18:16:45 efried which can be done as long as parent isn't already set.
18:17:48 sean-k-mooney efried: once that reshaper stuff is reviewed :)
18:18:23 sean-k-mooney although i guess you could manually reparent now its just not as simple
18:20:40 efried Now Nova starts. It creates the compute node RP. It also somehow discovers that the NIC and device RPs are out there, and reparents them (PUT /resource_provider/{u} with parent_provider_uuid specified in the payload) to the compute node RP.
18:20:40 efried Cyborg starts next. Creates the dev RPs. They have no parent (and aren't related to the NIC RPs).
18:20:40 efried Neutron starts first. Creates the NIC RPs. They have no parent.
18:20:40 efried actually I don't think reshaper accounts for reparenting at all. I was talking about:
18:24:11 Sundar efried: Why can't Cyborg and Neutron wait for Nova compute to create the top level first?
18:26:50 Sundar If that is simpler, that is
18:28:15 efried I don't know. Maybe they can. Ask sean-k-mooney
18:48:23 openstackgerrit Merged openstack/nova master: Py3 fix in fake image service https://review.openstack.org/592056
18:53:12 openstackgerrit Eric Fried proposed openstack/nova-specs master: Repropose reshaper spec for Stein https://review.openstack.org/592650
19:04:25 openstackgerrit Jay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg https://review.openstack.org/592654
19:26:16 openstackgerrit Eric Fried proposed openstack/nova-specs master: Repropose reshaper spec for Stein https://review.openstack.org/592650
20:50:08 melwitt nova meeting in 10 minutes
20:59:02 openstackgerrit Dan Smith proposed openstack/nova master: WIP: Batch results per cell when doing cross-cell listing https://review.openstack.org/592698
21:02:33 openstackgerrit Jay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg https://review.openstack.org/592654
21:24:13 openstackgerrit Dan Smith proposed openstack/nova master: WIP: Batch results per cell when doing cross-cell listing https://review.openstack.org/592698
22:00:51 openstackgerrit Jay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg https://review.openstack.org/592654
22:04:49 openstackgerrit Jay Pipes proposed openstack/nova master: placement: use single-shot INSERT/DELETE agg https://review.openstack.org/592654
22:16:34 openstackgerrit Eric Fried proposed openstack/nova-specs master: Delete approved template in move_implemented_specs https://review.openstack.org/592755
22:17:13 efried melwitt: In case you were busily working on this enhancement ^
22:19:59 melwitt hah, thanks
22:24:52 efried melwitt: Mind if I combine your patches for dynamically finding releases and counting blueprints into a series that commonizes the fs walk bit?
22:25:11 melwitt efried: nope, go ahead

Earlier   Later