| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-04-06 | |||
| 20:59:53 | dansmith | heh | |
| 21:00:27 | mriedem | dansmith: you're talking about this right? https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L767 | |
| 21:00:30 | mriedem | where it does the full pull | |
| 21:00:43 | dansmith | mriedem: yeah | |
| 21:00:51 | mriedem | but that's only if it doesn't already have a list for the host | |
| 21:01:02 | mriedem | https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L752 | |
| 21:01:09 | mriedem | that gets created when the compute starts up | |
| 21:01:19 | mriedem | which is what we were talking about yesterday | |
| 21:01:35 | dansmith | sure, but we're still sending it | |
| 21:01:36 | dansmith | for no reason | |
| 21:01:42 | mriedem | yes that's dumb too | |
| 21:01:43 | dansmith | we'd have to tweak the logic there just a bit, agreed | |
| 21:01:54 | mriedem | i'm just thinking https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L767 is a complete fallback | |
| 21:02:05 | mriedem | if the compute host didn't send the instances on startup | |
| 21:02:09 | mriedem | to register itself with the scheduler | |
| 21:02:15 | mriedem | err host manager | |
| 21:02:32 | mriedem | https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L761 is the compute start case | |
| 21:02:41 | mriedem | well, unless it's a new compute with no hosts | |
| 21:02:42 | dansmith | we only get here in the rpc call right? | |
| 21:02:46 | mriedem | yeah | |
| 21:02:58 | dansmith | otherwise if we need host info before it has called in, we just generate it ourself anyway | |
| 21:03:04 | mriedem | _update_scheduler_instance_info in the compute manager is called on (1) startup and (2) when a new instance is created | |
| 21:03:13 | mriedem | yeah | |
| 21:03:16 | dansmith | and in a periodic right? | |
| 21:03:23 | mriedem | no | |
| 21:03:31 | mriedem | that's _sync_scheduler_instance_info | |
| 21:03:44 | mriedem | that sends the uuids | |
| 21:04:18 | chatter29 | hey guys | |
| 21:04:24 | mriedem | dansmith: and in instance_uuids | |
| 21:04:38 | mriedem | if the set that the host manager knew about doesn't match what was passed in, it pulls them all from the db again | |
| 21:04:40 | mriedem | for that compute | |
| 21:04:47 | chatter29 | allah is doing | |
| 21:04:53 | chatter29 | sun is not doing allah is doing | |
| 21:04:55 | chatter29 | to accept Islam say that i bear witness that there is no deity worthy of worship except Allah and Muhammad peace be upon him is his slave and messenger | |
| 21:05:06 | dansmith | mriedem: okay yeah, I see | |
| 21:05:16 | dansmith | mriedem: I was conflating those two I guess | |
| 21:06:27 | dansmith | mriedem: why do we have to send the whole instance? | |
| 21:06:42 | mriedem | because the host manager caches the list by host | |
| 21:06:47 | dansmith | mriedem: we only really care about the uuids for affinity, AFAIK, but we're sending the instance a bunch of places from computemaanger | |
| 21:06:51 | mriedem | and then the affinity filter/weigher uses those | |
| 21:06:55 | dansmith | sure, but we neednly send the whole object for that | |
| 21:07:02 | dansmith | needn't | |
| 21:07:02 | mriedem | let me see how they use them | |
| 21:07:15 | mriedem | back when these were simple single-level dicts it was maybe not as big a deal | |
| 21:07:21 | mriedem | now it's a behemoth of nested goodness | |
| 21:07:35 | mriedem | maybe this is why we fucking kill ram usage in a single node devstack run :) | |
| 21:07:50 | dansmith | same number of instances regardless :) | |
| 21:08:19 | clarkb | mriedem: "remember when devstack needed 1GB of ram to run?" :) | |
| 21:08:35 | mriedem | instance group members are just instance uuids right? | |
| 21:08:59 | mriedem | yeah, so this whole big ass cache of hosts -> instances for the affinity filter/weigher JUST NEEDS UUIDS | |
| 21:09:30 | mriedem | 'members': fields.ListOfStringsField(nullable=True), | |
| 21:09:36 | mriedem | Yeah InstanceGroup.members is just instance uuids | |
| 21:09:56 | dansmith | mriedem: well, you can do single-instance affinity without a gorup | |
| 21:09:58 | dansmith | but again, | |
| 21:10:00 | dansmith | just an uuid :) | |
| 21:10:20 | mriedem | so we agree we don't need to catch the full instance object in the host manager right? | |
| 21:10:24 | mriedem | *cache | |
| 21:10:34 | dansmith | AFAIK yeah | |
| 21:10:36 | mriedem | HOWEVER | |
| 21:10:43 | mriedem | you know if we changed it to just uuids, | |
| 21:10:50 | mriedem | we'd break someone's out of tree filter/weigher | |
| 21:11:00 | mriedem | that was using this to lookup some shit on the instance object fields | |
| 21:11:44 | dansmith | EDONTCARE | |
| 21:11:58 | mriedem | like, my super kewl filter checks all instances that have instance.config_driver = True and puts them on the same host! | |
| 21:12:30 | mriedem | i guess the answer for them is, well, you take the uuid, and go back to the db to look up the full instance object | |
| 21:12:42 | cdent | yes, that | |
| 21:14:15 | mriedem | ok, well, maybe i'll spend some time tinkering with this for the rest of the day, starting with the unnecessary joins in the db api | |
| 21:14:23 | mriedem | since that shouldn't be controversial | |
| 21:14:36 | mriedem | what to name my topic branch... | |
| 21:15:36 | openstackgerrit | melanie witt proposed openstack/nova-specs master: Amend spec for placement project/user association https://review.openstack.org/454352 | |
| 21:16:00 | cdent | mriedem: I'm very interested in this stuff so if you could keep me in the loop that would be dandy and write a helacious complete commit message that would be awesome. I'm not sure I was able to extact all the good bits from you and dan talking | |
| 21:16:12 | mriedem | i'll try | |
| 21:16:21 | cdent | thanks | |
| 21:28:05 | openstackgerrit | melanie witt proposed openstack/nova-specs master: Amend spec for placement project/user association https://review.openstack.org/454352 | |
| 21:28:18 | mriedem | dansmith: https://blueprints.launchpad.net/nova/+spec/put-host-manager-instance-info-on-a-diet | |
| 21:28:20 | mriedem | sdague: ^ | |
| 21:28:22 | mriedem | johnthetubaguy: ^ | |
| 21:28:31 | mriedem | i'm going for specless blueprint just since it's a couple of changes | |
| 21:30:27 | mnaser | hi everyone, i suspect that we're running into weird race conditions in the CI of Kolla related to the order of services being started.. is there a 'recommended' start list? | |
| 21:30:39 | openstackgerrit | Mathieu Gagné proposed openstack/nova-specs master: Multiple Fixed-IPs support in network information https://review.openstack.org/312626 | |
| 21:30:47 | mriedem | mnaser: controller services first | |
| 21:30:51 | mriedem | computes at the end | |
| 21:31:02 | mnaser | mriedem besides that, the rest shouldn't matter too much (i.e. scheduler before conductor?) | |
| 21:31:20 | mriedem | scheduler depends on placement as of ocata | |
| 21:31:33 | mriedem | scheduler and conductor are codependent | |
| 21:31:44 | mriedem | bring up the api last? | |
| 21:31:46 | mriedem | yeah | |
| 21:32:37 | mnaser | mriedem right now, order is: placement => nova-api => nova-scheduler => nova-conductor => nova-consoleauth => nova-novncproxy => nova-spicehtml5proxy => nova-serialproxy => nova-compute | |
| 21:32:45 | mnaser | however that order leads to this happening in gate sometimes: NotFound: Basic.publish: (404) NOT_FOUND - no exchange 'scheduler_fanout' in vhost '/' | |
| 21:33:11 | mriedem | this is a new problem? | |
| 21:33:31 | mnaser | i dont think its a nova issue but it seems to come and go so i was wondering if it was racing somewhere | |
| 21:33:38 | mriedem | never heard of that | |
| 21:33:57 | mnaser | okay, so must be a kolla thing, ill try to look some more | |
| 21:34:59 | mnaser | mriedem nvm, docker issue. its killing the process and mucking things up, sorry for noise | |
| 21:35:09 | mriedem | np | |
| 21:48:06 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix test_instance_get_all_by_host https://review.openstack.org/454360 | |
| 21:48:07 | mriedem | yay! ^ me help good | |
| 21:58:54 | openstackgerrit | Sivasathurappan Radhakrishnan proposed openstack/nova master: Add live-migration-force-after-timeout https://review.openstack.org/430218 | |
| 22:08:11 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/453896 | |
| 22:09:06 | openstackgerrit | Anusha Unnam proposed openstack/nova master: Add description to policies in remote_consoles.py https://review.openstack.org/454372 | |