Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-11-14
14:31:13 sean-k-mooney aspiers[m]: well the code is there for you to use...
14:34:37 aspiers[m] indeed. I don't think the actual coding is gonna be too hard
15:21:53 mriedem dansmith: so belmiro gave me another issue to worry about, for which he's going to open a bug. they drop the allocation candidates limit down to 10 to make scheduling faster, but they also at times disable a large number of computes in a cell for $reasons (mnaser also does that for load balancing the cloud); the problem then is b/c of the low alloc candidates limit, placement gives back 10 disabled computes and gets novalidhost
15:22:08 mriedem to workaround it he set reserved = total for inventory in placement on the disabled nodes,
15:22:12 mriedem but that sucks for hundreds of vms
15:22:20 dansmith ouch
15:22:24 mriedem i suggested a pre-request placement filter to somehow filter out disabled nodes
15:22:40 mriedem maybe forbidden trait/aggregate, idk
15:22:48 dansmith hmm
15:22:59 dansmith they disable compute or disable a cell?
15:23:04 dansmith *computes
15:23:08 sean-k-mooney mriedem: i was going to say yes tag with a trait and use forbindin traits but then we are using placmeent to sotre metadata
15:24:34 dansmith mriedem: I think the right thing to do is to make placement results better so they don't need such a tiny result set to mitigate performance issues
15:24:42 dansmith but, regardless of what that number is, the same problem could happen
15:25:02 dansmith so if it's a disabled compute situation, I think having compute decorate its RP in some way is probably the thing to do there,
15:25:15 dansmith although it generates a lot of chatty traffic from api->compute->placement
15:25:17 sean-k-mooney stephenfin: bauzas by the way is there a nova cyborg session at the ptg? they demoed something that does not work upstream in the key note. im reviewing the spec again but it would be good to check in with them.
15:25:44 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Use external placement in functional tests https://review.openstack.org/617941
15:28:05 mriedem i would really want to avoid needing to make placement api calls from the pre-placement filter for sure
15:28:30 mriedem i.e. to rope disabled computes into a resource provider aggregate and then tag a forbidden aggregate filter on the request
15:28:54 dansmith no,
15:29:01 dansmith you can't really do that I don't think
15:29:03 mriedem the compute tagging it's resource provider as disabled with a trait is one way yeah - not really the same for ironic (they set reserved = total)
15:29:14 dansmith I meant have computes decorate their own RP with a trait or reserved=total
15:29:19 openstackgerrit melanie witt proposed openstack/nova-specs master: Propose counting quota usage from placement and API database https://review.openstack.org/509042
15:29:21 mriedem sean-k-mooney: traits are going to be used as metadata in placement
15:29:28 mriedem dansmith: right
15:29:44 mriedem anyway, i told belmiro to open a bug so we don't lose track of the issue
15:30:29 mriedem we also talked about the upt stuff in compute
15:30:40 dansmith ack
15:30:42 mriedem i think he's ok with what's going on in efried's series to clean that up and sighup on changes
15:30:54 mriedem and my short history lesson on how we got here
15:31:00 mriedem my reward was his other issues
15:34:34 dansmith heh
15:39:22 sean-k-mooney mriedem: ya traits are kind of metata but im not sure how peopel would fell about a mainantnce trait or whatever we would use to make the compute node RP as down in placemetn
15:40:02 dansmith I would expect jaypipes to not want that
15:40:11 dansmith since "cant do stuff" isn't a capability
15:40:22 dansmith reserved=total is a lot more accurate
15:40:29 dansmith "these resources aren't schedulable right now"
15:40:33 sean-k-mooney yes that was my original concern with a trait
15:40:38 mriedem yeah, plus that matches what the ironic driver already does today
15:40:42 dansmith yep
15:41:19 sean-k-mooney ya that does seem alot cleaner
15:41:42 mriedem i'm not sure if that happens immediately or in a periodic? when the admin disables the compute service in the api, does the api set reserved=total for all inventory on the compute node resource provider?
15:42:09 mriedem seems it should be immediately mirrored
15:42:11 dansmith disable is an rpc call right?
15:42:16 mriedem no
15:42:21 dansmith because xen used to do stuff when you disabled it I think
15:42:47 dansmith maybe I'm mixing in the maintenance mode call
15:42:49 mriedem it's just a db change
15:42:53 mriedem yeah i think yo uare
15:43:27 dansmith okay, so I was going to say let the compute service do the disable,
15:43:33 dansmith but same difference just having api do it
15:43:44 sean-k-mooney mriedem: well renebaling it would need the virt driver to resore the old reserved value right
15:43:45 dansmith it's more better to let compute do it I think, just because it's the one that owns the resources
15:43:47 dansmith and the RP
15:43:51 mriedem set_host_enabled was the thing you're thinking of
15:43:54 mriedem in the os-hosts api
15:44:07 jaypipes sean-k-mooney, dansmith, mriedem: I'm still reviewing Virgil's Aeneid^W^Wthe Cyborg-nova interaction spec.
15:44:09 dansmith and that doesn't get called when disable?
15:44:16 sean-k-mooney mriedem: what inventor was reserver=total set on by the way
15:44:19 mriedem dansmith: no
15:44:35 mriedem PUT /os-services/{compute_node_id} is just a db change on the services record
15:44:43 sean-k-mooney jaypipes: im reviewing that currently too
15:44:52 sean-k-mooney jaypipes: this one right https://review.openstack.org/#/c/603955/10/specs/stein/approved/nova-cyborg-interaction.rst,unified
15:44:59 dansmith mriedem: okay, well, it's rather unfortunate to have the api muck with the inventory of the compute node, but mechanically it would work
15:45:13 jaypipes sean-k-mooney: yes. that one.
15:45:19 dansmith mriedem: un-disabling is a good thought though, since you'd really not want to do that from the api I think
15:45:23 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add API ref guideline for body text https://review.openstack.org/605628
15:45:35 sean-k-mooney jaypipes: i jsut pushed what i had it take a while to get through that end to end
15:45:52 mriedem dansmith: you mean fixing the reserved value once the service is re-enabled?
15:45:58 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (7) https://review.openstack.org/571992
15:46:16 dansmith mriedem: yeah
15:46:19 mriedem yeah i don't think we can do that from api, compute knows that info
15:46:19 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (8) https://review.openstack.org/571993
15:46:30 mriedem some of it is stored in the compute node record, but maybe not all inventory records
15:46:33 mriedem only ram/vcpu/disk
15:46:33 dansmith mriedem: right.. also there's a potential race I guess,
15:46:38 mriedem yeah
15:46:51 mriedem let the compute b/c the source of truth for inventory (unless the API reserves it all...?)
15:46:58 dansmith yeah
15:46:58 dansmith mriedem: which is api racing with compute to update the inventory.. they could both use generation to catch the race, but probably better to just let one own it
15:47:16 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add description of custom resource classes https://review.openstack.org/616721
15:47:39 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.openstack.org/566153
15:48:05 openstackgerrit Takashi NATSUME proposed openstack/nova master: api-ref: Add descriptions for vol-backed snapshots https://review.openstack.org/615084
15:48:25 openstackgerrit Takashi NATSUME proposed openstack/nova master: Use links to placement docs in nova docs https://review.openstack.org/614056
15:48:39 sean-k-mooney jaypipes: are you at the summit by the way. you ususally only attend the ptgs as of late
15:48:48 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove Placement API reference https://review.openstack.org/614437
15:48:52 jaypipes sean-k-mooney: no. I'm not in Berlin
15:49:08 openstackgerrit Takashi NATSUME proposed openstack/nova master: Transform compute_task notifications https://review.openstack.org/482629
15:49:26 sean-k-mooney jaypipes: ah ok same. its weird not being there but also i got to review that spec again
15:49:28 openstackgerrit Takashi NATSUME proposed openstack/nova master: Use oslo_db.sqlalchemy.test_fixtures https://review.openstack.org/609352
15:50:05 openstackgerrit Takashi NATSUME proposed openstack/nova master: Fix best_match() deprecation warning https://review.openstack.org/611204
15:50:23 openstackgerrit Takashi NATSUME proposed openstack/nova master: Fix server query examples https://review.openstack.org/616834
15:51:08 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Use external placement in functional tests https://review.openstack.org/617941
15:56:15 jaypipes sean-k-mooney: hell of a reward...
16:04:26 mriedem ha
16:50:58 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Use external placement in functional tests https://review.openstack.org/617941
16:56:34 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Use external placement in functional tests https://review.openstack.org/617941
17:35:36 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Use external placement in functional tests https://review.openstack.org/617941

Earlier   Later