Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-02-14
15:15:50 mriedem cell_id = Column(Integer, ForeignKey('cell_mappings.id'),
15:16:06 mriedem and nullable=False
15:16:14 mriedem yeah, so we have to remove host mappings if we remove a cell mapping
15:16:17 dansmith but,
15:16:29 dansmith you can delete a cell with delete_cell and it will nuke all the host mappings
15:16:33 dansmith so then you're really toast
15:16:41 mriedem if you use --force yeah
15:17:27 dansmith so we have delete_host
15:17:58 dansmith which unmaps the compute node, and deletes the hostmapping
15:18:06 dansmith so in both of those I guess we need to update placement
15:18:23 mriedem i'm not sure why delete_host requires a --cell_uuid arg,
15:18:26 mriedem but that's unrelated
15:18:37 dansmith although nova-manage doesn't do any rpc or calls to other services right now, AFAIK, so we kinda need a --no-placement flag to each of those in case you're operating in an island
15:19:13 mriedem well,
15:19:29 mriedem nova-manage could use the scheduler report client and handle an http error response if it can't talk to placement
15:19:43 dansmith yes, but you dont' want to proceed then,
15:19:48 dansmith unless you tell it to
15:20:00 dansmith because if we can't talk to placement, we can't proceed without corrupting the linkage (as we are now)
15:20:07 dansmith so you need to fail in that case, unless instructed to ignore
15:20:36 dansmith someone with placement down during an upgrade pruning a host wants to know that they're about to orphan some placement records
15:20:38 mriedem could just throw another --force on delete_host
15:20:48 dansmith we need it for both though
15:20:54 dansmith delete_cell and delete_host
15:21:00 mriedem sure, we have --force on delete_cell already
15:21:04 dansmith and delete_cell has --force already which means something else though
15:21:15 dansmith it means "delete all the things, yeah", but this would be "and also, don't delete some stuff"
15:21:21 mriedem ha
15:21:24 dansmith right?
15:21:40 mriedem it's just confusing ux
15:21:56 dansmith but really, they're two meanings of force
15:22:03 dansmith what we have now is really --recursive
15:23:20 stephenfin dansmith: This NUMA aware vSwitch spec is basically turning into a book. Be prepared!
15:23:41 dansmith regardless, we need that thing and some appropriate flaggage to control intent
15:23:48 dansmith stephenfin: noted
15:24:00 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove old flavor_get_by_name db api method https://review.openstack.org/544379
15:24:25 dansmith belmoreira: tssurya: can one of you open a bug for this so I can point a patch at it?
15:24:48 tssurya dansmith : sure
15:24:59 belmoreira dansmith yes we can do it, we can also work on it if you prefer
15:25:06 dansmith or that ;)
15:25:26 mriedem dansmith: in the scenario you said, "someone with placement down during an upgrade pruning a host wants to know that they're about to orphan some placement records" - if i'm removing a compute host (node really here), are you just saying that you want to *not* delete the compute node if placement is down, right?
15:25:44 dansmith mriedem: right
15:25:54 mriedem default behavior would be, we try to remove from placement and if that fails, we fail, unless instructed to ignore it
15:25:56 dansmith mriedem: unless they say "I know what I'm doing" or something
15:26:01 dansmith mriedem: yes
15:26:18 mriedem that still seems like --force should cover that case to me
15:26:34 dansmith but they may want --force to mean "delete all the hosts too",
15:26:46 dansmith but then they would be opting into "and orphan all the data in placement if it's down"
15:27:21 dansmith so let's say you're deleting a cell while you're down for an upgrade, and that cell has 10 hosts
15:27:22 mriedem well, --force is a destructive option
15:27:51 dansmith it is but it's not a "and corrupt my data"
15:27:52 dansmith so you're down and deleting a cell with ten hosts
15:27:54 dansmith you run delete_cell --force, to delete all ten hosts and the cell record
15:27:57 dansmith if placement is down,
15:28:04 dansmith you now just orphaned the placement records for those ten hosts,
15:28:22 dansmith so when you come back up, ten providers in placement are no longer present, but will be returned by placement forever,
15:28:29 dansmith and unless you know what the uuids were of those records you deleted,
15:28:33 dansmith you can't really clean that up
15:28:52 openstackgerrit Merged openstack/nova-specs master: Update Queens NRP spec to reflect reality https://review.openstack.org/544086
15:29:10 dansmith our current --force is poorly named.. it should be --recursive
15:29:23 dansmith because --recursive means "and delete everything you have to to delete the cell"
15:29:51 dansmith cdent: do you not get what I'm saying here?
15:30:11 cdent No, I totally get it. I'm not kidding. It is an astute observation.
15:30:26 dansmith okay
15:30:33 mriedem i get the scenario, i just, idk
15:30:50 cdent one way around the problem is "don't have placement down"
15:30:59 cdent which is probably a good rule of thumb anyway
15:31:05 dansmith mriedem: in belmoreira's case, he would run --force to delete all the hosts, and then he would be in exactly the same position that started this conersation
15:31:24 dansmith cdent: aye, it'd be better to not allow skipping the placement cleanup at all
15:31:33 mriedem dansmith: did belmoreira say placement was down?
15:31:49 dansmith mriedem: no, but he'd want to use delete_cell --force to delete the cell and all the hosts,
15:32:14 dansmith but if placement was down when doing that (because maintenance), then we'd orphan all the records that he's currently saying we orphan, which is a problem
15:32:35 dansmith i.e. you can't opt into the recursiveness without choosing the "and also corrupt the data if you need to"
15:33:20 mriedem so we could just say, you can't do that if placement is down, but sure i agree people will want a backdoor
15:33:22 mriedem to bypass that
15:33:51 dansmith right, which is why I'm saying we need another flag
15:33:52 dansmith since we already burned --force to mean --recursive, we need something else
15:34:09 mriedem --ignore-placement?
15:34:19 mriedem --fudge-it?
15:34:32 dansmith yeah, I said --no-placement above, but exactly.. something like that
15:35:41 mriedem apparently --force doesn't also remove the instances/mappings
15:35:49 tssurya mriedem : I have a patch for that
15:36:00 tssurya in gerrit
15:36:17 dansmith yeah and I think I'm +2 on it already right?
15:36:18 mriedem force deleting instances from nova-manage is pretty scary
15:36:37 mriedem you're likely orphaning ports, volumes, etc
15:36:38 tssurya mriedem : no --force does not delete instances
15:36:47 mriedem just the mappings?
15:36:53 tssurya mriedem : it only removes/cleans mappings for deleted instances
15:36:54 tssurya yes
15:36:57 tssurya just mappings
15:36:58 dansmith ah, nm was thinking of the other one
15:37:08 tssurya we under no circumstance delete a cell if there are instances
15:37:10 mriedem i'm saying delete_cell --force will fail today if there are instance mappings in the cell
15:37:50 mriedem ok this patch https://review.openstack.org/#/c/540073/
15:38:08 tssurya mriedem : yes , my after merging my patch, delete_cell will fail if there any living instances
15:39:25 mriedem ok i see
15:39:47 jaypipes stephenfin: https://review.openstack.org/#/c/538217/3/specs/rocky/approved/integrate-mypy-type-checking.rst <-- well, sheeet, let's just rewrite Nova in Golang (or Rust). :P
15:40:49 stephenfin jaypipes: If you can find me a tool to autoconvert one language to another, I'm all for it ;)
15:40:59 mriedem vish suggested java after joining oracle
15:41:04 mriedem i know java, not go or rust

Earlier   Later