| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-05 | |||
| 20:00:47 | mriedem | yes | |
| 20:01:11 | mriedem | it relies on the cell mappings in the api db to find the instances in each cell | |
| 20:01:29 | mgagne | ok, just trying to figure out how I will be able to test without updating the whole planet or doing the actual work ;) | |
| 20:01:52 | mgagne | like, can I just import our prod database and run migrations and then the script for example. | |
| 20:02:16 | mriedem | mgagne: you mean a backup test copy of your prod db? | |
| 20:02:16 | mgagne | (with some "minor" details to be addressed) | |
| 20:03:10 | mriedem | mgagne: what do you have for cell mappings right now? just cell0> | |
| 20:03:10 | mriedem | ? | |
| 20:03:16 | mgagne | kind of. import somewhere, run release db migration to queens. then migrate somehow to cellsv2 (since it's required?) and run the healer with a dummy placement api. | |
| 20:03:27 | mgagne | nothing, I'm running mitaka. | |
| 20:04:11 | mgagne | trying to find a simpler way to get to queens with prod data | |
| 20:04:38 | mgagne | if cellsv2 is required, I will just find a way to make it happen. | |
| 20:05:12 | mriedem | mgagne: have you read through https://docs.openstack.org/nova/latest/user/cells.html#upgrade-with-cells-v1 ? | |
| 20:05:36 | mriedem | the cell mappings are going to be 1:1 with your child ells | |
| 20:05:38 | mriedem | *cells | |
| 20:05:44 | mriedem | plus cell0 | |
| 20:05:53 | mgagne | kind of but never went through it yet. I will read and see how far I can go. | |
| 20:06:08 | mriedem | you'll have num(child_cell) + 1 cell mappings in the api db | |
| 20:06:11 | mgagne | my main concern is with online data migration and maybe nova-compute exporting resources to placement ? | |
| 20:06:33 | mriedem | nova-compute will start reporting inventory and allocation information to placement in ocata | |
| 20:06:48 | mriedem | if you're going straight to queens, nova-compute will not be reporting allocation information to placement, the scheduler does that | |
| 20:06:50 | mgagne | right so I need to run the actual service at least once | |
| 20:07:05 | mriedem | in queens, nova-compute will report inventory to placement on startup | |
| 20:07:34 | mriedem | in queens, the allocations will not be automatically posted for existing instances, which is why you need the heal_allocations CLI | |
| 20:08:05 | mriedem | but before you can run that, you need (1) the cell mappings created for each of your cells (2) the instances mapped to the cells | |
| 20:08:26 | mgagne | yes, will create mapping | |
| 20:08:39 | mgagne | my concern is with resources being exported by compute node before. right? | |
| 20:09:00 | mriedem | i'm not sure what that means | |
| 20:09:12 | mriedem | the cell mappings and placement stuff are unrelated | |
| 20:09:14 | mgagne | I mean, you need providers to be created somehow first? | |
| 20:09:32 | mgagne | I'm not that familiar with placement terminology btw | |
| 20:09:37 | mriedem | yes, you'll need the compute nodes reporting into placement before heal_allocations can run | |
| 20:10:24 | mriedem | heal allocations will iterate the cell mappings looking for instances that have a host but don't have allocations in placement, and then attempt to post allocations to plcaement for the given instance and compute node - so yeah you need the compute nodes reporting as providers into placement first | |
| 20:10:48 | mgagne | ok, that's what I wanted to understand. got it | |
| 20:13:04 | mriedem | btw, i think you can still do all of the cells v2 stuff (cell mappings, host mappings, instance mappings) and placement stuff in queens with the caching scheduler still in place - we have an experimental ci job that does that | |
| 20:13:29 | mriedem | well, we have a cells v1 job that does that too | |
| 20:14:17 | mgagne | yea, that's my plan. getting to queens, import database and run the tool and see how it goes | |
| 20:14:23 | mriedem | i think once you get all of the allocations healed, you can drop the caching scheduler and disable cells v1 | |
| 20:15:24 | mriedem | would be ideal if you could even kick the tires with a single child cell test env to get used to this before doing it | |
| 20:17:32 | mgagne | yea. I guess so. I need to test with Ironic too... but I thought I needed cells v2 ? I'm not sure what you mean by "drop the caching scheduler and disable cells v1" | |
| 20:26:04 | mriedem | mgagne: the goal is to not using the caching scheduler or cells v1 since both are deprecated | |
| 20:26:25 | mriedem | you need cells v2 since ocata yes | |
| 20:26:36 | mgagne | right | |
| 20:26:44 | mriedem | the IronicHostManager is also deprecated since pike | |
| 20:27:15 | mriedem | placement + custom resource classes removes the need for the ironic host manager in the scheduler and the Exact* filters | |
| 20:27:23 | mriedem | but that's another thing you can deal with separately before rocky | |
| 20:28:19 | mriedem | melwitt: this is the last day for https://review.openstack.org/#/q/topic:bp/granular-placement-policy+status:open in the runway, has all +2s, could use another core to go through it | |
| 20:28:35 | mgagne | right. I'm trying to get in a state where I can test your patch with production data without spending 900 man hours to get there. I think with your help I got enough info to kick start the process. | |
| 20:30:18 | mgagne | 1) Create test environment with Nova Mitaka+CellsV1 2) Import production database 3) Upgrade to Nova Queens, with FFU or whatever process I will find. 4) Migrate to CellsV2 5) Test allocation healer 6) Profit | |
| 20:33:34 | mriedem | mgagne: yeah, 3 and 4 are going to get fuzzy | |
| 20:33:45 | mgagne | can't wait =) | |
| 20:33:46 | mriedem | there is a blocker migration in ocata that won't let you finish the api db sync until you have cell0 for example | |
| 20:34:01 | mriedem | i just mean, 3 and 4 are kind of co-dependent | |
| 20:34:05 | mgagne | good to know, will add that step | |
| 20:34:21 | mriedem | once you get to ocata+ you can also run the nova-status upgrade check CLI | |
| 20:34:23 | mgagne | ok, so ocata will require some steps for cellsv2 | |
| 20:34:24 | mriedem | to check your progress | |
| 20:34:27 | mriedem | yup | |
| 20:34:38 | mgagne | cool, can't wait to test that tool too | |
| 20:43:09 | melwitt | mriedem: ack | |
| 20:44:00 | mriedem | i'm +2 on the vmware live migration spec if another specs core wants to put it in https://review.openstack.org/#/c/299207/ - i'm ok with holding up the merge of the actual code until the multinode vmware 3rd party ci job shows up | |
| 20:44:53 | melwitt | that reminds me of the zvm spec amendment, still not approved | |
| 20:45:05 | melwitt | but that's not subject to spec freeze | |
| 20:45:15 | melwitt | I don't think, anyway | |
| 20:57:28 | mriedem | nova list --limit -1 works like a charm http://paste.openstack.org/show/722757/ | |
| 21:01:59 | melwitt | mriedem: kinda wonder why that's not the default. I guess so you have to opt into paging | |
| 21:07:27 | mriedem | frickler: are you happy with this now? https://review.openstack.org/#/c/312626/ | |
| 21:07:33 | mriedem | i see you've done quite a bit of review on it already | |
| 21:08:04 | mriedem | melwitt: idk, for a smaller cloud with <1000 instances it's not going to be a problem, | |
| 21:08:15 | mriedem | for a public cloud the admin probably wants to opt into getting back a million records | |
| 21:08:23 | melwitt | yeah, good point | |
| 21:12:19 | mriedem | anyway, that patch made around ~400 LOC change for something that wasn't even broken | |
| 21:12:35 | mriedem | no release note on the behavior change either | |
| 21:13:21 | melwitt | those are the best | |
| 21:24:05 | melwitt | mriedem: I proposed adding the nova-lvm job to pike (as experimental) and it passed (phew) https://review.openstack.org/572130 | |
| 21:24:36 | melwitt | the ocata one however is busted, but on a cinder problem AFAICT, not the nova regression/fix https://review.openstack.org/572132 | |
| 21:29:47 | mriedem | ok | |
| 21:31:31 | mriedem | mgagne: a couple of questions for clarification in your spec https://review.openstack.org/#/c/312626/ | |
| 21:31:34 | melwitt | gah why can I never find the novaclient cli command list reference | |
| 21:31:44 | mriedem | melwitt: i don't think there is one | |
| 21:31:51 | mriedem | you're welcome | |
| 21:31:55 | melwitt | oh, to make people use osc. I see how it is | |
| 21:31:59 | mriedem | https://docs.openstack.org/python-novaclient/pike/cli/index.html | |
| 21:32:08 | mriedem | no, i think just because it never existed | |
| 21:32:21 | melwitt | it existed, unless I dreamed it | |
| 21:32:36 | melwitt | I shall google | |
| 21:32:39 | melwitt | some more | |
| 21:33:12 | mriedem | https://docs.openstack.org/mitaka/cli-reference/nova.html ? | |
| 21:33:26 | melwitt | yeah, that's what I'm thinking of. maybe we didn't import that when things got decentralized | |
| 21:33:28 | mriedem | so it existed in the central docs repo | |
| 21:33:34 | mriedem | yeah must not have | |
| 21:33:54 | melwitt | probably to discourage use of the novaclient CLI | |
| 21:34:12 | mriedem | no, probably just because no one imported the docs | |
| 21:34:15 | mriedem | it was on the projects to do that | |
| 21:34:21 | mriedem | https://docs.openstack.org/ocata/cli-reference/nova.html | |
| 21:34:27 | mriedem | https://docs.openstack.org/pike/cli-reference/nova.html | |
| 21:34:55 | melwitt | okay, I thought maybe someone decided it wasn't a great idea because we want to encourage osc use. I could see that point of view | |
| 21:35:28 | melwitt | but, I do know that boot from volume created from image is not possible in osc in one command, still something that needs to be fixed. and there are probably other things people still need novaclient CLI for | |
| 21:36:14 | mriedem | https://bugs.launchpad.net/python-novaclient/+bug/1775281 | |
| 21:36:15 | openstack | Launchpad bug 1775281 in python-novaclient "nova command line reference is gone since Pike" [Undecided,New] | |