Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-11-29
21:44:09 melwitt k, doing
21:47:47 oomichi_afk alex_xu: nice work, +2 for both
21:49:15 oomichi alex_xu: gmann_afk: happy to see your feedback on https://review.openstack.org/#/c/523194
21:54:00 mriedem melwitt: a couple of questions in your quota dead code removal patch https://review.openstack.org/#/c/511689/
21:54:13 melwitt mriedem: cool, thanks for looking
21:57:21 openstackgerrit Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766
21:57:35 openstackgerrit melanie witt proposed openstack/nova stable/ocata: Save updated libvirt domain XML after swapping volume https://review.openstack.org/523911
22:25:02 melwitt mriedem: replied. I can push a new rev if you don't find any other issues with it
22:30:37 mriedem melwitt: replied, +W, just deal with the other cleanup in a follow up
22:30:57 melwitt mriedem: k, thanks
22:33:49 openstackgerrit melanie witt proposed openstack/nova master: qemu-img do not use cache=none if no O_DIRECT support https://review.openstack.org/523554
22:34:31 melwitt I've had to fix that a billion times because that direct io test function tries to create a file to check something, but the file create fails in CI but not locally
22:38:44 mgagne I'm reading https://docs.openstack.org/nova/pike/user/placement.html#ocata-15-0-0
22:38:50 mgagne what does "Users of CellsV1 will need to deploy a placement per cell, matching the scope and cardinality of the regular nova-scheduler process." mean?
22:42:46 mriedem mgagne: means you'd have a placement service running per child cell, like the nova-scheduler process using the FilterScheduler (not the cell scheduler at the top api cell); that's what belmiro is doing at CERN, but I think you could also do a global placement (like keystone), which is what smorrison at NeCTAR is planning on doing.
22:43:46 mriedem i can't exactly remember why belmiro was going to do it per-cell to start, because he was also going to run into some service catalog entry issues with that, and maybe have to put the per-cell placement endpoints in per-cell regions in the catalog
22:43:54 mgagne ok, what I'm wondering is: isn't placement a HTTP api? if so, how do I declare it in the catalog if catalog is cell agnostic? I also couldn't find (when I checked) a way to override the placement API url in the config.
22:44:32 mriedem right, i think that's what belmiro ran into and was going to have to do some funny stuff with regions in the catalog
22:44:35 mgagne right... imo, the upgrade note suggests a solution which isn't really possible without thinking outside the box or something
22:44:39 mriedem there isn't an option to override the endpoint url for placement in nova.conf
22:44:49 mgagne there is not
22:44:55 mriedem mgagne: yeah, i think that was written before belmiro started looking at doing that
22:45:23 mriedem dansmith: tssurya_: do you remember why belmiro decided to do a per-cell placement service rather than a global one?
22:45:54 mriedem mgagne: would probaly be good if we could start an etherpad or get a ML thread started with notes from belmiro and smorrison about what they are doing
22:45:57 mriedem i could start that if you want
22:47:26 mgagne sure, I'm trying to understand what I need to do for newton and I keep reading stuff I don't fully understand or I don't know how it can technically be doable
22:48:42 mriedem yeah let me start an etherpad for notes
22:49:00 mgagne so if the recommended way to run placement api is globally, I don't understand why upgrade notes suggest something else
22:49:54 INeedaUUID dansmith: Ohhh. You're gonna hate what I did to sort changing the uuids.
22:50:12 INeedaUUID You're gonna tel me it's not a good idea. So happy for you to facepalm and tell me i'm stupid.
22:51:56 mriedem mgagne: i think that was written before we released the catalog issue
22:52:25 mgagne would there be a way to update the document?
22:53:08 mriedem yeah
22:53:56 openstackgerrit Merged openstack/nova master: Add 'all_tenants' for GET sec group api ref https://review.openstack.org/522910
22:53:57 mgagne also, is the placement api supposed to be available to the end user? if so, for what purpose?
22:55:26 mriedem no, it's admin-only
22:56:00 mriedem i think i remember why belmiro was doing per-cell, it's because then the per-child cell scheduler is only asking placement for hosts in that cell, so it's a perf thing in the scheduler
22:56:17 mriedem because cellA scheduler doesn't need to wittle out computes from cellB
22:56:22 mriedem https://etherpad.openstack.org/p/cellsv1-to-v2-migration
22:57:27 openstackgerrit Merged openstack/nova master: Remove unnecessary self.flags and ConfPatcher https://review.openstack.org/522993
22:58:16 mgagne "no way to override the placement endpoint URL in nova.conf per child cell before Queens" interesting. so it's now possible
22:58:50 mriedem mgagne: yeah, i forgot that with the keystoneauth adapter stuff that efried worked on, we now have an endpoint_override option for [placement] in nova.conf
22:58:51 mriedem in queens
22:59:54 mgagne good stuff, we heavily rely on endpoint override. and no, we don't care about DNS split view, not gonna happen for us. ;)
23:00:47 mriedem that's part of the point of https://specs.openstack.org/openstack/nova-specs/specs/queens/approved/use-ksa-adapter-for-endpoints.html
23:00:57 mriedem consistent configuration of all of the services that nova talks to
23:01:08 mgagne super good news!
23:02:28 mgagne I was a bit worried because I keep reading on the list that DNS split view is the (only) way to go. and everything else is "garbage" or a state of the art solution.
23:04:54 mriedem dansmith: melwitt: tssurya_: fyi https://etherpad.openstack.org/p/cellsv1-to-v2-migration
23:05:25 mgagne how don't know how to work that question: When should you move to cellsv2? Maybe based on multiple use cases: you current have no cells. you have only one cell, you have multiple cells. You need X features which is only available in release Y. etc.
23:05:42 mgagne and eventually: how
23:09:10 mriedem dansmith did a talk about that question in sydney
23:09:13 mriedem i'll link in the video
23:09:34 mriedem takashin: hey, i'm trying to fix https://review.openstack.org/#/c/408964/
23:09:46 mriedem takashin: i think the multi-cell functional test is intermittently failing
23:10:59 mriedem mgagne: https://www.openstack.org/videos/sydney-2017/adding-cellsv2-to-your-existing-nova-deployment
23:11:29 melwitt mriedem: if we have a nova change that depends on an os-brick fix, and we get a new os-brick release and it's in our requirements.txt, how would that work for a backport?
23:11:44 melwitt are we able to bump requirements.txt os-brick version on a stable branch or?
23:11:48 mriedem melwitt: this is your crazy newton hard reboot thing isn't it
23:11:53 mriedem melwitt: no
23:11:55 mriedem not on stable
23:12:19 mriedem melwitt: you can do whatever you want downstream of course
23:12:25 melwitt yes ... it's the compute host reboot thing
23:12:30 mriedem melwitt: but upstream it would likely mean somehow patching in the os-brick changes to nova stable
23:12:43 melwitt yeah, I know. just trying to think how we could fix this up for upstream ppl too
23:14:16 mgagne ok, now I have some talks to watch
23:14:41 melwitt I had been thinking, what if I backport the os-brick changes to their stable branches but I wasn't sure if that can map to our stable branches
23:20:44 mgagne mriedem: ok, so lets say you are running cellsv1 mitaka with multiple cells, your objective is to get to Pike with cellsv2 and with your cells intact. what's the plan? is that possible?
23:21:18 mgagne because that's my biggest concern and nowhere I can read about how to make it happen.
23:22:37 mriedem melwitt: you could backport the os-brick changes to stable, and release a patch .z release of os-brick on stable, you just can't bump the minimum required version of os-brick in g-r on stalbe
23:22:46 mriedem upper-constraints can be updated on stable though if needed
23:23:28 mriedem mgagne: i think that's what belmiro is working on as well, moving from mitaka with multiple child cells to v2
23:23:43 mriedem i'm not sure if his goal is pike, or just ocata
23:24:19 mgagne well, at this point, as long as I can get to Pike or a place where cells aren't a concern anymore, I'm fine.
23:24:43 mgagne it's just that it looks like I won't be able to skip releases like I did before due to online data migration
23:24:44 melwitt mriedem: yeah, okay. that's along the lines of what I was thinking. because basically, without the os-brick fix, the nova fix doesn't fully work. we'd have to handle the volume detach exit code in the libvirt driver to fix it on stable if there's no way to get it from a stable branch os-brick
23:25:29 mriedem melwitt: i'd just backport the brick stuff, release a .z on stable, bump upper-constraints and then if anyone needs the fix for libvirt in nova on stable, they have to pick up the updated os-brick
23:25:35 mriedem we just won't force it via g-r
23:26:09 mriedem mgagne: i don't think you've been able to skip the online data migrations since kilo
23:26:18 mriedem or shouldn't have
23:26:29 mgagne well, I upgraded from kilo to mitaka just fine =)
23:26:31 mriedem we've put in blocker schema migrations if you haven't completed the online data migrations
23:26:41 mriedem sure, maybe you just didn't need to migrate any of that stuff
23:27:20 mgagne well, there is a glitch with cellsv1 where service rpc version is checked in database and since you aren't running conductor or scheduler in API, migration fails. I just commented that part.
23:27:48 mriedem i might have been thinking of this https://github.com/openstack/nova/blob/stable/newton/nova/db/sqlalchemy/migrate_repo/versions/330_enforce_mitaka_online_migrations.py
23:27:58 mriedem newton schema migration requires that you have done the mitaka online data migrations
23:28:11 mgagne yes, I'm not there yet :P
23:29:08 mriedem "there is a glitch with cellsv1 where service rpc version is checked in database and since you aren't running conductor or scheduler in API, migration fails. I just commented that part."
23:29:11 mriedem where does that happen?
23:29:29 melwitt mriedem: just to clarify, if someone installs nova ocata, they will be prevented from installing the latest os-brick because of upper-constraints? or no? trying to understand the difference between them updating to a os-brick .z release vs just pulling the latest os-brick
23:29:46 mriedem that does remind me of something i think i've talked with dansmith about before which i don't think we're handling yet, maybe we are, but when the api checks the minimum compute service version, i don't know if it's iterating all cells
23:30:09 mriedem melwitt: totally depends on how nova is installed
23:30:20 mgagne will need to check
23:30:23 mriedem if you're pip installing into a tox venv from the nova git repo, then sure :)
23:30:28 mriedem but i doubt most people are doing that
23:30:40 melwitt mriedem: :\ okay. I was thinking whether going through all the .z release stuff helps much or not
23:30:49 mriedem fedora rpm specs null out the requirements.txt so that the rpms are just installed from the versions in the rpm spec fedora says is good
23:31:19 dansmith mriedem: it is iterating
23:31:41 melwitt I see
23:31:44 mriedem get_minimum_version_all_cells is

Earlier   Later