Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-07-06
11:09:13 gibi bauzas: +Ad the mdev spec
11:11:58 gibi sean-k-mooney: re: claim in conductor: so you say that the claim still happens on the compute service via a dedicated rpc call?
11:15:54 sean-k-mooney[m] yes via a dedicated rpc call since the instance claim need to claim things in the resource tracker on that node. to do it entirely on the conductor we would need to make other change so that the resource tracker state is entirely reflected in the db which i don’t think it currently is
11:16:10 sean-k-mooney[m] the pci devices and numa info should be
11:16:41 sean-k-mooney[m] but mdevs are not and there may be other info missing to do it entirely in the conductor
11:17:48 sean-k-mooney[m] we could try to confirm what the gaps would be i have not looked closely enough at what the claim is doing to confirm
11:18:01 gibi but if the conductor calls the compute to claim, then how is that different from the currently existing rpc cast from the conductor to the compute to do the instance lifecycle operation?
11:19:21 sean-k-mooney[m] well its a call for one so its synchronous, it will also happen earlier before we call any other service
11:19:57 sean-k-mooney[m] the claims themselves are not currently stored in the db at least not directly
11:20:37 sean-k-mooney[m] pci devices are claimed in the db and the instance numa topology blob contains any cpu or huge page assignments
11:21:38 sean-k-mooney[m] but as far as i know there is no instance claim object stored in the db at any point which the compute node could retrieve in the periodic task when its doing its resource update
11:22:34 sean-k-mooney[m] i could be wrong?
11:23:28 sean-k-mooney[m] but even for migrations ect i think the move claims only exist in memory inside the migration context
11:26:40 gibi so in case of a new instance build the scheduling happens here https://github.com/openstack/nova/blob/e7a7fd51d12d045ff134d55a4a1749c1feee0386/nova/conductor/manager.py#L1518 and the rpc cast to the compute is at the bottom of the function https://github.com/openstack/nova/blob/e7a7fd51d12d045ff134d55a4a1749c1feee0386/nova/conductor/manager.py#L1692
11:26:56 gibi inbetween I only see external calls to cyborg to bind arqs
11:27:02 gibi the rest is pure python code afaik
11:28:56 gibi anyhow my point is that I cannot really see the gain just by looking at it in theory.
11:30:16 gibi I think a small PoC would be good to show that i) moving the claim to a separate rpc call is possible without blowing up both service and ii) we can make a claim a lot earlier than today by that move
11:32:18 opendevreview Merged openstack/nova-specs master: Add generic mdevs to Nova https://review.opendev.org/c/openstack/nova-specs/+/792796
11:34:46 sean-k-mooney gibi: ack a small poc is resonable i can see if we can whip one up quickly
11:41:16 sean-k-mooney /away
11:42:58 sean-k-mooney basically i was hping to move it before we create the block device mappings https://github.com/openstack/nova/blob/e7a7fd51d12d045ff134d55a4a1749c1feee0386/nova/conductor/manager.py#L1645
11:43:12 sean-k-mooney proably to here at the latest https://github.com/openstack/nova/blob/e7a7fd51d12d045ff134d55a4a1749c1feee0386/nova/conductor/manager.py#L1625
11:44:05 sean-k-mooney i was orginally hopping to do it around here https://github.com/openstack/nova/blob/e7a7fd51d12d045ff134d55a4a1749c1feee0386/nova/conductor/manager.py#L1539 but that might be too early
11:46:58 sean-k-mooney realistically it proably should be after the az and quota check https://github.com/openstack/nova/blob/e7a7fd51d12d045ff134d55a4a1749c1feee0386/nova/conductor/manager.py#L1586-L1614
11:48:17 sean-k-mooney so doing it per instance near the start of this loop would be earlest we could do it https://github.com/openstack/nova/blob/e7a7fd51d12d045ff134d55a4a1749c1feee0386/nova/conductor/manager.py#L1616-L1631
11:51:36 gibi sean-k-mooney: that bdm creation also just calling the db nothing realy time consuming happens there either
11:52:19 sean-k-mooney the current instance claim happens here in the compute https://github.com/openstack/nova/blob/052cf963583ab7c6bbe4fcbf7bfe69f8f6733bdb/nova/compute/manager.py#L2383
11:52:33 sean-k-mooney so tere is not much on the comptue side before it
11:54:15 sean-k-mooney i dont know it kindof feels wrong to me that we dont do the instance claim and placement allocation creation at the same time
11:54:51 sean-k-mooney that woudl involve moveign the instnace claim even eairlier
11:58:05 gibi I agree that we should do placement and compute claim as close as possible
11:58:18 gibi for me a distance is mostly due to the RPC boundary
11:58:40 gibi not the code between the placemetn allocation call and the instance claim
11:58:54 gibi (except that cybor arq thing that also trigger external calls :/)
12:12:18 sean-k-mooney this is where that happens correct
12:12:20 sean-k-mooney https://github.com/openstack/nova/blob/052cf963583ab7c6bbe4fcbf7bfe69f8f6733bdb/nova/scheduler/filter_scheduler.py#L238-L241
12:13:21 sean-k-mooney we could do it here i guess https://github.com/openstack/nova/blob/052cf963583ab7c6bbe4fcbf7bfe69f8f6733bdb/nova/scheduler/utils.py#L1255
12:14:57 sean-k-mooney basiclly if the placment claim succeed we then to the instance claim
12:15:04 gibi if we can do that there then that would be the closest thing possible
12:15:47 sean-k-mooney ok ill see if i can poc that then the only issue with that is we need to make sure to release the instance_claim on the host if we fail for some reaosn but that shoudl be doable
12:15:53 gibi lyarwood: left some comments in the nova-manage refres connection_info spec https://review.opendev.org/c/openstack/nova-specs/+/799624
12:16:51 opendevreview Stephen Finucane proposed openstack/nova master: api: Align availability zone info with forced host https://review.opendev.org/c/openstack/nova/+/798145
12:17:28 gibi sean-k-mooney: yeah, cleanup could be tricky as now a claim might need to be cleaned both from the conductor if something fails before the rpc cast or cleaned from the compute if the e.g. the virt driver fails
12:17:39 stephenfin bauzas: gibi: I think that matches up with what was requested? ^
12:17:46 stephenfin (the AZ-host mismatch patch)
12:17:50 gibi stephenfin: looking
12:18:00 bauzas stephenfin: will look
13:40:26 opendevreview Stephen Finucane proposed openstack/nova master: api: Align availability zone info with forced host https://review.opendev.org/c/openstack/nova/+/798145
13:40:47 stephenfin bauzas: ^
13:41:33 bauzas stephenfin: ta, will look
13:58:55 gmann gibi: ack, will check today
13:59:00 gibi gmann: thanks
14:55:10 opendevreview sean mooney proposed openstack/nova master: [DNM] testing os-vif use unreleased os-vif version to test live migration and upgrades https://review.opendev.org/c/openstack/nova/+/799680
15:02:45 bauzas sean-k-mooney: around ?
15:15:23 gibi stephenfin: seems like the keystone doc job is broken since sqlalchemy 1.4 due to the the warning that is actually fixed by oslo.db 10.0.0
15:15:57 stephenfin oh, wonderful. So a catch-22?
15:16:08 gibi a bit yes
15:16:41 gibi you have more knowledge about sphinx, can we temporary turn off in it to treat warnings as errors?
15:16:58 gibi that would be a way out
15:17:23 stephenfin what's the link to the patch again?
15:17:32 gibi btw, I'm OK to delete the keystone test, so I will abandon my patch
15:17:38 stephenfin wait, got it
15:17:59 gibi https://review.opendev.org/c/openstack/keystone/+/799670
15:18:10 gibi waiting...
15:21:15 gibi stephenfin: at least the 10.0.0 req bump now green with a depends on on my keystone patch https://review.opendev.org/c/openstack/requirements/+/799054
15:22:43 stephenfin gibi: https://review.opendev.org/c/openstack/keystone/+/799682
15:24:04 gibi I'm not sure I can follow what that patch does. Does this syntax override the cache_ok.__doc__ ?
15:24:08 ganso bauzas, lyarwood, melwitt: When you have a minute could you please take a look at this backport https://review.opendev.org/c/openstack/nova/+/798717 ? Thanks in advance!
15:24:10 stephenfin yujp
15:24:14 stephenfin *yup
15:24:33 gibi jezus that feels like a bug in python :)
15:24:46 gibi or at least an obscure feature
15:25:29 gibi OK, so this will fix the doc bug, then we move your test fix top of it and then update the version bump patch to point to your test fix
15:26:48 gibi I will update the requirement bump patch...
15:26:56 stephenfin It's not Python - it's Sphinx. It's just attempting to do docstring inheritance which would be a sensible thing if you owned the entire subclassed stack
15:27:06 stephenfin Sounds good to me
15:27:25 gibi done
15:29:08 gmann gibi: done, it is as expected because we still support legacy policy so old token can create server. added comment and marked as invalid until author has further query on usage part https://bugs.launchpad.net/nova/+bug/1934478
15:31:09 opendevreview Stephen Finucane proposed openstack/nova master: WIP: db: Integrate alembic https://review.opendev.org/c/openstack/nova/+/799530
15:31:09 opendevreview Stephen Finucane proposed openstack/nova master: db: Trivial style changes https://review.opendev.org/c/openstack/nova/+/799529
15:31:10 opendevreview Stephen Finucane proposed openstack/nova master: db: Normalize migrations tests https://review.opendev.org/c/openstack/nova/+/799684
15:31:10 opendevreview Stephen Finucane proposed openstack/nova master: fixup! db: Post reshuffle cleanup https://review.opendev.org/c/openstack/nova/+/799683
15:35:03 gibi gmann: thank you
15:41:34 opendevreview Stephen Finucane proposed openstack/nova master: db: Add initial alembic migration for main DB https://review.opendev.org/c/openstack/nova/+/799527
15:41:34 opendevreview Stephen Finucane proposed openstack/nova master: db: Post reshuffle cleanup https://review.opendev.org/c/openstack/nova/+/799526
15:41:35 opendevreview Stephen Finucane proposed openstack/nova master: db: Trivial style changes https://review.opendev.org/c/openstack/nova/+/799529
15:41:35 opendevreview Stephen Finucane proposed openstack/nova master: db: Add initial alembic migration for API DB https://review.opendev.org/c/openstack/nova/+/799528
15:41:36 opendevreview Stephen Finucane proposed openstack/nova master: WIP: db: Integrate alembic https://review.opendev.org/c/openstack/nova/+/799530
15:41:36 opendevreview Stephen Finucane proposed openstack/nova master: db: Normalize migrations tests https://review.opendev.org/c/openstack/nova/+/799684
15:45:53 gibi nova meeting starts in 15 minutes here in the channel
15:49:01 gibi stephenfin: I only have a naming question in https://review.opendev.org/c/openstack/nova/+/798145/4/nova/tests/unit/api/openstack/compute/test_servers.py#4537
15:55:50 stephenfin gibi: open to ideas on naming. I don't have anything better :)
15:56:18 gibi stephenfin: I'm wondering if that is really a mismatch case as both the default az is zone1 and the host is in zone1
15:57:09 gibi never mind
15:57:16 gibi I see your reasoning in the review now
16:00:10 opendevmeet The meeting name has been set to 'nova'
16:00:10 opendevmeet Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:10 opendevmeet Meeting started Tue Jul 6 16:00:10 2021 UTC and is due to finish in 60 minutes. The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:10 gibi #startmeeting nova

Earlier   Later