Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-03-20
11:35:15 sean-k-mooney c-vol is on the sub node so cinder can do testing with multiple backends too
11:35:19 openstackgerrit Brin Zhang proposed openstack/nova master: libvirt:driver:Disallow AIO=native when 'O_DIRECT' is not available https://review.opendev.org/682772
11:36:11 sean-k-mooney lyarwood: im not sure we we need eitehr in our specific jobs but other project want that for there jobs
11:40:53 lyarwood sean-k-mooney: right, the way we layout the services at the moment isn't granular enough IMHO, we should really have a subnode layout *just* for compute migration testing
11:45:13 sean-k-mooney lyarwood: im not sure i agree but we could
11:45:31 lyarwood sean-k-mooney: https://review.opendev.org/#/c/714057/ - can you hit that regardless
11:45:37 sean-k-mooney lyarwood: in a hyper convered deployment i wound not find it unusal to have cinder vol on a compute
11:45:49 sean-k-mooney glance api is a littel weired but we only have two nodes
11:45:58 lyarwood sean-k-mooney: our default deployment topologies shouldn't be HCI
11:46:05 sean-k-mooney i think it should
11:46:07 lyarwood sean-k-mooney: it's just adding overhead for no reason
11:46:14 sean-k-mooney if we use it
11:46:16 lyarwood duplicating c-vol and g-api is utterly pointless
11:46:22 lyarwood on the computes upstream
11:46:31 lyarwood there's a place for that but it isn't in openstack/nova's gate
11:46:38 lyarwood using devstack
11:46:52 lyarwood as it's no reflective of reality at all
11:46:56 sean-k-mooney right but i think glance or cinder made the change and we just picked it up
11:47:18 lyarwood right and that's fine if they need to test volume migration or multi g-api backends etc
11:47:23 sean-k-mooney e.g. they change the common job definition and we never diabled it again
11:47:29 lyarwood but it shouldn't be the default for all subnode users
11:48:17 sean-k-mooney sure but its jsut that they didnt do it in there job they did it in either the base devstack multinode job or tempest
11:48:36 lyarwood it's in devstack-gate iirc in the feature matrix
11:48:49 lyarwood that builds the enabled services for subnodes etc
11:49:18 sean-k-mooney ok so ya if this was zuulv3 then it would be simpelr
11:49:53 lyarwood yeah indeed
11:50:00 sean-k-mooney with devstack gate you dont have the same level of contol in project repos to alter the job
11:50:17 sean-k-mooney i guess thats is another thing to alter in your migration
11:50:39 lyarwood right sorry my brain has just caught up to where I was 9 weeks ago on this
11:50:57 lyarwood and the need to migrate to control this easily per multinode job
11:51:05 lyarwood friday--
11:55:13 sean-k-mooney but but fridays are good :(
11:55:24 lyarwood after 5:30
11:55:36 lyarwood I'm just burnt out today
11:55:48 sean-k-mooney then have a lazy friday
11:56:34 stephenfin bauzas: You about? Want to rubberstamp this for me, please? https://review.opendev.org/#/c/713611/
12:23:08 lyarwood gibi: https://review.opendev.org/#/c/714057/ if you have bandwidth can you check this fup to the nova-live-migration fixes yesterday.
12:26:10 lyarwood elod: https://review.opendev.org/#/q/topic:bug/1867380+(status:open+OR+status:merged) - would you mind going through the rest of this series so we can unblock stable/pike
12:29:16 elod lyarwood: just commented one
12:30:36 elod lyarwood: but otherwise it would be good to push through these soon :) and thanks for the fixes!
12:30:41 openstackgerrit Lee Yarwood proposed openstack/nova stable/rocky: Replace ansible --sudo with --become in live_migration/hooks scripts https://review.opendev.org/713839
12:30:41 openstackgerrit Lee Yarwood proposed openstack/nova stable/rocky: nova-live-migration: Wait for n-cpu services to come up after configuring Ceph https://review.opendev.org/713840
12:30:44 lyarwood elod: yup thanks, addressed above ^
12:31:15 elod lyarwood: cool, thanks!
12:40:52 artom stephenfin, yep, thanks - lemme coffee up and read that
12:45:59 gibi lyarwood: done. Thanks
13:31:15 luyao dansmith, artom : are you about?
13:31:33 artom lyarwood, I am, dansmith probably isn't, given the time zone
13:32:19 artom Err, luyao ^^
13:33:06 luyao artom: I replied on https://review.opendev.org/#/c/687856, could you look it again
13:35:04 artom luyao, is there a specific question you'd like answered though? Reading the conversations quickly, it looks like the ball is in your court to update the patch
13:35:44 openstackgerrit Kevin Zhao proposed openstack/nova master: Add default cpu model for aarch64 https://review.opendev.org/709494
13:41:34 luyao artom: I don't update my patch since dansmith have some concern on moving cleanup before rpc call and some other details I don't know which solustion is better.
13:42:27 luyao artom: I saw you mentioned functional test, I run your live migration functional test on top of my patch, they runs well
13:58:07 artom luyao, yeah, the fact that the functional test is passing is reassuring (and thanks for the review - I have it fixed locally, will push a new version soon)
14:13:28 luyao artom: If you still have questions or concerns about my reply on vpmem live migration , please leave comments there. Or you can ping me in IRC, I'll be online for about two hours I can response promply during this time.
14:13:42 artom luyao, ok, thanks
14:26:46 openstackgerrit Merged openstack/nova master: libvirt: Correctly resize encrypted LUKSv1 volumes https://review.opendev.org/706900
14:34:07 openstackgerrit Artom Lifshitz proposed openstack/nova master: Increase code reuse in test_numa_servers https://review.opendev.org/714125
14:34:31 artom stephenfin, ^^ quick side-fix for you
14:34:48 artom (as I work on NUMA LM func tests)
14:44:00 luyao dansmith: Are you about?
14:44:18 dansmith luyao: yes, but I don't really want to argue about moving that rpc call
14:45:22 dansmith I've read your comments, I understand that it may not matter right now, but what I said is still my opinion: that changing the order of critical points of synchronization is not something to take lightly
14:45:31 dansmith so, my opinion is still the same
14:51:25 luyao dansmith: it will not break anything, so I don't understand what's the risk
14:52:39 luyao dansmith: I try other solution which alex_xu help clarifing, what's you comments about those solusion
14:54:10 alex_xu maybe we should move the instance's state update to later, but that across the src and dest node, in the end, we have to deal with the upgrade issue.
14:57:24 belmoreira Hi, I need some help...
14:57:41 belmoreira nova-compute (dedicated to ironic) market all ironic nodes (compute_nodes table) and as consequence removed all the resource_providers/inventories/allocations for those nodes
14:59:09 belmoreira restarting nova-compute and removing the soft delete of the compute_nodes, the resource provider/inventory is created again
14:59:31 belmoreira but not the allocations... I remember that this was removed from nova some time ago
14:59:54 belmoreira any suggestion how I can recreate thousands of allocations?
15:00:35 dansmith belmoreira: I assume heal instance allocations, just a sec
15:00:51 belmoreira I remember that mnaser and bauzas did some kind of tool
15:01:10 belmoreira dansmith I using stein
15:01:15 dansmith in nova-manage, I think there's an instance heal
15:01:18 dansmith oh
15:02:17 bauzas placement-audit you mean ?
15:02:50 belmoreira placement-audit doesn't recreate allocations if they are missing?
15:03:05 dansmith heal_allocations is the subcmd
15:03:25 dansmith belmoreira: that was in stein
15:03:34 mnaser yeah heal allocations is the thing you need to add missing entries, placement audit is what you need to remove extra entries
15:04:54 belmoreira thanks all, let me have a look
15:28:23 mriedem i think https://review.opendev.org/#/c/678100/ is still relevant to ^ for the compute side delete of allocations there
15:28:26 mriedem but it needs love
15:33:05 openstackgerrit Balazs Gibizer proposed openstack/nova stable/pike: Avoid circular reference during serialization https://review.opendev.org/714148
15:39:19 belmoreira thanks dansmith, mnaser, bauzas. 4305 allocations recreated! In Stein it goes through all deployment. Needed to patch it to do it per cell (this will be a great addition to master).
15:39:28 belmoreira For some reason nova deleted all the ironic resource providers/inventories/allocations.
15:39:44 belmoreira I'm investigating what trigger this... no idea for now... I will let you know when I have more info
15:40:30 dansmith I thought it does do it per cell in master, but I'd have to go back and look
15:42:56 belmoreira it iterates per cell but is not possible to select the cell
15:43:44 dansmith ah okay
15:51:38 mnaser belmoreira: changed hostname or compute node by any chance ?
15:55:32 belmoreira mnaser maybe... I'm configuring "conductor_groups" for ironic. To avoid this precise issue I configured only one conductor group to keep the same nova-compute... and move from there. Still looking... for now I don't understand what went wrong
15:55:58 mnaser belmoreira: maybe check the compute nodes table
15:56:42 belmoreira mnaser the compute_nodes table had all the resources deleted
15:59:12 belmoreira mnaser to recover I made the entries available again (deleted=0, deleted_at=NULL)
15:59:36 belmoreira new entries are not created because the uuid conflict
15:59:49 mnaser belmoreira: pretty sure you might look at the compute logs and see that it was deleting them

Earlier   Later