Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-09-19
15:17:36 mriedem it sounds like you're trying to auto-heal/reshape duringa cold migrate so we don't have to run the maybe more expensive reshape-on-compute-startup upgrade thing
15:17:37 bauzas during the migration, we have 2 allocations on two different hosts, right?
15:18:00 bauzas ah shit, that's what I fucked
15:18:07 dansmith um, what? :)
15:18:14 dansmith lol.
15:18:23 mriedem maybe you want to just block az renames while the az has instances?
15:18:26 mriedem that's probably easier
15:18:26 bauzas can we have a migration running, and both services to be restarted ?
15:18:43 mriedem sure
15:18:45 bauzas in that case, it would trigger a reshape non?
15:18:55 mriedem sure
15:18:56 bauzas for both ?
15:19:12 bauzas so in that interim period, we have two allocations, nope ?
15:19:26 mriedem if the allocations have moved from the instance to the migration record on the source host and the source host is restarted and a reshape happens, we should still move the allocations for the migration record
15:19:41 mriedem the reshape in this case happens on distinct provider trees
15:19:42 bauzas that's my point
15:20:00 bauzas so in that case, I have a migration UUID that is the consumer
15:20:03 bauzas on the source host
15:20:06 mriedem if the resize is confirmed, we drop the source node tree allocations for the migration record,
15:20:25 mriedem on revert we drop the target node tree allocations for the instance and move them back to the instance for the source tree
15:20:28 mriedem but the reshape should be ok
15:20:56 bauzas ok, but then I have to figure out the real instance UUID for the migration record
15:21:01 bauzas that's a special case
15:21:06 mriedem the migration consumer is just a fill in so someone doesn't quack quack seat back the source node resources
15:21:33 mriedem honestly i don't know what problem you're trying to solve
15:21:52 mriedem we should probably identify that a problem exists before discussing designs on how to fix it
15:21:57 bauzas probably
15:22:02 bauzas and I'm unclear
15:22:25 bauzas I'll just leave a comment in my patch and I move on
15:28:27 dansmith mnaser: how much of your two minutes is writing and reading that dumpfile on disk? would you prefer we just pipe it between the export and import (optionally?) and maybe tee it out for forensics? might be faster
15:30:58 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove deprecated hide_server_address_states option https://review.openstack.org/603831
15:31:02 mriedem gmann: ^
15:34:51 mnaser dansmith: like 3-5 seconds for the dump. The write took the longest
15:35:48 dansmith mnaser: 3-5 seconds overhead for writing the file? or 3-5 seconds to do the dump and the rest of the minute to write it out?
15:36:00 dansmith on my tiny database it's immeasurable of course
15:36:14 dansmith I get a constant 3.6s to do the dump and import regardless
15:36:35 dansmith mnaser: I have a diff that does it pipely if you want to try it
15:36:48 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove deprecated hide_server_address_states option https://review.openstack.org/603831
15:37:32 dansmith mnaser: https://termbin.co/FXJg
15:38:05 dansmith I also put a "tee $tmpfile |" in the middle of that pipe (hence the comment) but removed it for your test
15:56:09 openstackgerrit Ben Nemec proposed openstack/nova master: WIP: Migrate upgrade checks to oslo.upgradecheck https://review.openstack.org/603499
16:00:26 dansmith tssurya: mriedem shall we meet about cells? we just talked about things last week and melwitt is not around this week
16:01:44 mnaser dansmith: sorry i was in a meeting, the dump was really quick but the push was much slower because of (what i assume) replication
16:01:58 dansmith ah the load, I see
16:02:08 dansmith yeah I'm sure that's the heavy part
16:02:16 dansmith it's heavy on my toy devstack even
16:03:05 dansmith in that case, I'd just say we should leave separate the dump/load like it is now so that one can complete without the other, leaving a file you can just import if you want it
16:05:15 jaypipes stephenfin: so, unless I'm mis-reading your comments on the cpu-resource-tracking spec, you'd actually favor an "opt-in" approach to hyperthread usage. Is that correct? i.e. the virt driver defaults to *not* counting hyperthreads as CPU resources for guests unless some knob is turned on?
16:06:06 stephenfin jaypipes: For instances with dedicated CPUs, yes. However, we need to avoid breaking users
16:06:13 stephenfin So that knob would have to default to on
16:06:58 stephenfin The reason I want that is because a hyperthread != a core and I think the decision to model it as one was a mistake on day one
16:06:59 jaypipes stephenfin: ack. no disagreement from me on that
16:07:12 jaypipes stephenfin: agreed.
16:10:21 openstackgerrit sean mooney proposed openstack/nova-specs master: [WIP] generic device discovery policy https://review.openstack.org/603805
16:20:37 openstackgerrit Merged openstack/nova stable/queens: Fix DB archiver AttributeError due to wrong table name attribute used https://review.openstack.org/599882
16:20:44 openstackgerrit Merged openstack/nova stable/queens: VMware: fix TypeError while get console log https://review.openstack.org/591365
16:20:51 openstackgerrit Merged openstack/nova stable/queens: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/599200
16:28:59 mriedem dansmith: i'm good to skip; plan on starting my poc for cross-cell resize today
16:29:11 dansmith oh boy
16:30:33 openstackgerrit Merged openstack/nova stable/ocata: Return 400 when compute host is not found https://review.openstack.org/590649
16:31:38 stephenfin jaypipes: If you don't have one already, a placement cheatsheet (in PDF and therefore printable) would be super useful
16:31:58 jaypipes stephenfin: also, I agree with you about getting rid of emulator_threads_policy eventually, but is it 100% critical that I update the cpu-resource-tracking spec with information about that?
16:32:24 jaypipes stephenfin: you mean cheatsheet about the REST API?
16:32:29 jaypipes stephenfin: or something else?
16:32:41 jaypipes stephenfin: or you mean a glossary of placement terms?
16:33:26 stephenfin jaypipes: No, it can be be handled separately. However, I imagine you do need to take cpu overhead, which is incremented for each dedicated emulator thread core, into account
16:33:31 openstackgerrit Jack Ding proposed openstack/nova master: Correct instance port binding for rebuilds/reboots https://review.openstack.org/603844
16:33:55 stephenfin jaypipes: Both. Glossary of terms followed by examples of either the REST API or using osc-placement
16:34:17 jaypipes stephenfin: ack
16:34:21 stephenfin jaypipes: Just an idea, obviously. I might even start drafting something myself some point this week
16:34:44 jaypipes stephenfin: cool. let me know when you get to the "what is a consumer?" part...
16:36:43 stephenfin jaypipes: FYI, the reason I brought up the whole "let's kill 'isolate' idea" was that that approach seemed easier than handling the CPU overhead thing *and* it reduced complexity for the user in the process, which feels like one of the goals
16:37:26 stephenfin But again, not a blocker :)
16:39:07 jaypipes stephenfin: yes, agreed. glad it's not a blocker though.
16:39:27 openstackgerrit Sylvain Bauza proposed openstack/nova master: libvirt: mdevs returning parent and vendor PCI info https://review.openstack.org/562304
16:40:21 bauzas jaypipes: stephenfin: can one of you reapprove https://review.openstack.org/562304 ? I just rebased it out of the existing series and I need it for the reshaper change
16:40:52 bauzas I'm surprised I lost my +2s on the rebase but meh
16:41:02 bauzas probably because I changed the commit id
16:41:15 bauzas oh yeah, I changed the message, hence why
16:41:30 jaypipes stephenfin: feel free to re-+W bauzas patch
16:41:44 bauzas jaypipes: thanks
16:42:15 bauzas I need the parenting relationship for reshaping the VGPU resources to the right PGPU RP
16:42:54 stephenfin bauzas: Done
16:43:15 bauzas stephenfin: ack, thanks
17:01:41 openstackgerrit Jay Pipes proposed openstack/nova-specs master: Standardize CPU resource tracking https://review.openstack.org/555081
17:09:08 tobias-urdin ping on a possible upgrade path bug q->r https://bugs.launchpad.net/nova/+bug/1793353
17:09:09 openstack Launchpad bug 1793353 in OpenStack Compute (nova) "broken upgrade path q->r requirement for oslo.db" [Undecided,New]
17:37:11 openstackgerrit David Rabel proposed openstack/nova master: Really use source image format as default for snapshot_image_format https://review.openstack.org/603855
17:51:23 jiteka hello, could someone confirm that migrate feature is working with flavor using vCPU pinning and Numa in Mitaka ?
18:01:58 openstackgerrit Merged openstack/nova-specs master: Propose configurable maximum number of volumes to attach https://review.openstack.org/597306
18:10:49 mriedem jiteka: cfriesen might know
18:11:09 mriedem there were some patches that mellanox and windriver worked on to make that work, but i can't remember which release in which those patches landed
18:11:28 mriedem jiteka: cold migrate i mean, not live migration
18:11:36 mriedem numa/pinned cpus does not work with live migration
18:11:53 mriedem https://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/numa-aware-live-migration.html
18:28:40 openstackgerrit Merged openstack/nova stable/ocata: Add recreate test for RT.stats bug 1784705 https://review.openstack.org/588076
18:28:40 openstack bug 1784705 in OpenStack Compute (nova) ocata "ResourceTracker.stats can leak across multiple ironic nodes" [High,In progress] https://launchpad.net/bugs/1784705 - Assigned to Matt Riedemann (mriedem)
18:31:27 mriedem dansmith: would be good to get this pike backport in https://review.openstack.org/#/c/599883/
18:32:14 dansmith mriedem: hmm, did I do that? ISTR some reasoning there.. maybe sqla version?
18:32:59 mriedem it was originally your code yes, but table.name is standard

Earlier   Later