| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-09-19 | |||
| 15:18:26 | mriedem | that's probably easier | |
| 15:18:43 | mriedem | sure | |
| 15:18:45 | bauzas | in that case, it would trigger a reshape non? | |
| 15:18:55 | mriedem | sure | |
| 15:18:56 | bauzas | for both ? | |
| 15:19:12 | bauzas | so in that interim period, we have two allocations, nope ? | |
| 15:19:26 | mriedem | if the allocations have moved from the instance to the migration record on the source host and the source host is restarted and a reshape happens, we should still move the allocations for the migration record | |
| 15:19:41 | mriedem | the reshape in this case happens on distinct provider trees | |
| 15:19:42 | bauzas | that's my point | |
| 15:20:00 | bauzas | so in that case, I have a migration UUID that is the consumer | |
| 15:20:03 | bauzas | on the source host | |
| 15:20:06 | mriedem | if the resize is confirmed, we drop the source node tree allocations for the migration record, | |
| 15:20:25 | mriedem | on revert we drop the target node tree allocations for the instance and move them back to the instance for the source tree | |
| 15:20:28 | mriedem | but the reshape should be ok | |
| 15:20:56 | bauzas | ok, but then I have to figure out the real instance UUID for the migration record | |
| 15:21:01 | bauzas | that's a special case | |
| 15:21:06 | mriedem | the migration consumer is just a fill in so someone doesn't quack quack seat back the source node resources | |
| 15:21:33 | mriedem | honestly i don't know what problem you're trying to solve | |
| 15:21:52 | mriedem | we should probably identify that a problem exists before discussing designs on how to fix it | |
| 15:21:57 | bauzas | probably | |
| 15:22:02 | bauzas | and I'm unclear | |
| 15:22:25 | bauzas | I'll just leave a comment in my patch and I move on | |
| 15:28:27 | dansmith | mnaser: how much of your two minutes is writing and reading that dumpfile on disk? would you prefer we just pipe it between the export and import (optionally?) and maybe tee it out for forensics? might be faster | |
| 15:30:58 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove deprecated hide_server_address_states option https://review.openstack.org/603831 | |
| 15:31:02 | mriedem | gmann: ^ | |
| 15:34:51 | mnaser | dansmith: like 3-5 seconds for the dump. The write took the longest | |
| 15:35:48 | dansmith | mnaser: 3-5 seconds overhead for writing the file? or 3-5 seconds to do the dump and the rest of the minute to write it out? | |
| 15:36:00 | dansmith | on my tiny database it's immeasurable of course | |
| 15:36:14 | dansmith | I get a constant 3.6s to do the dump and import regardless | |
| 15:36:35 | dansmith | mnaser: I have a diff that does it pipely if you want to try it | |
| 15:36:48 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove deprecated hide_server_address_states option https://review.openstack.org/603831 | |
| 15:37:32 | dansmith | mnaser: https://termbin.co/FXJg | |
| 15:38:05 | dansmith | I also put a "tee $tmpfile |" in the middle of that pipe (hence the comment) but removed it for your test | |
| 15:56:09 | openstackgerrit | Ben Nemec proposed openstack/nova master: WIP: Migrate upgrade checks to oslo.upgradecheck https://review.openstack.org/603499 | |
| 16:00:26 | dansmith | tssurya: mriedem shall we meet about cells? we just talked about things last week and melwitt is not around this week | |
| 16:01:44 | mnaser | dansmith: sorry i was in a meeting, the dump was really quick but the push was much slower because of (what i assume) replication | |
| 16:01:58 | dansmith | ah the load, I see | |
| 16:02:08 | dansmith | yeah I'm sure that's the heavy part | |
| 16:02:16 | dansmith | it's heavy on my toy devstack even | |
| 16:03:05 | dansmith | in that case, I'd just say we should leave separate the dump/load like it is now so that one can complete without the other, leaving a file you can just import if you want it | |
| 16:05:15 | jaypipes | stephenfin: so, unless I'm mis-reading your comments on the cpu-resource-tracking spec, you'd actually favor an "opt-in" approach to hyperthread usage. Is that correct? i.e. the virt driver defaults to *not* counting hyperthreads as CPU resources for guests unless some knob is turned on? | |
| 16:06:06 | stephenfin | jaypipes: For instances with dedicated CPUs, yes. However, we need to avoid breaking users | |
| 16:06:13 | stephenfin | So that knob would have to default to on | |
| 16:06:58 | stephenfin | The reason I want that is because a hyperthread != a core and I think the decision to model it as one was a mistake on day one | |
| 16:06:59 | jaypipes | stephenfin: ack. no disagreement from me on that | |
| 16:07:12 | jaypipes | stephenfin: agreed. | |
| 16:10:21 | openstackgerrit | sean mooney proposed openstack/nova-specs master: [WIP] generic device discovery policy https://review.openstack.org/603805 | |
| 16:20:37 | openstackgerrit | Merged openstack/nova stable/queens: Fix DB archiver AttributeError due to wrong table name attribute used https://review.openstack.org/599882 | |
| 16:20:44 | openstackgerrit | Merged openstack/nova stable/queens: VMware: fix TypeError while get console log https://review.openstack.org/591365 | |
| 16:20:51 | openstackgerrit | Merged openstack/nova stable/queens: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/599200 | |
| 16:28:59 | mriedem | dansmith: i'm good to skip; plan on starting my poc for cross-cell resize today | |
| 16:29:11 | dansmith | oh boy | |
| 16:30:33 | openstackgerrit | Merged openstack/nova stable/ocata: Return 400 when compute host is not found https://review.openstack.org/590649 | |
| 16:31:38 | stephenfin | jaypipes: If you don't have one already, a placement cheatsheet (in PDF and therefore printable) would be super useful | |
| 16:31:58 | jaypipes | stephenfin: also, I agree with you about getting rid of emulator_threads_policy eventually, but is it 100% critical that I update the cpu-resource-tracking spec with information about that? | |
| 16:32:24 | jaypipes | stephenfin: you mean cheatsheet about the REST API? | |
| 16:32:29 | jaypipes | stephenfin: or something else? | |
| 16:32:41 | jaypipes | stephenfin: or you mean a glossary of placement terms? | |
| 16:33:26 | stephenfin | jaypipes: No, it can be be handled separately. However, I imagine you do need to take cpu overhead, which is incremented for each dedicated emulator thread core, into account | |
| 16:33:31 | openstackgerrit | Jack Ding proposed openstack/nova master: Correct instance port binding for rebuilds/reboots https://review.openstack.org/603844 | |
| 16:33:55 | stephenfin | jaypipes: Both. Glossary of terms followed by examples of either the REST API or using osc-placement | |
| 16:34:17 | jaypipes | stephenfin: ack | |
| 16:34:21 | stephenfin | jaypipes: Just an idea, obviously. I might even start drafting something myself some point this week | |
| 16:34:44 | jaypipes | stephenfin: cool. let me know when you get to the "what is a consumer?" part... | |
| 16:36:43 | stephenfin | jaypipes: FYI, the reason I brought up the whole "let's kill 'isolate' idea" was that that approach seemed easier than handling the CPU overhead thing *and* it reduced complexity for the user in the process, which feels like one of the goals | |
| 16:37:26 | stephenfin | But again, not a blocker :) | |
| 16:39:07 | jaypipes | stephenfin: yes, agreed. glad it's not a blocker though. | |
| 16:39:27 | openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: mdevs returning parent and vendor PCI info https://review.openstack.org/562304 | |
| 16:40:21 | bauzas | jaypipes: stephenfin: can one of you reapprove https://review.openstack.org/562304 ? I just rebased it out of the existing series and I need it for the reshaper change | |
| 16:40:52 | bauzas | I'm surprised I lost my +2s on the rebase but meh | |
| 16:41:02 | bauzas | probably because I changed the commit id | |
| 16:41:15 | bauzas | oh yeah, I changed the message, hence why | |
| 16:41:30 | jaypipes | stephenfin: feel free to re-+W bauzas patch | |
| 16:41:44 | bauzas | jaypipes: thanks | |
| 16:42:15 | bauzas | I need the parenting relationship for reshaping the VGPU resources to the right PGPU RP | |
| 16:42:54 | stephenfin | bauzas: Done | |
| 16:43:15 | bauzas | stephenfin: ack, thanks | |
| 17:01:41 | openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Standardize CPU resource tracking https://review.openstack.org/555081 | |
| 17:09:08 | tobias-urdin | ping on a possible upgrade path bug q->r https://bugs.launchpad.net/nova/+bug/1793353 | |
| 17:09:09 | openstack | Launchpad bug 1793353 in OpenStack Compute (nova) "broken upgrade path q->r requirement for oslo.db" [Undecided,New] | |
| 17:37:11 | openstackgerrit | David Rabel proposed openstack/nova master: Really use source image format as default for snapshot_image_format https://review.openstack.org/603855 | |
| 17:51:23 | jiteka | hello, could someone confirm that migrate feature is working with flavor using vCPU pinning and Numa in Mitaka ? | |
| 18:01:58 | openstackgerrit | Merged openstack/nova-specs master: Propose configurable maximum number of volumes to attach https://review.openstack.org/597306 | |
| 18:10:49 | mriedem | jiteka: cfriesen might know | |
| 18:11:09 | mriedem | there were some patches that mellanox and windriver worked on to make that work, but i can't remember which release in which those patches landed | |
| 18:11:28 | mriedem | jiteka: cold migrate i mean, not live migration | |
| 18:11:36 | mriedem | numa/pinned cpus does not work with live migration | |
| 18:11:53 | mriedem | https://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/numa-aware-live-migration.html | |
| 18:28:40 | openstack | bug 1784705 in OpenStack Compute (nova) ocata "ResourceTracker.stats can leak across multiple ironic nodes" [High,In progress] https://launchpad.net/bugs/1784705 - Assigned to Matt Riedemann (mriedem) | |
| 18:28:40 | openstackgerrit | Merged openstack/nova stable/ocata: Add recreate test for RT.stats bug 1784705 https://review.openstack.org/588076 | |
| 18:31:27 | mriedem | dansmith: would be good to get this pike backport in https://review.openstack.org/#/c/599883/ | |
| 18:32:14 | dansmith | mriedem: hmm, did I do that? ISTR some reasoning there.. maybe sqla version? | |
| 18:32:59 | mriedem | it was originally your code yes, but table.name is standard | |
| 18:44:04 | mriedem | dansmith: does anything jump out at you for the source of the IndexError in this failure? http://logs.openstack.org/72/600372/1/gate/openstack-tox-py35/dcfd363/job-output.txt.gz | |
| 18:44:56 | mriedem | looks like something between oslo.db/sqla/pymysql/eventlet switches context and we timeout after some huge amount of time | |
| 18:45:07 | mriedem | nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_models_sync [664.512994s] ... FAILED | |
| 18:45:11 | mriedem | nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_models_sync [39.644814s] ... ok | |
| 18:45:30 | mriedem | clearly we're not just missing some short timeout window | |
| 18:46:09 | dansmith | still waiting for the damn log to load | |
| 18:48:14 | dansmith | wow | |