| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-14 | |||
| 15:20:14 | mriedem | did/can | |
| 15:20:37 | mriedem | i assume keystoneauth1 has some magic that we can use to just replace this | |
| 15:20:41 | mriedem | or osc-lib | |
| 15:21:04 | jmlowe | osc works just fine | |
| 15:21:15 | mriedem | right, but i'd like to fix nova client too :) | |
| 15:21:37 | jmlowe | does help when trying to use heat | |
| 15:21:52 | mriedem | yeah, if you can report a novaclient bug that would be awesome | |
| 15:26:13 | aarents | mriedem: sahid yes our needs is to convert host and instance on it from qcow2 to raw in "live". we can run live local storage migration (blockcopy) but during sometime for adminitrative purpose we need to have live-migration working even if they are not yet converted | |
| 15:27:05 | mriedem | aarents: if live migrating from qcow2 to raw is safe, then we could probably build that into a pre-live migration check for allowable conversions | |
| 15:27:11 | mriedem | so there would be 2 rules: | |
| 15:27:16 | mriedem | 1. exact image backend type match - ok | |
| 15:27:24 | mriedem | 2. qcow2 -> raw: ok? | |
| 15:27:30 | aarents | but I will dig into migrate_data / check_can_live_migrate_destination as suggested to avoid incompatible migration | |
| 15:27:32 | mriedem | everything else is not a match and we need to find another dest host | |
| 15:27:40 | aarents | yes | |
| 15:27:44 | mriedem | i honestly don't know what dragons lurk on #2 | |
| 15:28:05 | mriedem | but we're also trying to do this same kind of thing with live migrating instances between hosts with different networking backends | |
| 15:28:08 | mriedem | i.e. ovs -> lb and back | |
| 15:28:28 | mriedem | https://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/neutron-new-port-binding-api.html | |
| 15:28:47 | mriedem | ^ gets the vif info for the dest host and uses it to build the guest xml from the source host prior to the transfer | |
| 15:31:40 | jmlowe | https://bugs.launchpad.net/python-novaclient/+bug/1776928 | |
| 15:31:41 | openstack | Launchpad bug 1776928 in python-novaclient "Nova endpoint for v2.1 not parsed correctly, tries http instead of https" [Undecided,New] | |
| 15:34:12 | sahid | mriedem: it would be great to do that for vhostuser server mode to client mode... basically when OVS is updated from a version which does not support dpdkvhostuserclient to one which supports it | |
| 15:34:56 | mriedem | jmlowe: thanks | |
| 15:35:25 | mriedem | sahid: maybe you can help review my libvirt driver changes in that series then, that's where they are stalled now | |
| 15:35:29 | mriedem | it's also in a runway atm | |
| 15:35:42 | mriedem | https://review.openstack.org/#/c/558001/ is the bottom of the series right now | |
| 15:35:45 | sahid | mriedem: sure, can you share to me a link | |
| 15:35:47 | sahid | ok | |
| 15:35:51 | mriedem | libvirt starts here https://review.openstack.org/#/c/566932/ | |
| 15:47:46 | openstackgerrit | Merged openstack/nova master: Add enhanced KVM storage QoS quotas https://review.openstack.org/558530 | |
| 15:52:36 | mriedem | woot | |
| 15:52:38 | mriedem | simondodsley: ^ | |
| 15:53:01 | sahid | mriedem, about aarents's patch, so you are OK to accept conversion qcow2->raw since it's working but we still have to add that check in pre-live-migration to raise a MigrationPreCheckError is we try to convert from raw to qcow2 | |
| 15:53:06 | sahid | right? | |
| 15:57:38 | mriedem | sahid: that's what i think we should do, | |
| 15:57:50 | mriedem | like i said, i can't say there aren't side effects of cow -> raw conversion after the fact | |
| 15:58:10 | mriedem | like once you live migrate the instance, then try to snapshot or something else, does that work | |
| 15:59:28 | aarents | there is two solution: | |
| 15:59:29 | aarents | 1) allow qcow2 instance to be live-migrated on raw host but not converted. target host have a mix of qcow2 & raw instance. | |
| 15:59:31 | aarents | 2) allow qcow2 instance to be live-migrated but converted (bigger patch) | |
| 16:01:55 | aarents | mriedem: are you open on 2 solution or only on solution 2 ? | |
| 16:02:55 | mriedem | aarents: i think 1 is a can of worms | |
| 16:03:02 | mriedem | does 'can of worms' translate? | |
| 16:03:32 | sahid | bucket of bugs i guess | |
| 16:03:34 | mriedem | because that means, the code is going to have things like, "if CONF.libvirt.images_type OR SPECIAL CASE LIVE MIGRATION" | |
| 16:10:18 | openstackgerrit | Merged openstack/nova master: placement: Allocation.consumer field https://review.openstack.org/565405 | |
| 16:13:38 | aarents | mriedem: we made test in our environnement, for solution one, all main action seems to work great for qcow2 instance one a raw host resize/suspend/hardreboot | |
| 16:23:37 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add NUMACell.network_info https://review.openstack.org/564439 | |
| 16:23:38 | openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Unchain _get_phynet_info from _get_port_vnic_info https://review.openstack.org/564443 | |
| 16:23:39 | openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Add 'create_resource_requests' to network API https://review.openstack.org/564444 | |
| 16:23:40 | openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Retrieve tunneled status in '_get_network_info' https://review.openstack.org/564445 | |
| 16:23:41 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove NUMATopologyLimits.obj_from_db_obj https://review.openstack.org/537412 | |
| 16:23:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove legacy '_to_dict' functions https://review.openstack.org/537413 | |
| 16:23:43 | openstackgerrit | Stephen Finucane proposed openstack/nova master: hardware: Start accounting for networks in NUMA placement https://review.openstack.org/564448 | |
| 16:23:44 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add RequestSpec.numa_networks https://review.openstack.org/564442 | |
| 16:23:45 | openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Start utilizing RequestSpec.numa_networks https://review.openstack.org/564452 | |
| 16:23:46 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Add '[neutron] physnets' and related options https://review.openstack.org/564440 | |
| 16:23:47 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start populating NUMACell.network_info field https://review.openstack.org/564441 | |
| 16:23:48 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add NUMATopologyLimits.networks https://review.openstack.org/575486 | |
| 16:23:49 | openstackgerrit | Stephen Finucane proposed openstack/nova master: DNM: objects: Store RequestSpec.numa_networks https://review.openstack.org/575487 | |
| 16:23:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Remove 'create_pci_requests_for_sriov_ports' https://review.openstack.org/575488 | |
| 16:23:54 | stephenfin | That was the single worst "rebase" of my life | |
| 16:24:04 | stephenfin | Three whole days, gone like that | |
| 16:31:20 | crazik | hello. is it safe to truncate migrations table when all migrations are done? completed/confirmed/error/... | |
| 16:31:51 | mriedem | crazik: as in purge? | |
| 16:31:58 | mriedem | archive then and then purge for deleted instances | |
| 16:32:05 | crazik | i did it. | |
| 16:32:08 | mriedem | nova-manage db archive_deleted_rows --until-complete --verbose --purge | |
| 16:32:18 | mriedem | crazik: if they are archived, then yeah purge is fine | |
| 16:32:31 | crazik | still >400 entries | |
| 16:32:58 | crazik | my question in general is: why should I need this | |
| 16:33:32 | crazik | I think I have some troubles similar to https://bugzilla.redhat.com/show_bug.cgi?id=1403728 | |
| 16:33:34 | openstack | bugzilla.redhat.com bug 1403728 in openstack-nova "Unexpected "Instance not resizing, skipping migration" caused by nova evacuation" [Medium,Closed: errata] - Assigned to ratailor | |
| 16:33:59 | crazik | and wonder what are dependencies on this table. | |
| 16:34:10 | dansmith | mriedem: heh, we have no tests for check_can_live_migrate_destination in rpcapi | |
| 16:34:15 | crazik | if no migration is running now | |
| 16:39:05 | mriedem | dansmith: psh who needs tests | |
| 16:39:20 | mriedem | crazik: why do you need migration records for instances that aren't migrating? | |
| 16:39:33 | mriedem | there is an API to see them for one thing, for like auditing and debugging failures | |
| 16:39:57 | mriedem | there are also places in the code that rely on them existing for making decisions, like bringing up an evacuated host | |
| 16:40:30 | crazik | mriedem: I had some issues with migration and need to exclude one reason: old migrations | |
| 16:44:03 | mriedem | crazik: which release are you on? | |
| 16:44:17 | mriedem | the archive stuff was broken a few releases ago | |
| 16:44:25 | mriedem | or, didn't completely cleanup | |
| 16:44:40 | crazik | Pike | |
| 16:44:57 | crazik | well, at migration between Ocata and Pike | |
| 16:45:13 | crazik | few first compute were ok | |
| 16:45:27 | crazik | but next one: few of running - refused to live migrate | |
| 16:45:32 | crazik | (error status in migration) | |
| 16:46:12 | crazik | and ShutOff -also refused migration. done after nova-compute stop and evacuate-host | |
| 16:46:54 | mriedem | if the migration is failing, it's not likely due to old migration records in thedb | |
| 16:47:11 | mriedem | did you look at the nova-compute logs on each host involved in the live migration? | |
| 16:47:29 | openstackgerrit | Dan Smith proposed openstack/nova master: Make check_can_live_migrate_destination use long_rpc_timeout https://review.openstack.org/575493 | |
| 16:47:31 | dansmith | mriedem: ^ | |
| 16:47:32 | mriedem | could also be scheduler or conductor logs, dependong on where it failed | |
| 16:47:38 | mriedem | dansmith: sweet | |
| 16:47:51 | dansmith | now, I've been up for almost four hours without food, so.. food. | |
| 16:48:05 | mriedem | sustain yourself on a job well done | |
| 16:48:07 | crazik | mriedem: nothing special was there | |