| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-07 | |||
| 11:01:18 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: [WIP] Bump versions of objects dependent on CellMapping https://review.openstack.org/550432 | |
| 11:02:07 | Spazmotic | It's basically the same reason I overrode that variable, since it is the same way the live code handles itself, but i'll think on it as well | |
| 11:10:21 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec https://review.openstack.org/541290 | |
| 12:13:17 | ibmko | Hi everybody, I would like to ask one question which I am unable to find in documentation: We would like to upgrade our OpenStack with minimal downtime and use live migration for that. I cannot find any information about versions of compute nodes which mutually support live migration. For example - we are currently on Mitaka - is it possible to migrate from mitaka compute node to Queens compute node ? | |
| 12:13:52 | ibmko | *live migrate a VM from mitaka compute node to Queens compute node | |
| 12:18:31 | kashyap | Does anyone know the current status of Nova's port to Python 3? | |
| 12:19:16 | kashyap | Last status (from June-2016) I see is here: https://wiki.openstack.org/wiki/Python3 | |
| 12:20:21 | kashyap | ibmko: That's good question: I don't think Nova (yet) supports N+1 upgrades like that, but I'm not 300% sure. | |
| 12:20:30 | kashyap | lyarwood: ^ See imbko's question | |
| 12:20:44 | kashyap | Assuming you know about it top off your head. | |
| 12:22:30 | lyarwood | ibmko: no, you wouldn't be able to migrate between mitaka and queens computes as that would imply having either a mitaka control plane with queens computes or vice versa in the env. | |
| 12:23:16 | ibmko | lyarwood, what about mitaka -> ocata and then ocata -> queens (having ocata as a temporary "bridge") | |
| 12:26:11 | lyarwood | ibmko: nope, if you're running n-cpu during the upgrade then you can only jump one release at a time. | |
| 12:26:19 | lyarwood | ibmko: https://specs.openstack.org/openstack/tripleo-specs/specs/queens/fast-forward-upgrades.html#high-level-flow | |
| 12:26:38 | lyarwood | ibmko: with FFU where n-cpu is down but the instances are running we are able to jump between as many releases as we like | |
| 12:27:09 | lyarwood | ibmko: as we only bring n-cpu up once we reach the target release, obviously you can't live migrate instances during that time | |
| 12:27:32 | lyarwood | ibmko: but tbh, you shouldn't need to migrate instances if you're only updating n-cpu and the other openstack services | |
| 12:28:57 | ibmko | lyarwood, what is "n-cpu" ? | |
| 12:29:32 | lyarwood | ibmko: nova-compute | |
| 12:30:13 | kashyap | ibmko: Also FFU == Fast Forward Upgrades. (Which lyarwood linked to above) | |
| 12:34:01 | jaypipes | *yawn* ... morning supernovas | |
| 12:34:12 | gibi | jaypipes: good morning | |
| 12:35:26 | gibi | jaypipes: may I recommend a (not so) light read for your morning coffe? I have reworked the strict bandwidth support spec based on the PTG discussion https://review.openstack.org/#/c/502306/ | |
| 12:36:14 | ibmko | lyarwood, with FFU I understand that you can move from version N to version, say N+3 (as I understand it, the upgrade process will still go through all intermediate versions ) but I think that is not my question. If it is the case that hypervisor has to go through reboot, we need to live migrate VMs out of the hypervisor and after upgrading the hypervisor we need to migrate VMs back. Question is, if the live-migration process | |
| 12:36:15 | ibmko | itself will allow live-migration to happen from, say. mitaka node to ocata node | |
| 12:36:51 | jaypipes | gibi: for you? any time :) | |
| 12:37:10 | gibi | jaypipes: :) | |
| 12:37:22 | jaypipes | gibi: https://etherpad.openstack.org/p/rocky-ptg-scheduler-placement-summary <-- collecting my summary thoughts on sched/placement topics from PTG to be emailed to ML. | |
| 12:37:27 | jaypipes | gibi: your comments most welcome. | |
| 12:38:22 | lyarwood | ibmko: you wouldn't be able to run Mitaka and Queens computes on the same control plane | |
| 12:38:26 | gibi | jaypipes: I'm going to add my summar about the bandwidth discussion today | |
| 12:38:31 | lyarwood | ibmko: so you wouldn't be able to live migrate between them | |
| 12:38:59 | jaypipes | gibi: go for it | |
| 12:39:26 | gibi | jaypipes: should I write it under the point 6 in Key topics or somewhere below the main flow of the doc? | |
| 12:40:21 | ibmko | lyarwood, according to this article: https://www.mirantis.com/blog/openstack-upgrade-from-mitaka-to-ocata-across-2-releases-with-mirantis-cloud-platform/ | |
| 12:40:34 | ibmko | "The control plane upgrade is mostly independent of the data plane upgrade, and it does not have to be done all at once. Moreover, as you will see in the demo video, we can upgrade just the control plane to Ocata and still boot instances on Mitaka-based compute nodes." | |
| 12:41:02 | ibmko | the article doesn't give an answer about live migration though | |
| 12:43:34 | lyarwood | ibmko: right, that's also with all computes at the same level | |
| 12:43:50 | jaypipes | gibi: sure | |
| 12:44:16 | lyarwood | ibmko: even that case has zero test coverage upstream | |
| 12:45:10 | lyarwood | ibmko: can't you hold off rebooting the compute nodes until the end of the upgrade assuming you're picking up new kernels, OVS, QEMU etc during the upgrade? | |
| 12:45:53 | ibmko | lyarwood, chmm good point - let me think about it | |
| 12:49:28 | ibmko | ibmko, chmm and when we do this, and then we upgrade QEMU and libvirt, now the question comes down to live migration compatibility among different versions of libvirt/qemu/kernel | |
| 12:50:43 | lyarwood | ibmko: that's fine, libvirt and QEMU support forward migrations from old to new. | |
| 12:53:52 | sticker | fwiw, live migrating an instance from an oldish version of QEMU to new had a bug that caused havoc for windows guests | |
| 12:54:21 | sticker | i think it was anything pre 1.5 to post 1.5 would fail due to a balloon driver bug | |
| 12:57:39 | gibi | jaypipes: done. I added it as point 16 in Agreements and decisions | |
| 12:58:13 | ibmko | lyarwood, thank you, I think we will try and go this direction | |
| 13:00:57 | jaypipes | gibi: ty sir | |
| 13:01:50 | mordred | anybody know - in https://developer.openstack.org/api-ref/compute/#id302 <-- hypervisor_hostname_pattern says "The hypervisor host name or a portion of it. The hypervisor hosts are selected with the host name matching this pattern." | |
| 13:02:04 | mordred | what kind of pattern is it? is it a regex? a glob? | |
| 13:12:56 | ameeda | hello. can you please check this patch for me https://review.openstack.org/#/c/526900/ | |
| 13:12:58 | ameeda | thanks | |
| 13:15:02 | edleafe | mordred: here's where it is used: https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L677-L682 | |
| 13:15:14 | edleafe | mordred: so it's sorta-globbish | |
| 13:15:59 | mordred | edleafe: wow. so it's a passthrough to an SQL like syntax but wrapped in sql % wildcards? | |
| 13:17:38 | mordred | edleafe: meaning hypervisor_hostname_pattern of 'foo' would essentially be equiv to fnmatch.fnmatch('*foo*') - matching the string foo anywhere in the hostnames | |
| 13:17:42 | edleafe | mordred: yeah, pretty much | |
| 13:18:58 | mordred | edleafe: okie. that could stand a doc update - I never would have imagined that's what the input should look like from the words "matching this pattern" | |
| 13:19:33 | mordred | I totally would have tried '*foo*' and '.*foo.*' and been confused why I wasn't getting any results | |
| 13:19:42 | edleafe | mordred: it does seem to suggest regex | |
| 13:20:11 | mordred | yah. really it's not looking for a pattern at all -it's looking for a string that will be matched anywhere in the hostname | |
| 13:42:57 | openstackgerrit | Merged openstack/nova master: Return 400 when compute host is not found https://review.openstack.org/447355 | |
| 13:59:40 | Spazmotic | Morning folks. Let's see if I can fix up these unit tests before the end of my night and maybe finally put this thing to bed :D | |
| 14:07:51 | Spazmotic | I am honestly pretty grateful for this though.. it's giving me good, needed experience in Tox | |
| 14:07:55 | Spazmotic | well unittest | |
| 14:29:46 | openstackgerrit | Brooks Kaminski proposed openstack/nova master: XenAPI: XCP2.1+ Swallow VDI_NOT_IN_MAP Exception https://review.openstack.org/538415 | |
| 14:30:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send soft_delete from context manager https://review.openstack.org/476459 | |
| 14:30:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform missing delete notifications https://review.openstack.org/550488 | |
| 14:30:23 | Spazmotic | ^ jianghuaw_: "IF" you can find some time. Don't stress it though. | |
| 14:30:46 | Spazmotic | Also, thanks for the ideas.. I think the way I compare the result now is better using two variables. | |
| 14:31:02 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform missing delete notifications https://review.openstack.org/410297 | |
| 14:31:02 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send soft_delete from context manager https://review.openstack.org/476459 | |
| 14:38:31 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Enhance service restart in functional env https://review.openstack.org/512552 | |
| 14:38:32 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reproduce bug 1724172 in the functional test env https://review.openstack.org/512553 | |
| 14:38:33 | openstack | bug 1724172 in OpenStack Compute (nova) "Allocation of an evacuated instance is not cleaned on the source host if instance is not defined on the hypervisor" [Undecided,In progress] https://launchpad.net/bugs/1724172 - Assigned to Balazs Gibizer (balazs-gibizer) | |
| 14:38:33 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: cleanup evacuated instances not on hypervisor https://review.openstack.org/512623 | |
| 14:44:38 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Save admin password to sysmeta in libvirt driver https://review.openstack.org/550489 | |
| 14:46:27 | mriedem | lyarwood: bauzas: we need to do a queens 17.0.1 release for a few fixes, can you review https://review.openstack.org/#/c/550259/ and https://review.openstack.org/#/c/550221/ (there is one below this too) | |
| 14:46:35 | bauzas | ack | |
| 14:47:46 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: cover migration cases with functional tests https://review.openstack.org/493865 | |
| 14:50:40 | bauzas | mriedem: booom | |
| 14:51:37 | mriedem | thanks | |
| 14:55:56 | openstackgerrit | Merged openstack/nova master: Add a nova-caching-scheduler job to the experimental queue https://review.openstack.org/539260 | |
| 14:56:48 | stephenfin | kashyap: Don't know if you got your answer, but theoretically Python 3 support is complete | |
| 14:56:54 | dansmith | jaypipes: fwiw, table.truncate() doesn't seem to work, at least in our functional tests | |
| 14:56:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Run post-test archive against cell1 https://review.openstack.org/550194 | |
| 14:56:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Add simple db purge command https://review.openstack.org/550171 | |
| 14:56:56 | openstackgerrit | Dan Smith proposed openstack/nova master: Add --purge helper flag to archive_deleted_rows https://review.openstack.org/550182 | |
| 14:57:13 | mriedem | stephenfin: ceph doesn't work on py3 if you're taking notes | |
| 14:57:20 | mriedem | well, librados doesn't | |
| 14:57:24 | kashyap | stephenfin: I see; "theoretically complete" means, in practise it is not? :D | |
| 14:57:37 | stephenfin | kashyap: All unit and functional tests are passing with Python 3. However, I doubt anyone is running a Python 3 deployment | |
| 14:57:38 | mriedem | kashyap: nova works on py3 | |
| 14:57:39 | kashyap | mriedem: How about eventlet? | |
| 14:57:41 | stephenfin | mriedem: It does now? | |
| 14:57:50 | stephenfin | I submitted patches saying as much a few months back | |
| 14:57:53 | mriedem | the dependent libraries, based on your configuration, might not work on py3 | |
| 14:58:12 | mriedem | last i checked the py3 ceph job wasn't working, melwitt was investigating that | |
| 14:58:14 | kashyap | mriedem: I see, is there a test job with PY3? | |