| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-19 | |||
| 19:47:30 | SteelyDan | heh | |
| 19:47:33 | melwitt | as one does | |
| 19:47:54 | sean-k-mooney | mriedem: oh i actully have heard the last link you posted before | |
| 19:55:28 | openstackgerrit | Merged openstack/nova master: Add regression test for bug 1797580 https://review.openstack.org/610088 | |
| 19:55:28 | openstack | bug 1797580 in OpenStack Compute (nova) rocky "NoValidHost during live migration after cold migrating to a specified host" [High,In progress] https://launchpad.net/bugs/1797580 - Assigned to Matt Riedemann (mriedem) | |
| 19:56:13 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Add regression test for bug 1797580 https://review.openstack.org/611944 | |
| 19:56:14 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Don't persist RequestSpec.requested_destination https://review.openstack.org/611945 | |
| 19:57:17 | mriedem | SteelyDan: remember this? https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L4310 | |
| 19:57:50 | melwitt | wah wah | |
| 19:57:54 | mriedem | it's pretty safe to assume that all cold/live migrations are migration-based allocations now right? | |
| 19:58:01 | mriedem | b/c that was added to compute in queens | |
| 19:58:25 | mriedem | i'm wondering if we can start rolling those compat shims back, including https://github.com/openstack/nova/blob/master/nova/conductor/tasks/migrate.py#L102 | |
| 19:58:33 | mriedem | and just assume compute is new enough to always send a migration record | |
| 19:58:41 | mriedem | and do that hot swap action | |
| 19:59:07 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Use oslo_db.sqlalchemy.test_fixtures https://review.openstack.org/609352 | |
| 19:59:24 | mriedem | i think that's also safe because of https://github.com/openstack/nova/blob/master/nova/compute/rpcapi.py#L716 | |
| 19:59:26 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (3) https://review.openstack.org/574104 | |
| 19:59:29 | mriedem | we're unconditionally sending the migration record | |
| 19:59:44 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4) https://review.openstack.org/574106 | |
| 19:59:45 | SteelyDan | mriedem: yeah | |
| 19:59:51 | SteelyDan | mriedem: I leave those TODOs for others | |
| 19:59:57 | mriedem | tl;dr i would like to rip that code out before landing gibi's https://review.openstack.org/#/c/606050/ which breaks resize to same host allocatoins if we don't have the source allocations on the migration record | |
| 20:00:02 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5) https://review.openstack.org/574110 | |
| 20:00:16 | mriedem | ok cool | |
| 20:00:26 | mriedem | the other thing with that was caching scheduler which is also queued up for death | |
| 20:00:34 | melwitt | that would be a funny TODO. "TODO(dan): Other people remove this in Rocky" | |
| 20:00:44 | mriedem | SteelyDan: i have ont yet | |
| 20:00:45 | mriedem | *not yet | |
| 20:00:45 | SteelyDan | melwitt: next time. I promise. | |
| 20:00:53 | melwitt | awesome | |
| 20:01:06 | SteelyDan | whoa, errant /me in there, sorry | |
| 20:03:03 | cfriesen | ephemeral disks are supposed to last the life of the instance, right? | |
| 20:03:49 | openstackgerrit | Merged openstack/nova master: Don't persist RequestSpec.requested_destination https://review.openstack.org/610098 | |
| 20:04:46 | sean-k-mooney | cfriesen: yes and know | |
| 20:05:04 | sean-k-mooney | cfriesen: i dont know if they are ment to be copied if we move the guest or not | |
| 20:05:31 | sean-k-mooney | cfriesen: they are ment to exsits for the life of an insntace on a singel host definetly. | |
| 20:06:14 | sean-k-mooney | cfriesen: resize is totally undefiend what happens. live/coldmigrate i think we copy them but havent checked that is just my intuition | |
| 20:07:13 | sean-k-mooney | cfriesen: similarly for shelve unshelve not sure we persit tiem. it might be driver specific | |
| 20:07:55 | cfriesen | sean-k-mooney: hmm...I think this wording in the compute API has changed since I looked at it: "Ephemeral disks may be written over on server state changes. So should only be used as a scratch space for applications that are aware of its limitations." | |
| 20:09:18 | sean-k-mooney | cfriesen: ya root disk will be perserved obviourly but adtionl epheral disk i think its totally up to the driver on what state chages they are preserved and when they are recreated | |
| 20:09:37 | SteelyDan | sean-k-mooney: I don't think any of those things are right | |
| 20:09:59 | SteelyDan | I don't think it's well defined at all, but the original assumption was that you couldn't even rely on them across start/stop cycles on your instance | |
| 20:10:26 | cfriesen | on the other hand, nova/doc/source/user/flavors.rst says "Ephemeral disks offer machine local disk storage linked to the lifecycle of a | |
| 20:10:26 | cfriesen | VM instance. When a VM is terminated, all data on the ephemeral disk is lost." | |
| 20:10:29 | sean-k-mooney | SteelyDan: from an api persectiv i would totally aggree. | |
| 20:11:18 | sean-k-mooney | cfriesen: yes but terminated is not stopped | |
| 20:11:30 | cfriesen | okay...I had been thinking that they were supposed to be preserved over everything except termination. | |
| 20:11:40 | cfriesen | but it sounds like that was incorrect | |
| 20:11:42 | sean-k-mooney | SteelyDan: i think the libvirt driver preservs the ephemeral disk in more cases then its required too | |
| 20:11:44 | SteelyDan | I'm quite sure they're not included in any snapshot, so not for shelve | |
| 20:11:49 | SteelyDan | sean-k-mooney: yes | |
| 20:11:56 | melwitt | you mean ephemeral in the flavor, not ephemeral like a normal local disk of any instance without ephemeral in the flavor | |
| 20:12:03 | cfriesen | melwitt: yes | |
| 20:12:09 | melwitt | gotcha | |
| 20:12:59 | cfriesen | okay. that simplifies my life, though not the end user's. :) | |
| 20:32:16 | sean-k-mooney | cfriesen: so as far as i can tell this is what determins what disks are migrated https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L7208-L7247 | |
| 20:33:48 | cfriesen | sean-k-mooney: for live migration specifically | |
| 20:33:57 | sean-k-mooney | cfriesen: so for libvirt i think, that will include the ephermeral disk on live migration | |
| 20:34:01 | sean-k-mooney | yes | |
| 20:34:31 | sean-k-mooney | for rebuild/resize/cold migration i think we dont copy them | |
| 20:34:40 | cfriesen | the case I was looking into was a resize, followed by a resize-revert. | |
| 20:35:37 | sean-k-mooney | i can take a look. it makes sense that we copyt them on live migrate as that does not effect the life time of the instance as long as it succeeds | |
| 20:38:54 | sean-k-mooney | cfriesen: so for resize we copy the image here https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L8309-L8343 but i need to chec if the ephermeral disks are in elf._get_instance_disk_info( | |
| 20:40:34 | cfriesen | sean-k-mooney: don't waste time on my account, if the compute API says it can change, that's good enough for me. | |
| 20:40:41 | sean-k-mooney | cfriesen: it looks like its looping over all the disks too https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L7986 | |
| 20:42:22 | sean-k-mooney | the code is very similar to the live migration code, they shoudl proably be refactored to gether but is also differnt enough that im not sure it does the same thing exactly | |
| 20:46:02 | sean-k-mooney | cfriesen: ya from an api persective it can. libvirt appears to keep them for cold and live migrate. i would assume they are not kept for rebuild,shelve and evacuate. i also think libvirt may be expanding the existing ephmeral disk in resize | |
| 20:47:34 | sean-k-mooney | cfriesen: the libvirt direver is specifclaly checking that we are not resizeing the ephermerl disk down https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L8268-L8275 implying that its not recreateing them but may be expanding them on resize | |
| 20:58:41 | boden | anyone heard about a "oslo_db.exception.DBNonExistentTable" cropping up across projects? Appears to have cropped up around 10/10 http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22oslo_db.exception.DBNonExistentTable%5C%22 and includes neutron, nova and others | |
| 21:15:39 | mriedem | boden: those are unit/functional tests, | |
| 21:15:47 | mriedem | and 10 days is as far back as logstash goes | |
| 21:15:54 | mriedem | so 10/10 is probably not really when it started | |
| 21:16:00 | mriedem | that's just how far back we have logs | |
| 21:16:13 | mriedem | it's showing patches in pike | |
| 21:16:39 | mriedem | and also shows up in successful job runs, so most likely unrelated to anything that's failing | |
| 21:17:52 | boden | mriedem perhaps that's the case for nova (I haven't dug there), but it doesnt appear to be the case for all others; there are valid failures | |
| 21:21:41 | sean-k-mooney | mriedem: boden is this realated why we have to loop for nova-manage out of interest | |
| 21:21:51 | sean-k-mooney | e.g. why wer did https://review.openstack.org/#/c/608091/ | |
| 21:24:57 | sean-k-mooney | actully looking at the logs no this is unrealted | |
| 21:35:46 | boden | yeah my bad... I assumed nova was failing without digging... we have been getting this error on some other projects randomly; appears to be memory/resource related | |
| 21:50:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop legacy cold migrate allocation compat code https://review.openstack.org/611970 | |
| 21:52:10 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop legacy cold migrate allocation compat code https://review.openstack.org/611970 | |
| 22:18:30 | openstackgerrit | melanie witt proposed openstack/nova master: libvirt: set device address tag only if setting disk unit https://review.openstack.org/611974 | |
| 22:22:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop legacy live migrate allocation compat code https://review.openstack.org/611975 | |
| 22:23:11 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Drop legacy live migrate allocation compat code https://review.openstack.org/611975 | |
| #openstack-nova - 2018-10-20 | |||
| 00:13:23 | openstackgerrit | Merged openstack/nova master: Document each libvirt.sysinfo_serial choice https://review.openstack.org/611426 | |
| 02:40:18 | openstackgerrit | Merged openstack/nova master: Remove the caching the resource on Request object https://review.openstack.org/609403 | |
| 07:24:03 | openstackgerrit | Daniel Abad proposed openstack/nova master: Fix ironic client ironic_url deprecation warning https://review.openstack.org/611872 | |
| #openstack-nova - 2018-10-21 | |||
| 22:40:18 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (6) https://review.openstack.org/574113 | |
| 22:40:32 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (7) https://review.openstack.org/574974 | |
| 22:40:50 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (8) https://review.openstack.org/575311 | |
| #openstack-nova - 2018-10-22 | |||
| 00:07:21 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (9) https://review.openstack.org/575581 | |
| 00:07:36 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (10) https://review.openstack.org/576017 | |
| 00:07:52 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (11) https://review.openstack.org/576018 | |
| 00:08:09 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (12) https://review.openstack.org/576019 | |
| 02:03:52 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (13) https://review.openstack.org/576020 | |
| 02:04:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (14) https://review.openstack.org/576027 | |
| 02:04:19 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (15) https://review.openstack.org/576031 | |
| 02:05:03 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add API ref guideline for body text https://review.openstack.org/605628 | |
| 02:05:19 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.openstack.org/566153 | |