| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-19 | |||
| 19:41:41 | SteelyDan | nice | |
| 19:42:16 | SteelyDan | I enjoyed when this rotated in for me this morning: https://www.youtube.com/watch?v=KCdKBHdPz30 | |
| 19:43:20 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Don't persist RequestSpec.requested_destination https://review.openstack.org/611939 | |
| 19:43:20 | openstack | bug 1797580 in OpenStack Compute (nova) "NoValidHost during live migration after cold migrating to a specified host" [High,In progress] https://launchpad.net/bugs/1797580 - Assigned to Matt Riedemann (mriedem) | |
| 19:43:20 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Add regression test for bug 1797580 https://review.openstack.org/611938 | |
| 19:44:08 | mriedem | i know i'd never do it without the fez on | |
| 19:44:14 | openstackgerrit | Dan Smith proposed openstack/nova master: Minimal construct plumbing for nova list when a cell is down https://review.openstack.org/567785 | |
| 19:44:14 | openstackgerrit | Dan Smith proposed openstack/nova master: Modify get_by_cell_and_project() to get_not_deleted_by_cell_and_projects() https://review.openstack.org/607663 | |
| 19:44:15 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova show when a cell is down https://review.openstack.org/591658 | |
| 19:44:15 | openstackgerrit | Dan Smith proposed openstack/nova master: Refactor scatter-gather utility to return exception objects https://review.openstack.org/607934 | |
| 19:44:16 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova service-list when a cell is down https://review.openstack.org/584829 | |
| 19:44:18 | SteelyDan | heck no | |
| 19:44:50 | melwitt | never heard of either of those. I only know the major steely dan hits | |
| 19:45:13 | mriedem | kid charlemagne is a major hit | |
| 19:45:17 | mriedem | all the major dudes know that | |
| 19:45:18 | SteelyDan | well, | |
| 19:45:28 | SteelyDan | she probably means like reelin' in the years and rikki | |
| 19:45:29 | melwitt | major dudes?? | |
| 19:45:31 | SteelyDan | and dirty work | |
| 19:45:39 | melwitt | yeah and like NO STATIC AT ALLLL | |
| 19:45:41 | mriedem | https://www.youtube.com/watch?v=kND8TRZap8Y | |
| 19:45:53 | SteelyDan | like floyd, the major hits are not very representative of the larger work | |
| 19:45:56 | melwitt | and hey nineteen | |
| 19:46:05 | melwitt | and deacon blues | |
| 19:46:40 | melwitt | lol, of course it was a reference to another song I don't know | |
| 19:47:12 | mriedem | when the sax solo from FM comes on in the car, i always yell at the girls to quit down and crank it way up | |
| 19:47:16 | mriedem | obnoxiously so | |
| 19:47:24 | mriedem | *quiet | |
| 19:47:30 | SteelyDan | heh | |
| 19:47:33 | melwitt | as one does | |
| 19:47:54 | sean-k-mooney | mriedem: oh i actully have heard the last link you posted before | |
| 19:55:28 | openstack | bug 1797580 in OpenStack Compute (nova) rocky "NoValidHost during live migration after cold migrating to a specified host" [High,In progress] https://launchpad.net/bugs/1797580 - Assigned to Matt Riedemann (mriedem) | |
| 19:55:28 | openstackgerrit | Merged openstack/nova master: Add regression test for bug 1797580 https://review.openstack.org/610088 | |
| 19:56:13 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Add regression test for bug 1797580 https://review.openstack.org/611944 | |
| 19:56:14 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Don't persist RequestSpec.requested_destination https://review.openstack.org/611945 | |
| 19:57:17 | mriedem | SteelyDan: remember this? https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L4310 | |
| 19:57:50 | melwitt | wah wah | |
| 19:57:54 | mriedem | it's pretty safe to assume that all cold/live migrations are migration-based allocations now right? | |
| 19:58:01 | mriedem | b/c that was added to compute in queens | |
| 19:58:25 | mriedem | i'm wondering if we can start rolling those compat shims back, including https://github.com/openstack/nova/blob/master/nova/conductor/tasks/migrate.py#L102 | |
| 19:58:33 | mriedem | and just assume compute is new enough to always send a migration record | |
| 19:58:41 | mriedem | and do that hot swap action | |
| 19:59:07 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Use oslo_db.sqlalchemy.test_fixtures https://review.openstack.org/609352 | |
| 19:59:24 | mriedem | i think that's also safe because of https://github.com/openstack/nova/blob/master/nova/compute/rpcapi.py#L716 | |
| 19:59:26 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (3) https://review.openstack.org/574104 | |
| 19:59:29 | mriedem | we're unconditionally sending the migration record | |
| 19:59:44 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4) https://review.openstack.org/574106 | |
| 19:59:45 | SteelyDan | mriedem: yeah | |
| 19:59:51 | SteelyDan | mriedem: I leave those TODOs for others | |
| 19:59:57 | mriedem | tl;dr i would like to rip that code out before landing gibi's https://review.openstack.org/#/c/606050/ which breaks resize to same host allocatoins if we don't have the source allocations on the migration record | |
| 20:00:02 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5) https://review.openstack.org/574110 | |
| 20:00:16 | mriedem | ok cool | |
| 20:00:26 | mriedem | the other thing with that was caching scheduler which is also queued up for death | |
| 20:00:34 | melwitt | that would be a funny TODO. "TODO(dan): Other people remove this in Rocky" | |
| 20:00:44 | mriedem | SteelyDan: i have ont yet | |
| 20:00:45 | SteelyDan | melwitt: next time. I promise. | |
| 20:00:45 | mriedem | *not yet | |
| 20:00:53 | melwitt | awesome | |
| 20:01:06 | SteelyDan | whoa, errant /me in there, sorry | |
| 20:03:03 | cfriesen | ephemeral disks are supposed to last the life of the instance, right? | |
| 20:03:49 | openstackgerrit | Merged openstack/nova master: Don't persist RequestSpec.requested_destination https://review.openstack.org/610098 | |
| 20:04:46 | sean-k-mooney | cfriesen: yes and know | |
| 20:05:04 | sean-k-mooney | cfriesen: i dont know if they are ment to be copied if we move the guest or not | |
| 20:05:31 | sean-k-mooney | cfriesen: they are ment to exsits for the life of an insntace on a singel host definetly. | |
| 20:06:14 | sean-k-mooney | cfriesen: resize is totally undefiend what happens. live/coldmigrate i think we copy them but havent checked that is just my intuition | |
| 20:07:13 | sean-k-mooney | cfriesen: similarly for shelve unshelve not sure we persit tiem. it might be driver specific | |
| 20:07:55 | cfriesen | sean-k-mooney: hmm...I think this wording in the compute API has changed since I looked at it: "Ephemeral disks may be written over on server state changes. So should only be used as a scratch space for applications that are aware of its limitations." | |
| 20:09:18 | sean-k-mooney | cfriesen: ya root disk will be perserved obviourly but adtionl epheral disk i think its totally up to the driver on what state chages they are preserved and when they are recreated | |
| 20:09:37 | SteelyDan | sean-k-mooney: I don't think any of those things are right | |
| 20:09:59 | SteelyDan | I don't think it's well defined at all, but the original assumption was that you couldn't even rely on them across start/stop cycles on your instance | |
| 20:10:26 | cfriesen | VM instance. When a VM is terminated, all data on the ephemeral disk is lost." | |
| 20:10:26 | cfriesen | on the other hand, nova/doc/source/user/flavors.rst says "Ephemeral disks offer machine local disk storage linked to the lifecycle of a | |
| 20:10:29 | sean-k-mooney | SteelyDan: from an api persectiv i would totally aggree. | |
| 20:11:18 | sean-k-mooney | cfriesen: yes but terminated is not stopped | |
| 20:11:30 | cfriesen | okay...I had been thinking that they were supposed to be preserved over everything except termination. | |
| 20:11:40 | cfriesen | but it sounds like that was incorrect | |
| 20:11:42 | sean-k-mooney | SteelyDan: i think the libvirt driver preservs the ephemeral disk in more cases then its required too | |
| 20:11:44 | SteelyDan | I'm quite sure they're not included in any snapshot, so not for shelve | |
| 20:11:49 | SteelyDan | sean-k-mooney: yes | |
| 20:11:56 | melwitt | you mean ephemeral in the flavor, not ephemeral like a normal local disk of any instance without ephemeral in the flavor | |
| 20:12:03 | cfriesen | melwitt: yes | |
| 20:12:09 | melwitt | gotcha | |
| 20:12:59 | cfriesen | okay. that simplifies my life, though not the end user's. :) | |
| 20:32:16 | sean-k-mooney | cfriesen: so as far as i can tell this is what determins what disks are migrated https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L7208-L7247 | |
| 20:33:48 | cfriesen | sean-k-mooney: for live migration specifically | |
| 20:33:57 | sean-k-mooney | cfriesen: so for libvirt i think, that will include the ephermeral disk on live migration | |
| 20:34:01 | sean-k-mooney | yes | |
| 20:34:31 | sean-k-mooney | for rebuild/resize/cold migration i think we dont copy them | |
| 20:34:40 | cfriesen | the case I was looking into was a resize, followed by a resize-revert. | |
| 20:35:37 | sean-k-mooney | i can take a look. it makes sense that we copyt them on live migrate as that does not effect the life time of the instance as long as it succeeds | |
| 20:38:54 | sean-k-mooney | cfriesen: so for resize we copy the image here https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L8309-L8343 but i need to chec if the ephermeral disks are in elf._get_instance_disk_info( | |
| 20:40:34 | cfriesen | sean-k-mooney: don't waste time on my account, if the compute API says it can change, that's good enough for me. | |
| 20:40:41 | sean-k-mooney | cfriesen: it looks like its looping over all the disks too https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L7986 | |
| 20:42:22 | sean-k-mooney | the code is very similar to the live migration code, they shoudl proably be refactored to gether but is also differnt enough that im not sure it does the same thing exactly | |
| 20:46:02 | sean-k-mooney | cfriesen: ya from an api persective it can. libvirt appears to keep them for cold and live migrate. i would assume they are not kept for rebuild,shelve and evacuate. i also think libvirt may be expanding the existing ephmeral disk in resize | |
| 20:47:34 | sean-k-mooney | cfriesen: the libvirt direver is specifclaly checking that we are not resizeing the ephermerl disk down https://github.com/openstack/nova/blob/e2a39bb30f716c78af30d61efb3fb7526f9bdf6d/nova/virt/libvirt/driver.py#L8268-L8275 implying that its not recreateing them but may be expanding them on resize | |
| 20:58:41 | boden | anyone heard about a "oslo_db.exception.DBNonExistentTable" cropping up across projects? Appears to have cropped up around 10/10 http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22oslo_db.exception.DBNonExistentTable%5C%22 and includes neutron, nova and others | |
| 21:15:39 | mriedem | boden: those are unit/functional tests, | |
| 21:15:47 | mriedem | and 10 days is as far back as logstash goes | |
| 21:15:54 | mriedem | so 10/10 is probably not really when it started | |