| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-15 | |||
| 11:26:51 | mnestratov | mdbooth: hi | |
| 11:27:01 | mnestratov | regarding ^^ | |
| 11:27:22 | mdbooth | mnestratov: Yeah, I don't think I'm ever going to like that :( | |
| 11:28:09 | mdbooth | I'm very much in favour of multi-backend support, though. | |
| 11:28:14 | mnestratov | and I repeat what I just aksed in review - how do we change current default behavir? | |
| 11:28:14 | mdbooth | Just not with static config. | |
| 11:28:54 | mnestratov | *behavior | |
| 11:29:28 | mdbooth | 2 things: associate the backend in use somehow with the bdm, so it lives with the instance not with the host config | |
| 11:29:49 | mnestratov | I'm fine with it | |
| 11:30:10 | mdbooth | Second: configure a *default* somehow, or pass it in the api in a driver-agnostic way | |
| 11:30:32 | mdbooth | Advantage of doing the former is that you can change your mind about the latter | |
| 11:30:38 | mnestratov | are we going to ignore images_type config parameter? | |
| 11:30:51 | mdbooth | I would love to be able to do that, yes | |
| 11:30:57 | mdbooth | It's a bug | |
| 11:31:02 | mnestratov | me too | |
| 11:31:16 | mnestratov | are we going to do this? | |
| 11:31:58 | mdbooth | Given infinite time I will hopefully get to it eventually :) | |
| 11:32:18 | mdbooth | If you can get it done now I'll definitely support you in that. | |
| 11:32:34 | arslanq | rpodolyaka: i am running block migration and hitting with this issue: Error LiveMigrateException: Live Migration failed: Migration complete but instance did not change host: compute7 | |
| 11:32:34 | mnestratov | it doesn't solve my problems | |
| 11:33:15 | mnestratov | but ok I'll try to figure out how to do this via DBM | |
| 11:33:42 | rpodolyaka | arslanq: are there any additional errors on both source and destination compute nodes? nova-compute / libvirt logs | |
| 11:34:09 | mnestratov | I mean BDM :) | |
| 11:35:21 | arslanq | rpodolyaka: i found this error log in nova logs: Live Migration failure: Operation not supported: qemu does not support drive-mirror command | |
| 11:36:02 | johnthetubaguy | asettle: do you have more details on the issue he is hitting at all? | |
| 11:37:51 | asettle | johnthetubaguy: From Brian this morning "And all the steps work correctly on CentOS, I just have communication problems between the compute and the controller nodes. At this point I’m thinking it is a problem with CentOS packages or perhaps my environment." | |
| 11:37:58 | asettle | I don't have much more than that, unfortunately (as in, no error logs) | |
| 11:38:06 | mdbooth | mnestratov: Sorry, got disconnected. Scrolling back. | |
| 11:39:04 | rpodolyaka | arslanq: hmm, this sounds similar to https://bugs.launchpad.net/nova/+bug/1441054 | |
| 11:39:04 | openstack | Launchpad bug 1441054 in OpenStack Compute (nova) "live-migration --block-migrate fails with default libvirt flags" [Medium,Invalid] - Assigned to Mathieu Rohon (mathieu-rohon) | |
| 11:39:35 | rpodolyaka | arslanq: I remember there were some issues with block migration on some versions of libvirtd | |
| 11:40:01 | rpodolyaka | arslanq: specifically, we could not block migrate instances with block devices attached (e.g. config drives) | |
| 11:40:14 | mdbooth | mnestratov: Moving this into the BDM makes it unambiguous. It opens up the possibility of safely having a heterogenous storage config. | |
| 11:40:17 | rpodolyaka | arslanq: but, unfortunately, I do not remember details :( | |
| 11:40:20 | arslanq | rpodolyaka: Note: i don't have shared storage..... | |
| 11:40:36 | mdbooth | It means operators can change storage config after deployment without breaking all their existing instances. | |
| 11:40:38 | rpodolyaka | arslanq: anyway, this must have been fixed in the latest versions of libvirt/qemu | |
| 11:41:12 | mdbooth | It means migrating between hosts with different storage becomes unambiguous. | |
| 11:41:44 | arslanq | rpodolyaka: right now i am on openstack newton branch. Can i get some proper document of this issue? It will be very helpful for me. | |
| 11:42:12 | mdbooth | mnestratov: There are a bunch of advantages beyond just your use case. | |
| 11:42:31 | mdbooth | Whilst also fixing some edge cases where we currently have to skirt around security issues. | |
| 11:43:17 | mnestratov | mdbooth: what would you say if we will not introduce a new config parameter but instead just change the behavior. Like if images_type is none or default then image backend is chosen according to image disk_format metadata and stored in instance' BDM | |
| 11:44:37 | mdbooth | mnestratov: The behaviour should be that if it's present in instance BDM, that should always be used. We should only consult a config/whatever if it's not present, and then we should use that to set it in the bdm. | |
| 11:44:53 | jaypipes | morning folks | |
| 11:45:18 | mdbooth | mnestratov: That gets around the issue of changing config/different config between hosts. | |
| 11:45:27 | mdbooth | jaypipes: morning | |
| 11:45:27 | jaypipes | alex_xu, melwitt, dansmith, Vek: review bait: https://review.openstack.org/399679 | |
| 11:45:55 | mnestratov | mdbooth: it demands changing the way you boot your instances | |
| 11:46:12 | jaypipes | mdbooth: hey, morning. so how's it going with the libvirt storage pools patch series? haven't looked at it in a month or so. Need to get back on that. | |
| 11:46:15 | arslanq | rpodolyaka: i have the following libvirt version: libvirtd (libvirt) 2.0.0 | |
| 11:46:23 | mdbooth | jaypipes: I haven't done anything with it in that time, tbh. | |
| 11:46:40 | jaypipes | mdbooth: you planning to? or other priorities getting in the way? | |
| 11:46:44 | mdbooth | jaypipes: I need to make a new start on it. It's still on my list, though. | |
| 11:46:47 | cdent | I'm not coping well with jaypipes being 4 hours off instead of the usual 5. It's making me all confused and discombobulated. | |
| 11:46:50 | mdbooth | Yeah, just other priorities :( | |
| 11:47:09 | jaypipes | rpodolyaka: so, you finish the placement client plugin in OSC yet? :P | |
| 11:47:22 | jaypipes | cdent: :) | |
| 11:59:15 | mdbooth | lyarwood: Patch 1 done. I should hopefully pick up the pace a bit with growing context :) | |
| 12:07:56 | mnestratov | mdbooth: one more concern about ignoring images_type parameter. If we implement multibackend support, then it would be hard to report free disk space to scheduler. Is there a way to report free space for all supported backends? | |
| 12:13:26 | mdbooth | mnestratov: Free space needs to be reported per backend anyway. | |
| 12:14:55 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Fix formatting errors and warnings https://review.openstack.org/407364 | |
| 12:14:56 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: tox: Build docs with Python 2.7 https://review.openstack.org/445906 | |
| 12:14:56 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Treat doc build warnings as errors https://review.openstack.org/439883 | |
| 12:15:12 | sfinucan | johnthetubaguy: mriedem had. I've put them in a series now ^ | |
| 12:15:37 | sfinucan | bauzas: Easy +2s for your newfound spec powers above ^ :) | |
| 12:17:31 | johnthetubaguy | sfinucan: sweet, I want to see jenkins +2 all that first | |
| 12:17:49 | johnthetubaguy | sfinucan: did that last patch miss a link it your commit message? | |
| 12:18:07 | arslanq | rpodolyaka: I just tries this manually and i came to know that this happens only when i block migrate. "Live Migration failure: Operation not supported: qemu does not support drive-mirror command" | |
| 12:18:29 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: tox: Build docs with Python 2.7 https://review.openstack.org/445906 | |
| 12:18:52 | sfinucan | johnthetubaguy: Oops. Fixed | |
| 12:19:08 | arslanq | rpodolyaka: when i just use live migration it says that live migration is not possible without shared storage. | |
| 12:21:00 | mdbooth | I've stared at this long enough now that I assume I've just become blind to a typo or something. Could any who's got a second tell me what's causing the TypeError here: http://logs.openstack.org/42/445142/3/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/07e282a/logs/screen-n-api.txt.gz?level=ERROR | |
| 12:21:35 | mdbooth | Thank you #openstack-nova for helping me spot that typo | |
| 12:21:43 | mdbooth | You can ignore the above | |
| 12:24:17 | mdbooth | Also, mock-- for hiding an incorrect use of an interface | |
| 12:24:29 | johnthetubaguy | lyarwood: mdbooth: did you see the discussion over here: https://bugs.launchpad.net/nova/+bug/1672624 | |
| 12:24:29 | openstack | Launchpad bug 1672624 in OpenStack Compute (nova) "Ceph volumes attached to local deleted instance could not be correctly handled" [Undecided,New] | |
| 12:25:11 | mdbooth | johnthetubaguy: I didn't. Can look after lunch, though. | |
| 12:26:49 | openstackgerrit | Matthew Booth proposed openstack/nova master: Make server_external_events use an optimal number of db queries https://review.openstack.org/445142 | |
| 12:28:36 | mdbooth | johnthetubaguy: Do we want to have api punt 'local_delete' to conductor and have it be a bit more thorough? | |
| 12:28:43 | mnestratov | mdbooth: yeah I know, but IFAIK it's impossible now | |
| 12:29:58 | mdbooth | johnthetubaguy: That's very similar to the evacuate use case where you want to cleanup a volume attachment when the compute host isn't available. | |
| 12:30:21 | mdbooth | biab | |
| 12:38:35 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add extra-specs-image-meta-conflicts spec https://review.openstack.org/382382 | |
| 12:40:49 | lyarwood | johnthetubaguy: just back from lunch, that isn't a bug for me, more of an operator failure tbh. ./me adds a note to the bug | |
| 12:41:32 | johnthetubaguy | lyarwood: hmm, curious what you mean by that | |
| 12:42:23 | lyarwood | johnthetubaguy: while the compute service is down the original instance / domain is still active and connected to the backing rbd volume | |
| 12:42:37 | lyarwood | johnthetubaguy: so any attempt to remove the volume from the rbd pool is going to fail | |
| 12:42:49 | johnthetubaguy | lyarwood: ah, yes, it should do, good point | |
| 12:43:01 | johnthetubaguy | lyarwood: you have to wait for clean up on the compute node, which they haven't done | |
| 12:43:14 | lyarwood | johnthetubaguy: that or manually fence it | |
| 12:43:21 | lyarwood | johnthetubaguy: get it into a known state | |
| 12:45:29 | johnthetubaguy | lyarwood: yeah, we might need better tooling for that long term, post cinder API fix ups | |
| 12:45:47 | alex_xu | nova api meeting is 15 mins at openstack-meeting-4 | |
| 12:45:57 | johnthetubaguy | alex_xu: yeah, I should get me some lunch quickly! | |
| 12:46:08 | alex_xu | johnthetubaguy: :) | |
| 12:47:29 | alex_xu | johnthetubaguy: maybe we can move the meeting early 15mins, then you can get a better lunch | |
| 12:48:46 | johnthetubaguy | alex_xu: nah, that just means I am hungry during the meeting | |
| 12:49:04 | alex_xu | :) | |
| 12:50:59 | alex_xu | johnthetubaguy: sdague experiment for removing stevedore https://review.openstack.org/#/c/445864/ | |