Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-03-07
08:03:59 slaweq I have a question about rebuild of instance
08:05:24 slaweq (I'm using Juno release still), so sometimes when I rebuild active instance it's rebuild fine but in libvirt logs I can see something like "Cannot open log file: '/var/log/libvirt/qemu/instance-002d6c52.log'" and instance is then shutdown by libvirt
08:05:49 slaweq maybe someone of You got such issue already and know how to fix it?
08:06:58 openstackgerrit jichenjc proposed openstack/nova master: Trivial Fix https://review.openstack.org/442310
08:08:08 openstackgerrit Akira KAMIO proposed openstack/nova master: Replace root volume during rebuild https://review.openstack.org/442295
08:15:31 openstackgerrit jichenjc proposed openstack/nova master: Remove extension in API layer https://review.openstack.org/442312
08:50:56 openstackgerrit Zhenyu Zheng proposed openstack/nova master: [POC] Searchlight intergration https://review.openstack.org/442248
08:59:26 openstackgerrit Akira KAMIO proposed openstack/nova master: Replace root volume during rebuild https://review.openstack.org/442295
09:28:11 openstackgerrit Akira KAMIO proposed openstack/nova master: Replace root volume during rebuild https://review.openstack.org/442295
09:44:49 openstackgerrit Akira KAMIO proposed openstack/nova master: Replace root volume during rebuild https://review.openstack.org/442295
10:08:30 mdbooth lyarwood: What do you know of lvm thin pools?
10:13:12 openstackgerrit Akira KAMIO proposed openstack/nova master: Replace root volume during rebuild https://review.openstack.org/442295
10:14:17 lyarwood mdbooth: not much, I've never really had the chance to use them tbh
10:14:31 mdbooth lyarwood: k
10:14:59 lyarwood mdbooth: we don't use them in the lvm imagebackend do we?
10:15:20 mdbooth lyarwood: I'm just reviewing this patch: https://review.openstack.org/#/c/442126/2/nova/virt/libvirt/storage/lvm.py
10:15:44 mdbooth I'm curious, because I thought we already did use them if we specified sparse
10:15:55 mdbooth And this patch is adding something new
10:16:14 mdbooth I'm just reading some man pages to try to work out what's different from what we already do
10:19:21 markus_z lyarwood: Anything I can do for the stable/newton backport https://review.openstack.org/#/c/406898/ ? I answered all questions to that so far.
10:20:25 sfinucan johnthetubaguy: Could you take a look at https://review.openstack.org/#/c/435575? It's pretty noddy but it keeps getting into merge conflicts
10:20:39 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/nova-specs master: Enable SR-IOV NIC offload feature discovery https://review.openstack.org/435954
10:20:40 lyarwood markus_z: I was going to take another look at it today before matt gets online so we can hopefully close it out
10:20:54 lyarwood markus_z: I think you've answered his questions but just wanted to check first
10:21:03 johnthetubaguy sfinucan: oh, I can see what why would happen, yeah, looking
10:21:12 sfinucan johnthetubaguy: Cheers :)
10:21:46 lyarwood mdbooth: ah right, before we would just create a sparse LV and manually grow that LV (I *think*)
10:21:55 markus_z lyarwood: OK, thanks. JFYI, I might leave early today ~1600UTC
10:22:05 lyarwood at least, that's what RHEV would do back in the day with sparse LVs
10:22:14 mdbooth johnthetubaguy: Following our discussion last week, I suspect that even with Cinder V3 we're going to end up doing this: https://review.openstack.org/#/c/383859/
10:22:39 mdbooth i.e. nova-side management of shared connections
10:24:12 mdbooth johnthetubaguy: Any chance we could get that over the line? Fixes an issue today, I suspect the code can eventually be lifted elsewhere.
10:24:38 johnthetubaguy mdbooth: given my queue of work today, I can't see me getting the time to look at that I am afraid
10:25:00 mdbooth johnthetubaguy: Well it's been sitting for months, so I'm not going to stress about today :)
10:25:29 mdbooth If you have a 'sometime soon' list, though, that would be awesome :)
10:25:39 johnthetubaguy mdbooth: I understand it better now, having chatted through it, which is all good, yeah, its totally on that list
10:25:48 mdbooth Cool, thanks.
10:26:08 johnthetubaguy mdbooth: if I haven't been to review that early next week, totally bug me again
10:26:18 mdbooth johnthetubaguy: You may regret that.
10:26:21 mdbooth Sure.
10:26:28 johnthetubaguy mdbooth: probably, but its for the best :)
10:30:36 openstackgerrit Akira KAMIO proposed openstack/nova master: Replace root volume during rebuild https://review.openstack.org/442295
10:30:37 gibi johnthetubaguy: hi! a frindly reminder about the Show persisted scheduler hints in server details spec https://review.openstack.org/#/c/440580/ It gathered even more +1s yesterday :)
10:34:53 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Move 'floating_ips' opts into 'network' https://review.openstack.org/431039
10:34:54 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Add neutron.floating_pool https://review.openstack.org/431037
10:34:54 openstackgerrit Stephen Finucane proposed openstack/nova master: conf: Deprecate 'default_floating_pool' https://review.openstack.org/431038
10:35:40 jvarlamova__ Hi, can someone help me? I get error "VirtualInterfaceCreateException: Virtual Interface creation failed", when trying to boot multiple instances in parallel. What's the reason?
10:39:31 rpodolyaka jvarlamova__: could you please provide a complete stack trace from the nova-compute log?
10:40:35 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Standardize naming of variables https://review.openstack.org/430272
10:40:36 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Standardize indentation of test_vif https://review.openstack.org/430271
10:45:59 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Make 'get_domain' private https://review.openstack.org/417378
10:47:04 openstackgerrit Stephen Finucane proposed openstack/nova master: console: Move proxies to 'console/proxy' https://review.openstack.org/408192
10:53:31 openstackgerrit Akira KAMIO proposed openstack/nova master: Replace root volume during rebuild https://review.openstack.org/442295
11:07:13 openstackgerrit Evgeny Antyshev proposed openstack/nova master: get_model method missing for Ploop image https://review.openstack.org/441989
11:12:56 jvarlamova__ rpodolyaka: http://pastebin.com/0jY6Ax1C
11:20:13 rpodolyaka jvarlamova__: this exception is raised when nova-compute times out waiting for vif-plugged event - https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L5052-L5061
11:21:18 rpodolyaka jvarlamova__: you should check nova-api / neutron-server / neutron l2 agent logs for this device id (== instance uuid) to see if such event was delivered to nova at all
11:27:41 mdbooth alex_xu: Hey, just thinking about your response on https://review.openstack.org/#/c/441829/4 about the cache.clear(). This is all running with the placement service, right? Are we guaranteed to only have 1 of those globally?
11:27:52 mdbooth cdent: ^^^ ?
11:28:27 mdbooth alex_xu: I'm just thinking that if a Trait is created through 1 service, then accessed through another, the cache on the second won't have been refreshed.
11:28:47 mdbooth However, I don't know how this is intended to be deployed, so this may not make sense.
11:29:31 cdent mdbooth: there's only one placement database, but could be multiple placement api servers
11:29:44 mdbooth cdent: That's what I would have expected. Thanks.
11:30:15 cdent I'm a bit nonplussed about the traits and resource class caches
11:30:35 mdbooth cdent: Explain
11:30:56 cdent they are trying to optimize sql queries when some of the data is not in an rdbms
11:31:09 cdent and thus they are...weird
11:31:21 cdent I would have prefered the optimizations to come later
11:33:24 cdent mdbooth: bonus points for "Explain"
11:33:32 mdbooth :)
11:34:03 mdbooth cdent: They look like they're getting all their data from the db
11:34:08 mdbooth What did I miss?
11:34:23 mdbooth For context: I never looked at this before yesterday
11:34:37 cdent which "they" did you mean? the traits cache may be different (and I haven't looked closely at it yet)
11:35:00 mdbooth 'they' == resource class and traits caches
11:35:35 mdbooth cdent: Ah... Yes, some resource classes are hard-coded
11:35:45 mdbooth I've slept since I read that
11:36:03 cdent you sleep?
11:36:10 mdbooth Don't tell anyone
11:37:10 jvarlamova__ rpodolyaka: As I see from logs, neutron sent 'network-vif-plugged' event for that instance to nova, and there's message "Creating event network-vif -plugged ...for instance ..." in nova-api logs. Looks like nova received event.
11:41:49 alex_xu mdbooth: ah...i got you, good point
11:45:34 alex_xu mdbooth: cdent probably resource-class won't have such problem. the cache for resource-class is just for query single resource-class.
11:47:41 mdbooth alex_xu: The negative lookup is going to be tough for the startswith match, I think.
11:48:04 cdent alex_xu: is it possible to avoid having the cache through the initial implementation, and only add the cache at the end?
11:48:08 mdbooth alex_xu: Whereas the other 2 are definite lookups: it exists or it doesn't, and you normally expect that it exists
11:48:26 cdent building in cache from the outset almost always leads to harder to maintain code
11:48:29 mdbooth Returning no values or some values doesn't necessarily mean that you're not out of date
11:50:06 mdbooth cdent: How often will we hit this thing? Would an index optimised for a startswith query be appropriate?
11:50:47 cdent (thus my concerns about early optimizations and speculations before measuring, etc)
11:50:53 mdbooth From reading the context, the expectation seems to be a lot. Whose was that expectation?
11:51:02 alex_xu cdent: in the poc, actually I query the db directly. but I try to get the API "GET /traits?name=in:TRAIT1,TRAIT2" to use the cache, since that is the API used to validate whether the traits is valid or not
11:51:18 cdent mdbooth: it will be alot, eventually
11:51:31 cdent but that's based on abstractions, not measurement
11:52:03 mdbooth Are we carefully avoiding the db throughout this whole code path?
11:52:36 alex_xu wait, no, that is for "GET /traits?prefix=xxx", that can be from the db directly. "GET /traits?name=in:TRAIT1,TRAIT2" can get benefit from the single trait lookup
11:53:24 cdent but I haven't had a chance yet
11:53:29 mdbooth alex_xu: Yeah, I think the concrete lookup is ok to cache. Either you get a hit (but wait... what if it was deleted?), or you don't, in which case you can refresh.
11:53:30 cdent (been concentrating on the spec)
11:54:33 alex_xu mdbooth: yea, if user query a non-existed trait, that will trigger a db call
11:54:55 mdbooth alex_xu: Deleting a cached trait/resource class is also an interesting case.

Earlier   Later