Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-01-12
15:13:12 fried_rice nice. So if we were to use that same semaphore for whatever providery thing we're mucking with, we should be safe with the current (loose/lacking) generation handling.
15:15:39 mriedem i guess
15:15:46 mriedem it is just a big dumb lock though,
15:16:06 mriedem and there are known perf issues with holding that lock too long which starves other stuff
15:16:23 cdent does "too long" have a value?
15:16:33 mriedem at least 60 seconds yeah for default rpc timeout
15:16:35 fried_rice mriedem Yeah, I'll want to solve the generation thing properly eventually; I'm trying to determine the urgency of doing that.
15:16:47 mriedem we've had gate issues in the past where that lock was held for more than 60 seconds and we'd have messaging timeouts
15:17:19 mriedem otherwise "too long" is when some operator tells us in a year that their compute with 70 instances on it is holding the lock for 45 seconds and that blows up their server create time on that compute host
15:17:37 mriedem but of course that's hard to know ahead of time
15:17:41 mriedem or test for
15:21:45 cdent mriedem: is your vacation next week? back to mexico?
15:21:50 mriedem yup
15:21:53 mriedem ildikov: rebase looks good, nice work
15:22:00 mriedem gibi: thanks for reviewing the multiattach patches
15:22:06 mriedem gibi: just one more to go.... :)
15:23:03 ildikov mriedem: cool, thanks for double checking!
15:25:44 openstackgerrit Matt Riedemann proposed openstack/nova master: Add the nova-multiattach experimental queue job https://review.openstack.org/532689
15:25:50 mriedem put the new CI job on top of the series
15:26:31 mriedem wonder if i should make that non-voting check
15:26:37 openstackgerrit Balazs Gibizer proposed openstack/nova master: DNM: Testing if funct test is triggered https://review.openstack.org/533211
15:26:37 openstackgerrit Balazs Gibizer proposed openstack/nova master: DNM: Test how to override irrelevant-fiels in zuul jobs https://review.openstack.org/533210
15:27:56 gibi mriedem: I will have not time to dig into the last multi attach patch today but I will check back on Monday
15:28:04 mriedem cool, thanks
15:28:57 mriedem johnthetubaguy: i think this is probably something you as an ironic loving person should probably look at https://review.openstack.org/#/c/527541/ - i think we need that before we can remove the ironic flavor migration routine, and we also likely should backport that to pike
15:29:05 openstackgerrit Édouard Thuleau proposed openstack/nova master: Update plugs Contrail methods to work with prisep https://review.openstack.org/533212
15:30:24 gibi mriedem: based on what I learned today in infra, the project-config job definitions does not override the jobs.yaml at all so it is possible that we have more cases where the proper jobs are not triggered
15:31:01 mriedem gibi: so https://github.com/openstack-infra/project-config/blob/5017c2ba472bd0439392f037036dbf198e00c137/zuul.d/projects.yaml#L10604 makes no difference?
15:31:25 gibi 15:52 < smcginnis> gibi: Yep, if the job is added through a template, then you can't override that.
15:31:28 gibi 15:52 < smcginnis> gibi: A confusing and frustrating "feature" to me.
15:31:30 gibi 15:53 < smcginnis> And based on the number of jobs in project-config that are attempting to override that, I think it's widely misunderstood.
15:31:52 mriedem hmm, so where can we override it? or can't we?
15:32:08 mriedem oh i guess we'd have to add a nova-tox-functional job
15:32:14 mriedem with openstack-tox-functional as the parent
15:32:15 smcginnis mriedem: Only at the source.
15:32:18 mriedem and then do our overrides
15:32:24 smcginnis Yep
15:32:35 ildikov mriedem: did you mean the multi-attach job as non-voting?
15:32:35 gibi mriedem: I got a tip to try to override via .zuul.yaml in nova tree
15:32:35 mriedem well, that's probably better anyway, because
15:32:42 gibi mriedem: not yet working in https://review.openstack.org/#/c/533210/1
15:32:49 mriedem if you look at openstack-tox-py27 here http://status.openstack.org/elastic-recheck/data/integrated_gate.html
15:32:58 smcginnis But the sad thing there is, the other one will still get picked up from the template, so on some patches you will have extra job runs, IIUC.
15:33:00 mriedem that is ^ on all different projects using the same job name
15:33:03 ildikov mriedem: if yes, I support the idea :)
15:33:04 mriedem where before we have like nova-py27
15:33:16 mriedem ildikov: yes, right now the patch puts the job in the experimental queue
15:33:21 mriedem but people will forget to run that
15:33:44 mriedem gibi: yeah i think https://review.openstack.org/#/c/533210/1/.zuul.yaml is exactly what we're looking for
15:33:52 ildikov mriedem: cool, I think non-voting would be great
15:36:11 gibi mriedem: something is still not correct in my override so I need a bit more digging how to do it properly
15:36:18 mriedem ok
15:38:49 cfriesen has anyone run into problems live-migrating a boot-from-volume instance, with a config drive, on a compute node with local storage?
15:42:04 openstackgerrit Édouard Thuleau proposed openstack/nova master: Update plugs Contrail methods to work with privsep https://review.openstack.org/533212
15:43:20 fried_rice Has anyone else been following the development of ProviderTree? superdan figleaf gibi ?
15:43:51 figleaf fried_rice: a bit, but not in too much depth
15:43:52 gibi fried_rice: I try to follow it as time allows
15:44:13 fried_rice I ran into a design blockade yesterday and need to talk it out.
15:44:51 superdan definitely not enough to discuss design issues
15:45:03 fried_rice So we've been protecting _Provider very carefully, ostensibly for thread safety.
15:45:44 sean-k-mooney fried_rice: via the generation count
15:46:13 fried_rice sean-k-mooney Actually this is only superficially related to generation
15:46:49 fried_rice I think it's just so report client can keep its ProviderTree (its local cache of providers) consistent
15:46:59 fried_rice In order to consume ComputeDriver.update_provider_tree, resource tracker (via report client) is going to need to be able to get at the provider's details (inventory, traits, aggs, etc.).
15:47:03 openstackgerrit Balazs Gibizer proposed openstack/nova master: DNM: Test how to override irrelevant-fiels in zuul jobs https://review.openstack.org/533210
15:47:04 openstackgerrit Balazs Gibizer proposed openstack/nova master: DNM: Testing if funct test is triggered https://review.openstack.org/533211
15:47:17 fried_rice Today the only way you can get at those things is via these kinda awkward has_X_changed() methods.
15:47:39 fried_rice I.e. you have to have something to compare against.
15:48:28 fried_rice But consuming update_provider_tree, I'm going to need to compare what's in the returned ProviderTree against what's in the report client's cached ProviderTree. So I need to peel e.g. the traits list out of the former so I have something to pass to have_traits_changed.
15:48:58 fried_rice And then if I decide it *has* changed, I'm going to need that same thing in order to send it down to placement.
15:49:25 bauwser superdan: I saw your comment on https://review.openstack.org/#/c/528832/7/nova/virt/libvirt/driver.py@4902, what would you prefer ?
15:49:40 bauwser superdan: I actually copy-pasted the Xen docstring
15:49:59 superdan bauwser: which comment?
15:50:02 superdan oh
15:50:05 bauwser L4902
15:50:06 gibi fried_rice: so you have a ProviderTree instance in the consumer and want to compare that with the ProviderTree instance in the cache
15:50:17 fried_rice giblet: just so.
15:50:34 superdan bauwser: oh that, I was just expressing frustration with the code, because I've been fighting with internal allocation stuff lately
15:50:42 superdan bauwser: not really asking for a change
15:51:05 superdan bauwser: it would be better if you put "allocations at microversion 1.x" I guess, but it's probably not worth it here
15:51:18 superdan bauwser: they changed format in 1.13 or something around there
15:51:22 gibi fried_rice: can we simply implement ProviderTree.diff(another_tree) function?
15:52:01 gibi fried_rice: do you need to know what is changed or you just have to updat what is changed?
15:52:14 gibi fried_rice: I mean update in the cache
15:52:19 fried_rice gibi a) that method would then still have to be able to get at another_tree.get_me_a_provider().get_me_its_fields(), and b) it would still have to return something that we can send to placement
15:52:22 bauwser superdan: okay, no worries :-)
15:52:30 bauwser I'll add more details and saying which rev
15:52:43 sean-k-mooney gibi: its an versioned object right so you can jsut call to primitive on both and then use the dict diff method to see if they are in sync or not
15:52:46 bauwser maciejjozefczyk_: my bad, just saw your ping
15:52:47 fried_rice gibi I have to update the cache too, but I also have to send changes back to placement.
15:53:01 bauwser gosh, I really need to resurrect my ZNC bouncer
15:53:20 bauwser the one I currently use is missing me notifications
15:54:03 gibi fried_rice: so you have to know which RPs are changed and also have to know what fields (e.g. inventory, aggregate, trait) are changed
15:54:07 bauwser maciejjozefczyk_: so we changed the opt values in order to signal whether it was changed by the operator or not
15:54:11 fried_rice gibi yup
15:54:38 fried_rice gibi None of this is a problem if I can get at the fields of the _Provider; but today there's no (legal) way to do that.
15:54:56 bauwser maciejjozefczyk_: now that there is no longer upgrade concerns with old Newton computes, I think it's okay to set that back
15:55:05 bauwser maciejjozefczyk_: thanks for helping on that !
15:55:22 sean-k-mooney fried_rice: no legal way because of the _
15:55:37 fried_rice sean-k-mooney Yeah, and the locking.

Earlier   Later