| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-22 | |||
| 17:21:15 | bauzas | I'll review jaypipes's series for shared-rps | |
| 17:36:17 | edmondsw | johnthetubaguy can you point me to the patch you mentioned at the summit to swap network routing before pause/resume on live migrate? | |
| 17:44:43 | mriedem | melwitt: just so you're aware, i posted comments in the base patch but i'm still working through the rest | |
| 17:45:46 | gyee | mriedem: can you please take a look whenever you have a chance? https://review.openstack.org/#/c/466143/ | |
| 17:45:48 | melwitt | mriedem: cool, thanks | |
| 17:46:24 | openstackgerrit | Steve Noyes proposed openstack/nova master: WIP: update live migration to use v3 cinder api https://review.openstack.org/463987 | |
| 17:54:40 | jaypipes | mriedem, dansmith, melwitt, cdent, bauzas: ok, ML post sent about getting rid of automated reschedule behaviour. | |
| 17:55:15 | dansmith | ack | |
| 17:56:35 | jaypipes | mriedem, dansmith, melwitt, cdent, bauzas: pls let me know if I accurately explained the issue and the request. | |
| 17:56:48 | melwitt | out of curiosity I checked how kubernetes scheduling works and whether it retries, to be aware of what others are doing, | |
| 17:57:22 | melwitt | and it sounds like it does retry forever "remains in pending state until suitable machine becomes available" according to this post. I didn't find a doc about it on k8s site yet https://stackoverflow.com/questions/28857993/how-does-kubernetes-scheduler-work/28874577#28874577 | |
| 17:57:41 | jaypipes | melwitt: well, yeah, because k8s is an orchestrator :) | |
| 17:57:56 | dansmith | yeah, totally different | |
| 17:58:06 | jaypipes | melwitt: which fits nicely into my "nova shouldn't be an orchestrator" point ;) | |
| 17:58:28 | melwitt | but openstack doesn't have an orchestrator at all, does it? | |
| 17:58:57 | melwitt | so our thing just performs worse | |
| 17:59:25 | jaypipes | melwitt: heat? | |
| 17:59:27 | dansmith | heat | |
| 17:59:28 | dansmith | heh | |
| 17:59:36 | jaypipes | melwitt: or tacker for NFV. | |
| 18:00:45 | melwitt | I think heat is a manual orchestrator, you (the end user) have to tell it everything to do, I thought | |
| 18:01:28 | jaypipes | melwitt: well, k8s is the same. you declare a pod definition and k8s tries constantly to make your pod definition a reality. | |
| 18:01:51 | jaypipes | melwitt: or you just meant heat doesn't constantly try to do things? | |
| 18:03:04 | melwitt | jaypipes: more the latter. I'm not a heat expert, maybe there's a way to tell it to keep retrying a definition | |
| 18:03:29 | jaypipes | melwitt: yeah, I'm not a Heat expert either. | |
| 18:17:08 | cdent | jaypipes, melwitt: if want to play the odds, it is more likely that a "keep retrying orchestrator" (that actually does the job well) will come into existence if we do not include bad orchestration in nova. | |
| 18:17:36 | jaypipes | cdent: ++ | |
| 18:17:54 | cdent | it is also the case that the odds of "good orchestration" merging sooner (in nova) than something else trying to do it (outside of nova's constraints) are low | |
| 18:20:43 | melwitt | cdent: maybe. I'm not sure how many people would want to use an orchestrator-less system, but that works as long as no other viable options are available | |
| 18:21:37 | cdent | melwitt: Is what we have today orchestrator-less or bad-orchestrator? | |
| 18:22:25 | melwitt | cdent: bad-orchestrator | |
| 18:22:51 | cdent | as in makes a half hearted effort at retrying and then bails..., yeah? | |
| 18:23:23 | melwitt | yes | |
| 18:25:47 | jaypipes | dansmith: you were +2 previously. I'm good on https://review.openstack.org/#/c/460177/ now with bauzas' latest revision. | |
| 18:26:48 | melwitt | cdent: with the half hearted effort, an operators can detect a bad baremetal node and take it out of rotation without exposing end users to the fail. without it, the end user would have to suffer the failure at least once before the operator can take it out of rotation. unless the operator builds a porcelain API that maybe uses heat behind it and has end users point at that. is what I'm thinking | |
| 18:29:43 | melwitt | I can also see the point of, you can't serve everybody perfectly, and if it's really only 5% of people that would be affected, then I get that. we'll see what they say on the ML | |
| 18:35:38 | mriedem | melwitt: ok https://review.openstack.org/#/c/446239/ | |
| 18:35:44 | mriedem | melwitt: i didn't go through tests | |
| 18:36:38 | melwitt | mriedem: thanks. reading through | |
| 18:38:56 | mriedem | i have successfully finished a 60+ ounce bag of tootsie rolls from halloween | |
| 18:39:02 | mriedem | it was a grind, but it's done | |
| 18:40:47 | cdent | my jaw hurts now | |
| 18:54:07 | jaypipes | melwitt: hmm, I wonder if some combination of beer and cat toys will be enough to bribe your James into my worldview... | |
| 18:55:47 | mriedem | jaypipes: not until you can chug a beer faster than him | |
| 18:56:04 | mriedem | which from fenway we know isn't going to happen without proper training with a montage | |
| 18:56:52 | melwitt | heh | |
| 19:02:12 | melwitt | mriedem: replied on quotas | |
| 19:18:09 | cdent | mriedem, jaypipes, melwitt, dansmith: Does the idea that was bounced around on the call of claim in the scheduler, re-claim (when things are bad) in the conductor address the issues james is pointing out? Trying to clarify my confused brain. | |
| 19:18:42 | mriedem | cdent: yes and is what we discussed in boston | |
| 19:19:08 | mriedem | super conductor passes alternate hosts to the cell; if something fails, we reschedule via conductor within the cell using those alternate hosts | |
| 19:19:10 | cdent | i knew that the boston plan did, but wasn't clear on if the claim from two different places idea did | |
| 19:19:20 | cdent | gotchya, thanks | |
| 19:19:44 | mriedem | claiming in two places has the same long-term functionality if we have to retry within a cell | |
| 19:20:26 | mriedem | i think the idea of doing claims in the scheduler AND conductor is if we do it in the scheduler, we have (a) tighter retry loop and (b) if we eventually drop reschedules in the cell, then we've already got the work done in the scheduler - as dan pointed out on the call | |
| 19:20:26 | cdent | I think I'm leaning that way because of the "claim as early as possible" stuff I mentioned in one of my comments | |
| 19:20:30 | mriedem | the short-term downside is you have duplication | |
| 19:21:01 | mriedem | i do'nt really see how that's not hidden in the scheduler report client though | |
| 19:21:07 | cdent | DRY is what breeds and deeply nested object hierarchies and that makes me cry | |
| 19:21:10 | mriedem | or some utility method shared between conductor and scheduler | |
| 19:21:13 | cdent | yes | |
| 19:22:38 | cdent | woot! I managed a whole 4 things off my to do list today. That's less than 4%. ossum. | |
| 19:22:54 | cdent | goodnight | |
| 19:23:08 | mriedem | melwitt: did you give up halfway? | |
| 19:23:34 | melwitt | mriedem: did I miss some comments? I didn't mean to | |
| 19:23:47 | melwitt | argh, sorry | |
| 19:27:40 | larsks | Howdy. I have an instance that is attached both ipv4 and ipv6 networks. The ipv4 networks show up in network_data.json on the config drive, but there is no trace of the ipv6 network. I see the same thing with a single network that has both ipv4 and ipv6 subnets...only the ipv4 information shows up. What could be causing this? | |
| 19:33:39 | dansmith | mriedem: I may be remembering wrong, but wasn't there a thing in the last six months or something about us not including ipv6 stuff in network.json in certain circumstances? re ^ | |
| 19:33:54 | fungi | larsks: that's a known issue. i gather there's an open bug, mailing list thread and proposed change(s) addressing it, though i don't have references handy at the moment | |
| 19:34:40 | mriedem | dansmith: i vaguely remember sdague and jaypipes reviewing something in there recently, but don't know if it was for this | |
| 19:35:12 | larsks | fungi: thanks. do you happen to know in which release it should work right now? E.g. was this a regression in ocata or master or something? | |
| 19:35:23 | larsks | And would I expect it to work in release N-1? | |
| 19:35:43 | mriedem | https://github.com/openstack/nova/commit/ccebded0cbb654be18db5b4e0e7f8b8a3a7cacdb ? | |
| 19:35:58 | mriedem | ^ is what i was thinking of | |
| 19:36:02 | mriedem | but does'nt look like the right thing | |
| 19:36:39 | larsks | Hmmm, I found https://bugs.launchpad.net/nova/+bug/1514082 which suggests this is just a configuration issue. | |
| 19:36:39 | openstack | Launchpad bug 1514082 in OpenStack Compute (nova) "network_data.json does not provide information about ipv6 addresses." [Undecided,Invalid] | |
| 19:38:46 | fungi | larsks: on another channel, clarkb pointed me to bug 1676363 | |
| 19:38:46 | openstack | bug 1676363 in OpenStack Compute (nova) "The network metadata should be more useful" [Medium,In progress] https://launchpad.net/bugs/1676363 - Assigned to Dan Peschman (dpeschman) | |
| 19:39:04 | mriedem | dansmith: why do we need with instance.obj_alternate_context on https://review.openstack.org/#/c/458634/12/nova/compute/api.py@1802 ? | |
| 19:39:05 | fungi | larsks: related ml thread starts at http://lists.openstack.org/pipermail/openstack-dev/2017-March/114563.html | |
| 19:39:15 | larsks | fungi: thanks for both pointers. I like the bug title. I'm going to see if it was just a config issue. | |
| 19:40:05 | dansmith | mriedem: because the instance has the ctxt inside not the cctxt | |
| 19:40:42 | dansmith | mriedem: well, actually, maybe not there I guess | |
| 19:40:58 | dansmith | mriedem: might be able to remove that one I guess | |
| 19:41:05 | mriedem | hold up | |
| 19:41:08 | mriedem | still going through this | |
| 19:41:53 | dansmith | I need food anyway | |
| 19:43:09 | mriedem | this is probably tied to get_instance stashing the targeted context on the instance object | |
| 19:43:17 | mriedem | that's probably something that deserves a comment in the code then | |
| 19:49:12 | jaypipes | mriedem: and it's that "get me alternate hosts" thing that I am really against. | |
| 19:49:24 | jaypipes | mriedem: in the conductor that is.. | |
| 19:52:09 | bauzas | sorry folks, was afk for dinner purposes | |
| 19:52:51 | jaypipes | dansmith: the diaper exception. nice. | |
| 19:53:36 | dansmith | jaypipes: that | |
| 19:53:48 | melwitt | mriedem: replied to the rest of the comments | |
| 19:53:49 | dansmith | is definitely not my term, but someone used it around me recently and I made note to use it again | |
| 19:54:22 | bauzas | jaypipes: dansmith: unfortunatly, I think it's related to the thread mriedem created about the pendulum switching | |
| 19:54:41 | bauzas | ie. should nova be tolerant to misconfig? | |
| 19:55:11 | bauzas | if we assume the answer be no, then I agree, case c) is probably not worth it for rescheduling | |
| 19:55:56 | mriedem | bauzas: that's not really why i started that thread | |