| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-03-27 | |||
| 18:02:36 | mriedem | that's out of scope | |
| 18:02:41 | sean-k-mooney[m] | ah they would all have the same mode | |
| 18:02:55 | sean-k-mooney[m] | ya mixing modes would be messy on the comandline and in the api | |
| 18:02:57 | mriedem | the list of bdms in the server create request are copied per instance that gets created | |
| 18:04:22 | sean-k-mooney[m] | so is it basically just a check in the api that says you can pass a volume on multi boot today? | |
| 18:04:40 | efried | edleafe: commented on https://review.openstack.org/#/c/556971/ | |
| 18:04:44 | mriedem | sean-k-mooney[m]: can't | |
| 18:04:45 | mriedem | but yes | |
| 18:04:51 | mriedem | it's linked from the spec i believe | |
| 18:04:59 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: add a note about volume-backed rescue not being supported https://review.openstack.org/556996 | |
| 18:06:32 | sean-k-mooney[m] | mriedem: cool ill read it properly later. it makes sense to me how you discibe it | |
| 18:08:58 | edleafe | efried: thanks. Will update shortly | |
| 18:11:37 | mriedem | stephenfin: we don't need this mypy spec do we? https://review.openstack.org/#/c/538217/ i thought you already started making those changes, much to my chagrin | |
| 18:12:21 | mriedem | s/chagrin/vexation/ | |
| 18:12:24 | mriedem | thanks google | |
| 18:14:22 | efried | cfriesen: "efried: if you send in two granular resource requests for resources1 and resources2, wouldn't it make sense to get back a dict of {resource1:<rp>, resource2:<rp>} or similar? They could still be the same RP." Sorry, I let this scroll by... | |
| 18:14:38 | efried | cfriesen: Yes, something like that would make sense. But we don't have that in the plan at the moment. | |
| 18:16:35 | efried | cdent, edleafe, jaypipes: How do you feel about the granular microversion changing the response payload to something like ^ to preserve the division of the requests? | |
| 18:17:22 | efried | gibi has already identified a place where that would be useful. | |
| 18:17:23 | cdent | efried: which response payload are you talking about? | |
| 18:17:39 | efried | cdent: The response from GET /a_c | |
| 18:17:52 | edleafe | efried: the purpose of the granular request is to ensure that each of those requirements is met by a single RP. What would be the advantage of splitting the out if it turns out that they are the same RP? | |
| 18:17:54 | cdent | a_r or p_s? | |
| 18:17:55 | efried | cdent: The format of an allocation_request, I think. | |
| 18:18:04 | efried | a_r, definitely not p_s | |
| 18:18:37 | cdent | efried: I'm not visualizing what you're really suggesting, could you paste something somewhere, or link me to some context? | |
| 18:18:56 | cdent | (I'm stuck on "how is this different from what we already do") | |
| 18:19:54 | efried | cdent: edleafe: The example I gave above was: resources1=VF:1,BW:200&resources2=VF:1,BW:300 may result in an allocation_request like RP_X: { VF: 2, BW: 500 }. Without consulting the original request (which, as in gibi's neutron case, may not be available), the caller can't tell that one VF should get BW:200 and the other should get 300. | |
| 18:20:47 | efried | cdent, edleafe: So the suggestion is that the response should instead look like: { resources1: { RP_X: { VF: 1, BW: 200} }, resources2: { RP_X: { VF: 1, BW: 300 } } } | |
| 18:21:02 | cdent | efried: and in this case we're on the same pf, and it is the pf that is being the resource provider, yes? | |
| 18:21:09 | efried | cdent: yes | |
| 18:21:23 | efried | RP_X is a PF provider of VF and BW resources. | |
| 18:21:36 | cdent | efried: does the caller still know the request it made? | |
| 18:21:49 | cdent | or still have whatever info it used to make the call? | |
| 18:22:02 | efried | cdent: That depends on the scenario. But in general ima say no. | |
| 18:22:30 | sean-k-mooney[m] | mriedem: regarding https://review.openstack.org/#/c/532410/4/specs/rocky/approved/volume-backed-server-rescue.rst i think they just want to do a normal image based nova rescue but for boot from volume instance. so the image size bits of the spec are just saying the rescue image is not empty | |
| 18:22:31 | gibi | cdent: neutron provide resource request pieces via the neutron port API, nova combines them to one big a_c request | |
| 18:23:15 | cdent | In general ima say that changing the allocation format again would make me sad, especially to represent granular stuff which seems icky somehow (will have to think a bit longer on why). Which is not "god no, I hate that" but rather "hmm, I'm not immediately in love with this" | |
| 18:23:26 | efried | cdent: I think the compute manager is the thing that builds the request, and the report client sends it down and gets the response, but the thing that needs to correlate the allocations to real resources is the virt driver (or in gibi's case, neutron (maybe a neutron agent)), and we're not giving that information to those guys. | |
| 18:24:04 | cdent | I'm uncomfortable with placement being use as state manager for the interaction between nova and neutron | |
| 18:24:16 | efried | eh, state manager? I don't see that. | |
| 18:24:43 | efried | All we're doing is providing the caller of GET /a_c with more detailed information about how the allocation_request was fulfilled. | |
| 18:24:54 | sean-k-mooney[m] | cdent: well its not state management. we just need to know how to correlate placement RP inventory to phyical resouce on the host | |
| 18:25:03 | efried | I.e. I took *this* request group to make *this* chunk of allocations. | |
| 18:25:53 | cdent | sounds like communicating state by proxy to me but maybe I'm peculiar | |
| 18:25:56 | efried | cdent: In the case of the virt driver, he can *probably* glean that information by looking back at the flavor... but now we also have resources/traits in the image, and gibi will also have them in the port, and future us may have... So asking virt to reconstruct all of that, which we currently do in nova, seems like a really bad idea. | |
| 18:26:49 | efried | cdent: It's not state. I see it as the difference between, "You asked for X and Y. Here's XY," and "You asked for X and Y. Here's X, Y." | |
| 18:27:34 | cdent | who is making the request for allocations in the example you gave? | |
| 18:27:40 | cdent | sorry candidates | |
| 18:27:53 | efried | the scheduler report client is the thing making the placement API call. | |
| 18:28:14 | cdent | right so nova is asking placement to construct data in a particular way so it can communicate something to nova | |
| 18:28:20 | cdent | s/nova$/neutron/ | |
| 18:28:41 | efried | No, if that's the part that's bothering you, let's remove neutron from the equation. | |
| 18:28:48 | efried | Just looking at nova talking to nova. | |
| 18:29:43 | efried | nova synthesizes resource+trait info from flavor, image, request spec, whatever sources. It comes up with a ResourceRequest (which is a set of RequestGroup) by the time it's ready to ask for allocation candidates. | |
| 18:29:47 | sean-k-mooney[m] | cdent: we have the same problem with numa in nova alone | |
| 18:30:16 | sean-k-mooney[m] | cdent: placement has 2 numa RP both with cpus. placement claims against one of the RPs how do i know which phyical numa node that alloaction is against | |
| 18:30:28 | cdent | so you have some state in the flavor, you use it to make a request and because you don't want to look back at that state in the flavor, you want to extend placement to transmit that state (again I'm not saying this is the worst thing ever, just trying to identify what's being done) | |
| 18:30:30 | efried | sean-k-mooney[m]: Not that, no. | |
| 18:30:53 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Network bandwidth resource provider https://review.openstack.org/502306 | |
| 18:31:10 | efried | How is what's in the flavor "state"? It's a set of resource requests. | |
| 18:31:35 | efried | As currently conceived, we'd be summing up resource requests that happen to land in the same RP. | |
| 18:31:42 | cfriesen | does anyone know why we plug vifs at nova-compute startup, but then also plug them again (but don't wait for them) when powering on the instance? (for libvirt anyway) | |
| 18:31:43 | efried | That takes away information. | |
| 18:32:11 | sean-k-mooney[m] | cfriesen: linux bridge | |
| 18:32:13 | efried | If they happened to be assigned from different RPs, we would *have* that information. | |
| 18:32:28 | efried | But since they happened to be assigned from the same RP, they get rolled together, and we lose that information. | |
| 18:33:03 | sean-k-mooney[m] | cfriesen: the plug on startup is because on a reboot the tap wont be added to linux bridge because linux bridge does not preseits state. we then need to do it as part of boot a second time | |
| 18:33:05 | efried | Certainly when the virt driver is trying to grab real resources corresponding to an allocation, it needs to be able to correlate the RC+RP back to something real. | |
| 18:34:17 | gibi | mlavalle, jaypipes: I updated the bandwidth spec based on our discussion. | |
| 18:34:22 | cdent | I think may be using the term "state" far more generally than you are, but that's probably not all that germane | |
| 18:34:51 | jaypipes | gibi: ty gibi. | |
| 18:35:18 | gibi | efried: I don't have the brainpower any more to think through you mapping comments in the bandwidth spec today. I will get back to that tomorrow | |
| 18:35:19 | efried | cdent: looklook, here's another example. Today I have no way of requesting two different disks from placement. With granular, I could. I want a 1G disk and a 2G disk, so I would say: resources1=DISK_GB:1024&resources2=DISK_GB:2048. | |
| 18:35:35 | gibi | efried: thanks for pushing the discussion forward with cdent right now :) | |
| 18:35:38 | efried | cdent: In a scenario where I have sharing providers, or maybe multiple disk providers in my tree, or whatever, I may get back a candidate like { STOR_RP1: { DISK_GB: 1024 }, STOR_RP2: {DISK_GB: 2048 } } | |
| 18:36:03 | cdent | efried: I'm still trying to think this through, but it's slow going because I need to get past a fundamental issue for me: I really don't want to expose granularity in allocations... | |
| 18:36:19 | efried | cdent: And that would be fine, cause now my virt driver can tell it needs to get 1024 from STOR_RP1 and 2048 from STOR_RP2. | |
| 18:36:19 | edleafe | efried: in that scenario, there isn't anything preventing placement from getting both "disks" from the same disk RP | |
| 18:36:50 | efried | edleafe: Exactly. In which case, as currently conceived, we would report a candidate like: { STOR_RP1: { DISK_GB: 3072 } } | |
| 18:36:58 | cfriesen | sean-k-mooney[m]: I get the boot-time one...why do we need to do it again at instance boot if vifs_already_plugged=True ? | |
| 18:37:06 | cdent | didnt we have a discussion (or was it in my mind) recently about the difference between contiguous and non-contiguous resource providers? | |
| 18:37:20 | efried | cdent: Somehow virt needs to figure out that that's really a 1G disk and a 2G disk. How does he figure that out? Where does he get that information? | |
| 18:37:44 | efried | cdent: Yes, this is related to that discussion. It's why we couldn't e.g. split a VCPU:2 request across two numa nodes. | |
| 18:37:52 | edleafe | efried: what I'm saying is that there is no way to express "these must be two separate RPs" | |
| 18:38:06 | efried | edleafe: Yes, but in this case that's not something I need to express. | |
| 18:38:23 | edleafe | what granular gets you is "everything in this group must be on the same RP tree" | |
| 18:38:26 | efried | edleafe: What I *am* trying to express is that "these must be two separate *disks*". | |
| 18:38:38 | sean-k-mooney[m] | cfriesen: i dont think vifs_already_plugged will be true on the boot case. just on soft reboot | |
| 18:38:39 | efried | edleafe: ...which I expressed by putting them into separate granular groups. | |
| 18:38:39 | mriedem | are people ok with me just self approving this spec update to match the actual multiattach implementation? https://review.openstack.org/#/c/544152/ | |
| 18:38:47 | sean-k-mooney[m] | cfriesen: are you using hybrid plug? | |
| 18:39:02 | sean-k-mooney[m] | cfriesen: e.g. are you using the iptables firewall driver or conntrack | |
| 18:39:08 | efried | mriedem: Given the +1s that are on it, fine by me. | |
| 18:39:09 | cfriesen | sean-k-mooney[m]: for power-on vifs_already_plugged is true | |
| 18:39:24 | edleafe | efried: that's not what granular is for, is it? It's to ensure that all the requirements in a single group come from the same RP tree | |
| 18:39:39 | edleafe | not that every group must come from different trees | |
| 18:39:41 | mriedem | efried: done | |
| 18:39:42 | mriedem | thnaks | |
| 18:39:47 | mriedem | thanks even | |