| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-18 | |||
| 13:29:54 | cdent | efried: going back through the log looking at the conversation you had with sean-k-mooney and rgerganov; where in the powervm universe is the mapping from rp uuid to <other> going to live? | |
| 13:30:42 | stephenfin | efried: The problem I'm trying to get at is, even if we somehow managed to keep non-PCI-needing instances off of PCI-having NUMA nodes, we can still end up in the situation where we have no free PCI-having NUMA nodes for PCI-needing instances | |
| 13:31:09 | mdbooth | lyarwood: Could it be cause there's no request context at that point? | |
| 13:31:18 | mdbooth | i.e. no user, no api call? | |
| 13:31:26 | efried | cdent That's a pretty big TBD. We don't have any persistent data on our "hypervisor" (the NovaLink partition), which is a fairly fundamental point of architecture. You're supposed to be able to trash the partition and recreate it with no loss. | |
| 13:31:32 | stephenfin | This approach reduces the possibility but it doesn't mitigate it entirely. We want to use the two in tandem, which is what you've kind of hinted at (I think) | |
| 13:31:45 | lyarwood | mdbooth: yeah sorry, https://github.com/openstack/nova/blob/master/nova/context.py#L279 | |
| 13:31:59 | sean-k-mooney | stephenfin: that was the usecase we created https://github.com/openstack/nfv-filters/blob/master/nfv_filters/nova/scheduler/filters/aggregate_instance_type_filter.py to address | |
| 13:32:11 | openstackgerrit | Chris Dent proposed openstack/nova-specs master: Add spec for symmetric GET and PUT of allocations https://review.openstack.org/508164 | |
| 13:32:16 | efried | stephenfin Yes. You'll never get around that limitation. Unless you want to totally deny non-PCI-needing instances from booting on PCI-having NUMA nodes. Which ain't reasonable IMO. | |
| 13:32:43 | sean-k-mooney | it allows you to create aggregates of nodes with scarce resouces and require that they are requested in the flavor to schedule to those nodes | |
| 13:32:55 | stephenfin | efried: But you will with this spec, which allows booting PCI-needing instances from using non-PCI-having NUMA nodes | |
| 13:32:58 | efried | cdent Off the cuff, the idea would be to make some part of the RP relate to some part of whatever thingy we're "mapping" to. | |
| 13:33:16 | sean-k-mooney | this filter will work with traits in request in the flavor too by the way | |
| 13:33:19 | mdbooth | lyarwood: Yeah, that would be required for a glance request, I guess | |
| 13:33:24 | stephenfin | sean-k-mooney: I saw that, but it wasn't too inflexible, hence https://blueprints.launchpad.net/nova/+spec/reserve-numa-with-pci | |
| 13:33:41 | mdbooth | Although there's useful work it can do without that | |
| 13:33:50 | sean-k-mooney | that is your new weigher right | |
| 13:34:00 | lyarwood | mdbooth: it's the cinder encryption metadata lookup that's failing at the moment | |
| 13:34:08 | sean-k-mooney | stephenfin: i tought that merged in pike? | |
| 13:34:24 | efried | stephenfin The "problem" you won't get around is where I boot a hundred non-PCI-needing instances and run out of non-PCI-having NUMA nodes after the first fifty, so I start using my PCI-having NUMA nodes. Once those are full, I can no longer boot "required"-affinity PCI-needing instances. | |
| 13:34:28 | stephenfin | sean-k-mooney: Correct. If you had to divide your cloud into aggregates for PCI devices, it made it very inflexible if you wanted to, say, start using a lot of PCI-having instances or vice versa | |
| 13:34:43 | stephenfin | sean-k-mooney: It did. I'm saying https://review.openstack.org/#/c/361140/ forms a one-two combo :) | |
| 13:35:03 | mdbooth | lyarwood: Hmm, yeah. That's not gonna work unless we've got some special-sauce cinder admin login, or we cached it. | |
| 13:35:57 | stephenfin | efried: Yup, there's nothing we can do about that, I'm afraid. If you want to use 'required' then this is what you signed up for. Go buy more PCI devices :P | |
| 13:36:03 | sean-k-mooney | stephenfin: right we really need to merge https://review.openstack.org/#/c/361140/29/specs/queens/approved/share-pci-between-numa-nodes.rst and implemented this cycle it was originally ment to work in icehouse.. | |
| 13:36:33 | efried | stephenfin Cool, then we're on the same page mentally; but then I don't think that chunk of the spec is expressing any reasonable "Alternatives". | |
| 13:36:44 | cdent | efried: k, thanks. I’d rather we avoid putting that kind of mapping into placement if possible, but it seems there is a fairly generic need for a translation. I presumem using rp.name is insufficient. Next comes the sort of config info that rgerganov mentioned (things like the IP for the shared storage piece?). That, to me, especially should not go into placement as we’re doing a serious mission creep at that point. | |
| 13:36:46 | stephenfin | efried: However, the https://blueprints.launchpad.net/nova/+spec/reserve-numa-with-pci should help mitigate that, and the 'preferred' policy in https://review.openstack.org/#/c/361140/ should solve it completely | |
| 13:36:54 | sean-k-mooney | stephenfin: as an employee of a hardware vendor that sells pic devices i feel oblidged to +1 that responce :P | |
| 13:37:02 | stephenfin | *if* you're ok living with the downsides (possible performance impact) | |
| 13:37:13 | stephenfin | sean-k-mooney: :P | |
| 13:37:31 | stephenfin | efried: Sure, I'm happy to drop it, if that makes sense? :) | |
| 13:37:46 | stephenfin | *if that would help? | |
| 13:37:57 | efried | stephenfin If you don't mind, perhaps give me a crack at rewording it? | |
| 13:38:04 | stephenfin | efried: Knock yourself out | |
| 13:38:32 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Add pagination and timestamp filtering support for os-migrations API https://review.openstack.org/506030 | |
| 13:38:46 | markus_z | Here's a doc change waiting for review for 5 weeks: https://review.openstack.org/#/c/476188/ | |
| 13:38:55 | stephenfin | efried: Also, saw your reply on the "two spaces indicate age" comment. It was just an amusing article I read once upon a time that stuck with me. Not an actual ageist thing or anything! | |
| 13:39:19 | efried | cdent I agree with you (at least in the sense that I've accepted the stated limitations of the placement architecture). We could conceivably get some mileage out of overloading the 'name' field, or embedding information into gross custom traits. But I would be wanting to look for cleaner alternatives pretty hard before resorting to that. | |
| 13:39:21 | stephenfin | markus_z: Sure, I'll take a look | |
| 13:39:27 | markus_z | stephenfin: thanks! | |
| 13:39:58 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Explain the flow of the "serial console" feature https://review.openstack.org/476188 | |
| 13:41:33 | efried | stephenfin No offense taken, of course. It's an argument that's been made to me before, so you may have caught some blowback from arguments I've had with others who were... um... more serious about it (to the point of -1ing stuff). My take on it is that the reasons for "never do that" don't apply to programmers. | |
| 13:42:55 | dansmith | mriedem: tell me you don't care and I'll +W: https://review.openstack.org/#/c/512406/3 | |
| 13:44:17 | openstackgerrit | Ghanshyam Mann proposed openstack/nova-specs master: Spec for API extensions policy removal https://review.openstack.org/508101 | |
| 13:44:48 | efried | stephenfin Before I start, can you confirm that https://review.openstack.org/#/c/361140/29/specs/queens/approved/share-pci-between-numa-nodes.rst@131 was indeed supposed to be "without"? | |
| 13:45:29 | gmann | johnthetubaguy: sdague alex_xu ^^ updated spec - https://review.openstack.org/#/c/508101/ | |
| 13:45:46 | mriedem | dansmith: i could fix quick, but don't care too much | |
| 13:45:48 | gmann | janki: where i can see the nova logs? i see all networking things there | |
| 13:46:10 | stephenfin | efried: Ah, cool, just making sure. I don't do it because it was never something that was taught to do, though I have been known to strip them out when I rewrite stuff. All good point though | |
| 13:46:15 | stephenfin | efried: Yes, without | |
| 13:46:18 | dansmith | mriedem: if it were master I'd want it fixed, but I guess you could make the argument that it doesn't matter on a frozen branch | |
| 13:46:38 | peter-hamilton | hi everyone, i'm hoping to get final feedback on the updated cert validation spec: https://review.openstack.org/#/c/488541/ | |
| 13:46:44 | mriedem | dansmith: i'll fix it quick, 2 minutes | |
| 13:46:45 | peter-hamilton | let me know if you have any questions | |
| 13:46:48 | janki | gmann, https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-pike-upstream-stateful-carbon/36/control_1/n-api.log.2017-10-16-060444.gz | |
| 13:47:15 | mdbooth | dansmith: The problem with that is the hair pulling when some random other test fails because of it, and then you end up having to bisect a testrun to determine the ordering which causes a failure. | |
| 13:47:21 | janki | gmann, nova-cpu logs https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-pike-upstream-stateful-carbon/36/compute_1/n-cpu.log.2017-10-16-060456.gz | |
| 13:47:30 | dansmith | mdbooth: that's what I said in my review, yes | |
| 13:47:51 | dansmith | mdbooth: the thing is that it's in a branch that shouldn't really get a lot of debug anymore | |
| 13:48:04 | mdbooth | dansmith: I was agreeing with you, but with additional angst. | |
| 13:48:35 | mdbooth | lol | |
| 13:48:56 | johnthetubaguy | gmann: there is a bit of wording in there I am not totally sure about, agreed with what I think you mean. | |
| 13:50:06 | gmann | johnthetubaguy: right, ll update thanks | |
| 13:50:18 | gmann | janki: error is floating ip pool is not found | |
| 13:51:00 | janki | gmann, yes. because the API call to create it fails right. | |
| 13:51:21 | gmann | janki: API fail because there is no floating ip pool in your env | |
| 13:52:06 | janki | gmann, isnt that what /compute/v2.1/os-floating-ips do? | |
| 13:52:36 | gmann | janki: you mean POST? | |
| 13:53:00 | janki | gmann, ya. POST on compute/v2.1/os-floating-ips returns 404 | |
| 13:53:01 | gmann | janki: error is raised from here but i can double check the logs to confirm them same - https://github.com/openstack/nova/blob/fce56ce8c04b20174cd89dfbc2c06f0068324b55/nova/network/neutronv2/api.py#L1985 | |
| 13:54:37 | janki | gmann, https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-pike-upstream-stateful-carbon/36/tempest/tempest.log.gz - search for os-floating-ips | |
| 13:56:23 | openstackgerrit | Matthew Booth proposed openstack/nova master: libvirt: Don't VIR_MIGRATE_NON_SHARED_INC without migrate_disks https://review.openstack.org/507202 | |
| 13:58:03 | dansmith | mriedem: fyi, this is the last thing i think is critical to land for fixing up our weird cell0 listing wart: https://review.openstack.org/#/c/508595/8 | |
| 13:58:14 | dansmith | the smartness patches after that are not critical, just gravy | |
| 13:58:38 | dansmith | that jenkins -1 isn't going to disappear, in case that has been deterring review | |
| 13:58:50 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: libvirt: add check for VIR_DOMAIN_BLOCK_REBASE_COPY_DEV https://review.openstack.org/512406 | |
| 13:58:51 | mriedem | dansmith: ^ | |
| 13:59:07 | mriedem | dansmith: ok, what's been deterring review is the looming newton eol and spec freeze | |
| 13:59:21 | dansmith | mriedem: ack | |
| 13:59:47 | gmann | janki: yea floating ip pool is needed to create the floating ip and if there is nothing then raise error. | |
| 14:00:07 | gmann | janki: what is value of default_floating_pool in conf? under default or neutron section | |
| 14:00:28 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Extract instance allocation removal code https://review.openstack.org/513041 | |
| 14:00:34 | dansmith | mriedem: then speaking of spec freeze: https://review.openstack.org/#/c/510244/ | |
| 14:00:39 | openstackgerrit | Merged openstack/nova-specs master: Add pagination and timestamp filtering support for os-migrations API https://review.openstack.org/506030 | |
| 14:00:46 | dansmith | we *have* to have that for a variety of things | |
| 14:01:16 | dansmith | mriedem: it's a thick review, so you might just want to assume that efried has worded the ass off it and stamp it through | |
| 14:02:03 | efried | stephenfin Are you still looking at ^? | |
| 14:03:10 | janki | gmann, its "public" under [DEFAULT] In nova.conf | |
| 14:03:28 | janki | gmann, these will also depend on values in tempest.conf right? | |
| 14:03:34 | stephenfin | efried: At what now? | |
| 14:03:46 | efried | stephenfin The granular resource request spec | |
| 14:04:14 | stephenfin | efried: It's on my backlog, but I think I saw it merge this morning? | |
| 14:04:23 | stephenfin | Or at least get some +2s? | |
| 14:04:51 | efried | stephenfin Not merged yet, has dansmith +2 and some +1s. | |
| 14:05:42 | gmann | janki: hat tests did not pass the pool so default is being used and not found in neutron | |
| 14:05:55 | gmann | *that | |
| 14:07:15 | janki | gmann, ohhkk..so what next? I still doubt if tempest.conf has anything to do with this | |
| 14:08:17 | stephenfin | efried: Then yes, I should get to it before EOD | |