| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-18 | |||
| 12:29:57 | janki | sean-k-mooney, ya. looks like os-extended-server-attributes is a collection of REST APIs exposed by Nova. right? | |
| 12:30:33 | sean-k-mooney | janki: it was technically a buch of rest apis exposed by a nova api extention | |
| 12:31:04 | sean-k-mooney | janki: as of pike we nolonger support extending the api like that but the existing extions were more or less kept | |
| 12:31:59 | sean-k-mooney | janki: i think the useful ones will be adopted into the main api and the unsed ones will be drop. i would expect the extended server attribute to be adopted in to the server resouce | |
| 12:32:14 | janki | sean-k-mooney, ohk. so I am getting 404 on os-floating-ips and policy error for os-extended-server-attributes. could this be related? | |
| 12:32:51 | sean-k-mooney | janki: if you query with admin prviliges dose the same happen | |
| 12:33:16 | sean-k-mooney | janki: the reason for the clean up spec is that there are two set of policies applied to the extentions | |
| 12:33:36 | sean-k-mooney | the main api policies and the extention level policies that we are looking to remove | |
| 12:34:02 | janki | sean-k-mooney, I dont have acces to the setup. Its jenkins build. But I think the naswer is no. there is 1 more variable in policy.json "os_compute_api:os-floating-ips": "rule:admin_or_owner" so there are not related | |
| 12:36:45 | sean-k-mooney | dumb question im assumeing the setup is using neutron, im not sure how the extentions were enabled in the past but if it was using nova-network you would have no floating-ips | |
| 12:37:37 | alex_xu | janki: it looks like the floating pool isn't there | |
| 12:38:22 | janki | alex_xu, yes. because the API that creates floating ip is failing | |
| 12:39:24 | alex_xu | janki: ok, great, you already know that, I didn't read the full chat log yet :) | |
| 12:40:29 | janki | alex_xu, I am trying to figure out the reason for the API to fail. I checked, with help of efried, that microversion used is 2.1 (< 2.36). | |
| 12:42:11 | janki | alex_xu, also there is policy check failure for os-extended-server-attributes in log but no such failure for os-floating-ips | |
| 12:44:33 | alex_xu | janki: they are sounds unrelated | |
| 12:45:09 | janki | alex_xu, Ya. initially I thought they are but looking at sample policy file, I do think the same | |
| 12:48:22 | openstackgerrit | Jan Zerebecki proposed openstack/nova master: Only log not correcting allocation once per period https://review.openstack.org/508262 | |
| 12:50:53 | openstackgerrit | Jan Zerebecki proposed openstack/nova master: Only log not correcting allocation once per period https://review.openstack.org/508262 | |
| 12:55:22 | alex_xu | janki: looks like you needn't worry about that policy check failure, the log will be emitted each time when the user doesn't have the permission | |
| 12:55:33 | alex_xu | and that log looks like annoying | |
| 12:56:36 | janki | alex_xu, well then, the next step is look into tempest.conf | |
| 13:00:45 | alex_xu | nova api meeting is running at #openstack-meeting-4 | |
| 13:01:34 | mriedem | sdague: johnthetubaguy: can you take a look at this ocata-only fix? https://review.openstack.org/#/c/512406/ turns out we backported something awhile ago that requires some special handling based on the version of libvirt you're running | |
| 13:03:39 | gmann | janki: what is error actually, i can help from tempest.conf side | |
| 13:06:28 | janki | gmann, so I am running tempest for ODL + Pike setup and https://github.com/openstack/tempest/edit/master/tempest/scenario/test_server_basic_ops.py is failing with error https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-pike-upstream-stateful-carbon/36/tempest/tempest_results.html.gz | |
| 13:07:05 | stephenfin | mriedem: Any suggestions on who else I can ask to review https://review.openstack.org/#/c/361140/ in jaypipes' absence? | |
| 13:08:06 | janki | gmann, as per https://developer.openstack.org/api-ref/compute/#create-allocate-floating-ip-address, I verified that microverion is 2.1 and there are no logs about policy failure for os-floating-ips | |
| 13:08:28 | mriedem | stephenfin: i think that dansmith guy is pretty smart | |
| 13:09:00 | janki | gmann, next is to check tempest.conf for proper flag settings | |
| 13:10:32 | gmann | janki: its not policy related, what FIP pool you configured on nova> | |
| 13:11:29 | janki | gmann, I didnot manually configure anything. These logs are from jenkins build. Which file would those be in? I can dig that file up | |
| 13:11:54 | stephenfin | mriedem: I agree: dansmith is a super smart guy who'd surely love to review https://review.openstack.org/#/c/361140/ | |
| 13:11:57 | gmann | janki: sure in api meeting, ll ping/debug after that | |
| 13:12:18 | janki | gmann, ack. | |
| 13:13:08 | stephenfin | Then again, if jaypipes is back from the dead, he might also like to review https://review.openstack.org/#/c/361140/ ... | |
| 13:13:14 | stephenfin | No rest for the weary :) | |
| 13:26:42 | efried | stephenfin I'm probably just missing something fundamental on https://review.openstack.org/#/c/361140/ | |
| 13:29:28 | lyarwood | mdbooth: re https://github.com/openstack/nova/commit/29735973336b1038a2a9fb9072049fe3ed151502 - do you recall why context.auth_token isn't set? | |
| 13:29:48 | mdbooth | lyarwood: looking | |
| 13:29:51 | lyarwood | mdbooth: https://github.com/openstack/nova/commit/5e650e3681d40069dacf1ea2e43b07b362cf1bc3 that introduced the workaround doesn't really say why it isn't there via init_host | |
| 13:29:54 | cdent | efried: going back through the log looking at the conversation you had with sean-k-mooney and rgerganov; where in the powervm universe is the mapping from rp uuid to <other> going to live? | |
| 13:30:42 | stephenfin | efried: The problem I'm trying to get at is, even if we somehow managed to keep non-PCI-needing instances off of PCI-having NUMA nodes, we can still end up in the situation where we have no free PCI-having NUMA nodes for PCI-needing instances | |
| 13:31:09 | mdbooth | lyarwood: Could it be cause there's no request context at that point? | |
| 13:31:18 | mdbooth | i.e. no user, no api call? | |
| 13:31:26 | efried | cdent That's a pretty big TBD. We don't have any persistent data on our "hypervisor" (the NovaLink partition), which is a fairly fundamental point of architecture. You're supposed to be able to trash the partition and recreate it with no loss. | |
| 13:31:32 | stephenfin | This approach reduces the possibility but it doesn't mitigate it entirely. We want to use the two in tandem, which is what you've kind of hinted at (I think) | |
| 13:31:45 | lyarwood | mdbooth: yeah sorry, https://github.com/openstack/nova/blob/master/nova/context.py#L279 | |
| 13:31:59 | sean-k-mooney | stephenfin: that was the usecase we created https://github.com/openstack/nfv-filters/blob/master/nfv_filters/nova/scheduler/filters/aggregate_instance_type_filter.py to address | |
| 13:32:11 | openstackgerrit | Chris Dent proposed openstack/nova-specs master: Add spec for symmetric GET and PUT of allocations https://review.openstack.org/508164 | |
| 13:32:16 | efried | stephenfin Yes. You'll never get around that limitation. Unless you want to totally deny non-PCI-needing instances from booting on PCI-having NUMA nodes. Which ain't reasonable IMO. | |
| 13:32:43 | sean-k-mooney | it allows you to create aggregates of nodes with scarce resouces and require that they are requested in the flavor to schedule to those nodes | |
| 13:32:55 | stephenfin | efried: But you will with this spec, which allows booting PCI-needing instances from using non-PCI-having NUMA nodes | |
| 13:32:58 | efried | cdent Off the cuff, the idea would be to make some part of the RP relate to some part of whatever thingy we're "mapping" to. | |
| 13:33:16 | sean-k-mooney | this filter will work with traits in request in the flavor too by the way | |
| 13:33:19 | mdbooth | lyarwood: Yeah, that would be required for a glance request, I guess | |
| 13:33:24 | stephenfin | sean-k-mooney: I saw that, but it wasn't too inflexible, hence https://blueprints.launchpad.net/nova/+spec/reserve-numa-with-pci | |
| 13:33:41 | mdbooth | Although there's useful work it can do without that | |
| 13:33:50 | sean-k-mooney | that is your new weigher right | |
| 13:34:00 | lyarwood | mdbooth: it's the cinder encryption metadata lookup that's failing at the moment | |
| 13:34:08 | sean-k-mooney | stephenfin: i tought that merged in pike? | |
| 13:34:24 | efried | stephenfin The "problem" you won't get around is where I boot a hundred non-PCI-needing instances and run out of non-PCI-having NUMA nodes after the first fifty, so I start using my PCI-having NUMA nodes. Once those are full, I can no longer boot "required"-affinity PCI-needing instances. | |
| 13:34:28 | stephenfin | sean-k-mooney: Correct. If you had to divide your cloud into aggregates for PCI devices, it made it very inflexible if you wanted to, say, start using a lot of PCI-having instances or vice versa | |
| 13:34:43 | stephenfin | sean-k-mooney: It did. I'm saying https://review.openstack.org/#/c/361140/ forms a one-two combo :) | |
| 13:35:03 | mdbooth | lyarwood: Hmm, yeah. That's not gonna work unless we've got some special-sauce cinder admin login, or we cached it. | |
| 13:35:57 | stephenfin | efried: Yup, there's nothing we can do about that, I'm afraid. If you want to use 'required' then this is what you signed up for. Go buy more PCI devices :P | |
| 13:36:03 | sean-k-mooney | stephenfin: right we really need to merge https://review.openstack.org/#/c/361140/29/specs/queens/approved/share-pci-between-numa-nodes.rst and implemented this cycle it was originally ment to work in icehouse.. | |
| 13:36:33 | efried | stephenfin Cool, then we're on the same page mentally; but then I don't think that chunk of the spec is expressing any reasonable "Alternatives". | |
| 13:36:44 | cdent | efried: k, thanks. I’d rather we avoid putting that kind of mapping into placement if possible, but it seems there is a fairly generic need for a translation. I presumem using rp.name is insufficient. Next comes the sort of config info that rgerganov mentioned (things like the IP for the shared storage piece?). That, to me, especially should not go into placement as we’re doing a serious mission creep at that point. | |
| 13:36:46 | stephenfin | efried: However, the https://blueprints.launchpad.net/nova/+spec/reserve-numa-with-pci should help mitigate that, and the 'preferred' policy in https://review.openstack.org/#/c/361140/ should solve it completely | |
| 13:36:54 | sean-k-mooney | stephenfin: as an employee of a hardware vendor that sells pic devices i feel oblidged to +1 that responce :P | |
| 13:37:02 | stephenfin | *if* you're ok living with the downsides (possible performance impact) | |
| 13:37:13 | stephenfin | sean-k-mooney: :P | |
| 13:37:31 | stephenfin | efried: Sure, I'm happy to drop it, if that makes sense? :) | |
| 13:37:46 | stephenfin | *if that would help? | |
| 13:37:57 | efried | stephenfin If you don't mind, perhaps give me a crack at rewording it? | |
| 13:38:04 | stephenfin | efried: Knock yourself out | |
| 13:38:32 | openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Add pagination and timestamp filtering support for os-migrations API https://review.openstack.org/506030 | |
| 13:38:46 | markus_z | Here's a doc change waiting for review for 5 weeks: https://review.openstack.org/#/c/476188/ | |
| 13:38:55 | stephenfin | efried: Also, saw your reply on the "two spaces indicate age" comment. It was just an amusing article I read once upon a time that stuck with me. Not an actual ageist thing or anything! | |
| 13:39:19 | efried | cdent I agree with you (at least in the sense that I've accepted the stated limitations of the placement architecture). We could conceivably get some mileage out of overloading the 'name' field, or embedding information into gross custom traits. But I would be wanting to look for cleaner alternatives pretty hard before resorting to that. | |
| 13:39:21 | stephenfin | markus_z: Sure, I'll take a look | |
| 13:39:27 | markus_z | stephenfin: thanks! | |
| 13:39:58 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Explain the flow of the "serial console" feature https://review.openstack.org/476188 | |
| 13:41:33 | efried | stephenfin No offense taken, of course. It's an argument that's been made to me before, so you may have caught some blowback from arguments I've had with others who were... um... more serious about it (to the point of -1ing stuff). My take on it is that the reasons for "never do that" don't apply to programmers. | |
| 13:42:55 | dansmith | mriedem: tell me you don't care and I'll +W: https://review.openstack.org/#/c/512406/3 | |
| 13:44:17 | openstackgerrit | Ghanshyam Mann proposed openstack/nova-specs master: Spec for API extensions policy removal https://review.openstack.org/508101 | |
| 13:44:48 | efried | stephenfin Before I start, can you confirm that https://review.openstack.org/#/c/361140/29/specs/queens/approved/share-pci-between-numa-nodes.rst@131 was indeed supposed to be "without"? | |
| 13:45:29 | gmann | johnthetubaguy: sdague alex_xu ^^ updated spec - https://review.openstack.org/#/c/508101/ | |
| 13:45:46 | mriedem | dansmith: i could fix quick, but don't care too much | |
| 13:45:48 | gmann | janki: where i can see the nova logs? i see all networking things there | |
| 13:46:10 | stephenfin | efried: Ah, cool, just making sure. I don't do it because it was never something that was taught to do, though I have been known to strip them out when I rewrite stuff. All good point though | |
| 13:46:15 | stephenfin | efried: Yes, without | |
| 13:46:18 | dansmith | mriedem: if it were master I'd want it fixed, but I guess you could make the argument that it doesn't matter on a frozen branch | |
| 13:46:38 | peter-hamilton | hi everyone, i'm hoping to get final feedback on the updated cert validation spec: https://review.openstack.org/#/c/488541/ | |
| 13:46:44 | mriedem | dansmith: i'll fix it quick, 2 minutes | |
| 13:46:45 | peter-hamilton | let me know if you have any questions | |
| 13:46:48 | janki | gmann, https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-pike-upstream-stateful-carbon/36/control_1/n-api.log.2017-10-16-060444.gz | |
| 13:47:15 | mdbooth | dansmith: The problem with that is the hair pulling when some random other test fails because of it, and then you end up having to bisect a testrun to determine the ordering which causes a failure. | |
| 13:47:21 | janki | gmann, nova-cpu logs https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-pike-upstream-stateful-carbon/36/compute_1/n-cpu.log.2017-10-16-060456.gz | |