| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-15 | |||
| 19:49:04 | mriedem | you live in portland, you can get down with that | |
| 19:49:49 | dansmith | heh | |
| 19:59:13 | openstackgerrit | Corey Bryant proposed openstack/nova master: add python 3.7 unit test job https://review.openstack.org/610694 | |
| 19:59:53 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: API microversion 2.69: Handles Down Cells https://review.openstack.org/579563 | |
| 20:01:00 | dansmith | mriedem: comments on the docs patch.. I could just +2 and we could discuss/fix later, but probably not as much of a rush on that one I'm thinking | |
| 20:04:38 | tssurya | mriedem, dansmith: you had to respin for the links part ? sorry about that and again my never-ending list of "thank you"'s | |
| 20:07:03 | openstackgerrit | Corey Bryant proposed openstack/python-novaclient master: add python 3.7 unit test job https://review.openstack.org/637290 | |
| 20:11:31 | mriedem | dansmith: tssurya: i also just thought about this https://review.openstack.org/#/c/591657/44/nova/api/openstack/compute/services.py@78 | |
| 20:13:38 | tssurya | mriedem: hmm so you want it to be false if those filters are passed ? | |
| 20:14:11 | mriedem | dansmith: yeah i'm ok with those docs changes | |
| 20:14:15 | mriedem | tssurya: idk, hence the question | |
| 20:14:34 | mriedem | tssurya: if we were being consistent, we'd set cell_down_support=False if there were any filters on the request | |
| 20:14:37 | dansmith | I don't really have an opinion on the servers stuff | |
| 20:14:40 | dansmith | er, services | |
| 20:14:54 | mriedem | but like i said, we're not filtering in the db query, we're doing it in python once we get results | |
| 20:14:58 | mriedem | so *shrug*? | |
| 20:16:18 | tssurya | yea | |
| 20:16:38 | tssurya | besides technically the edge cases are only for "listing servers" :D | |
| 20:17:10 | tssurya | at least in the docs and everywhere its only for the server details | |
| 20:19:46 | mriedem | oh well if the docs say so... | |
| 20:20:41 | mriedem | anyway i'm fine with the way it is, not really worth losing sleep over it | |
| 20:31:44 | mriedem | looks like a whole bunch of 3rd party CIs are failing on this | |
| 20:31:44 | mriedem | ConnectionError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (Caused by ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)",)) | |
| 20:43:30 | fried_rice | okay, but cmon, do we actually *need* PyMySQL? | |
| 20:43:55 | melwitt | nah | |
| 20:44:07 | fried_rice | tssurya: Did you see the patch I proposed to improve the ironic ufpt thing? | |
| 20:44:34 | fried_rice | gibi: Is it too late to make the bw part of the binding profile look more like the resource request syntax in extra specs? | |
| 20:45:10 | tssurya | fried_rice: saw the WIP, but didn't review it yet | |
| 20:45:36 | fried_rice | tssurya: Less about review, more about trying it in your env and seeing if it takes the 6h processing ironic nodes down to something sane. | |
| 20:46:07 | tssurya | haha we kind of hacked the bit of code that tries to do the N^2 thing already got the node up | |
| 20:46:11 | tssurya | for now | |
| 20:46:32 | tssurya | I will be able to try your patch only on Monday :( | |
| 20:46:42 | fried_rice | tssurya: yeah, IIUC you're only processing the one node, but that's not a long term solution. | |
| 20:46:44 | fried_rice | okay | |
| 20:47:04 | tssurya | but trust me getting that to work properly is our priority | |
| 20:47:08 | fried_rice | gibi: IMO that's a more consistent user experience; and then also we would get to use common code for parsing into RequestGroup etc. | |
| 20:47:21 | tssurya | your concern is for provider sharing right ? | |
| 20:48:10 | fried_rice | tssurya: That's one example. It's really a question of why that code is there in the first place: it's because upt is allowed to muck with anything in the ProviderTree object. | |
| 20:48:30 | fried_rice | tssurya: Like what if ironic actually *wanted* to dork with more than one node at a time? | |
| 20:53:00 | tssurya | fried_rice: since we are talking about why that code exists, can you give me a ore concrete example of when ironic would want that ? | |
| 20:53:21 | tssurya | like I am trying to understand more use cases for this whole thing | |
| 20:53:45 | tssurya | since I am pretty sure we don't need them for now right ? | |
| 20:53:54 | tssurya | or maybe I am missing some documentation | |
| 20:54:33 | fried_rice | tssurya: Yeah, I don't have a specific use case; I could only contrive hypothetical ones. It's just that update_provider_tree is a generic method which passes the *whole* ProviderTree object down to the virt driver for modification, and then update_from_provider_tree is responsible for flushing any changes back to placement. | |
| 20:54:50 | fried_rice | I really didn't want upt/ufpt to be tightly bound to specific virt drivers' implementations or needs in that regard. | |
| 20:55:11 | fried_rice | and it's specifically designed to be future-looking to when we have nested providers, sharing providers, etc. | |
| 20:55:24 | fried_rice | ...just like the code that broke your Queens performance :) | |
| 20:55:37 | tssurya | yea I get that, its just that for ironic I don't know how this would scale with the more nodes that we would keep adding in future | |
| 20:55:38 | fried_rice | ...which wasn't actually going to be used until... I think stein actually. | |
| 20:55:56 | cdent | mriedem: i'll either get to it or get gary, but yeah | |
| 20:55:59 | fried_rice | tssurya: Well, let's see how this does. | |
| 20:56:35 | fried_rice | tssurya: Note that there are already a number of anticipatory optimizations in that code. | |
| 20:56:53 | fried_rice | tssurya: Like the fact that we do local compares and only call out to placement if something has changed. | |
| 20:57:20 | openstackgerrit | melanie witt proposed openstack/nova master: Add user_id field to InstanceMapping https://review.openstack.org/633350 | |
| 20:57:20 | openstackgerrit | melanie witt proposed openstack/nova master: Add online data migration for populating user_id https://review.openstack.org/633351 | |
| 20:57:27 | tssurya | fried_rice: yea I mean as far as most of the things we do right now the flushes to placement are very less | |
| 20:57:29 | fried_rice | tssurya: I just didn't anticipate that iterating through a list would bring the service to its knees. Even if you had told me the list was going to be a couple thousand long. | |
| 20:58:43 | tssurya | fried_rice: of course yea we didn't know this as well until we timed this, our test env is not exactly the size of our actual prod env | |
| 21:03:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Check hosts have no instances for AZ rename https://review.openstack.org/509206 | |
| 21:03:32 | mriedem | sounds like someone needs their nova-computes running on s390x mainframes | |
| 21:03:39 | mriedem | or Power9! | |
| 21:04:34 | tssurya | fried_rice: just saw your patch, so basically you are saving on the lookup time instead of iterating.. We will try this first thing Monday and let you know | |
| 21:04:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Check hosts have no instances for AZ rename https://review.openstack.org/509206 | |
| 21:05:21 | fried_rice | tssurya: Yes. We already have the list of UUIDs. Now instead of that loop taking O(N) to look up each node's provider object, it'll take O(1). | |
| 21:05:58 | tssurya | yeap nice | |
| 21:06:46 | fried_rice | tssurya: now, it's possible that that lookup wasn't the problem: it's possible that what's killing us is 1700 ProviderTree.data() calls (which copies the guts of the ProviderTree) plus 5100 compares of that information. | |
| 21:06:58 | fried_rice | that may be a tad harder to optimize. | |
| 21:08:08 | fried_rice | (...guts of the individual _Provider in the ProviderTree, that is) | |
| 21:15:18 | tssurya | yea you mean this part here (https://github.com/openstack/nova/blob/880327cc31fea7328d23355730d5458f3b74662b/nova/scheduler/client/report.py#L1440) | |
| 21:15:35 | tssurya | and then the set aggregates and inventories comparisions | |
| 21:24:34 | melwitt | mriedem: if I'm trying to remove the console-auth workaround in stein and nova-status upgrade check has the check for console auths, where it warns if compute services older than rocky are found and advises to set the [workarounds] option if so. if I'm removing the workaround, it seems like I should remove the upgrade check. is that right? | |
| 21:30:17 | mriedem | does the upgrade check just warn today? | |
| 21:30:23 | melwitt | yes | |
| 21:30:25 | mriedem | and is that upgrade check in stable/rocky? | |
| 21:30:45 | mriedem | i think you'd either remove the upgrade check in stein or change it to a hard failure | |
| 21:30:54 | mriedem | and then maybe mark it for removal in Train | |
| 21:31:05 | melwitt | yes the check is in stable/rocky | |
| 21:31:13 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827 | |
| 21:31:13 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Introduce live_migration_claim() https://review.openstack.org/635669 | |
| 21:31:14 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606 | |
| 21:31:14 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828 | |
| 21:31:15 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: update NUMA-related XML on the source https://review.openstack.org/635229 | |
| 21:31:42 | mriedem | off the top of my head i think i'd go conservative and make the upgrade check a hard failure in stein and remove it in train | |
| 21:31:44 | melwitt | ok, I considered changing it to a hard failure if any computes older than rocky are found. I guess the action would be "upgrade your computes | |
| 21:31:55 | mriedem | because someone could run the stein upgrade checks against a rocky deployment from a venv/container | |
| 21:32:08 | mriedem | dansmith might have an opinion | |
| 21:32:15 | melwitt | I was struggling a bit with what the action item is for something that generic. all of the other failures say, "run this command" | |
| 21:32:37 | mriedem | yeah i think the only recent upgrade check we removed was the one for placement resource providers because of the extracted placement | |
| 21:32:55 | mriedem | we could have re-worked that to hit the placement API rather than the nova_api db, but it had been around since ocata so figured it was ok to just drop it | |
| 21:33:06 | mriedem | this consoleauth one seems to have really confused some people and caught them off guard | |
| 21:33:19 | mriedem | so dropping the check with the workaround in the same release might be too aggressive | |
| 21:34:11 | melwitt | I wondered that too, because of how problematic it's been. whether to just keep the workaround around another cycle | |
| 21:34:49 | mriedem | if it's not hurting anything that might be fine | |
| 21:34:55 | mriedem | safer than sorryier | |
| 21:35:12 | melwitt | I was thinking to follow through with TODOs before they get too old and forgettable but this seems like one of those where it's best not | |
| 21:35:25 | melwitt | yeah, it doesn't hurt anything | |
| 21:35:36 | melwitt | ok, well, that's easy then | |
| 21:37:03 | mriedem | we've still got lots of todos to remove things from several releases ago | |
| 21:37:16 | mriedem | hell we've still got nova-net and cells v1 | |
| 21:38:54 | melwitt | oh I know. I was thinking instead, I should propose one of the old TODOs. I think the instance group migration to API database that I worked on is still around :( | |