| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-20 | |||
| 18:06:07 | mriedem | beyond performance? | |
| 18:06:27 | mnaser | rabbitmq is the worlds biggest pita :( | |
| 18:07:55 | mnaser | ok so at least to clean things up i will update the instance_mappings to point that the right cell | |
| 18:08:01 | mnaser | and then at least the instance will be delete-able | |
| 18:08:12 | mnaser | and then the cell0 will just get archived and disappear | |
| 18:08:52 | mriedem | dansmith: i'm not sure what you mean by "already a host mapping" in bury_in_cell0 | |
| 18:09:09 | dansmith | mriedem: when we bury in cell0, we also have to create a mapping for it to be there | |
| 18:09:22 | dansmith | mriedem: we should fail to do that if we're late to the party because we're a failed reschedule right? | |
| 18:09:37 | mnaser | hmm yeah, that's true, i guess the bury doesn't check if there's an existing one there | |
| 18:09:43 | mriedem | dansmith: so here https://github.com/openstack/nova/blob/stable/stein/nova/conductor/manager.py#L1282 ? | |
| 18:09:52 | mriedem | before doing that, double check that instance_mapping.cell_mapping is None, right? | |
| 18:11:03 | mriedem | i mean we'd want to check way before that though, before this https://github.com/openstack/nova/blob/stable/stein/nova/conductor/manager.py#L1260 | |
| 18:11:41 | mriedem | i'm not opposed to adding a sanity check | |
| 18:11:48 | mriedem | worst case is it doesn't hurt anything, | |
| 18:11:58 | mriedem | best case is it avoids crazy rabbit clustering split brain weirdness | |
| 18:12:10 | dansmith | mriedem: well, I'm thinking more like direct hit the database to make sure | |
| 18:12:21 | dansmith | mriedem: but again, we should fail to create a duplicate mapping there | |
| 18:13:10 | mnaser | is this just a matter of `if inst_mapping.cell_mapping is not None:` before updating it? | |
| 18:14:22 | dansmith | no | |
| 18:19:52 | dansmith | mriedem: oh, hmm | |
| 18:20:02 | dansmith | mriedem: we look up the mapping and set it to cell0 there and then save, not a create | |
| 18:20:05 | dansmith | that's why no dupe | |
| 18:20:21 | dansmith | so maybe the cell0 save happens first and then the cell1 one comes in later? | |
| 18:20:31 | dansmith | so, mnaser maybe yes to your above question | |
| 18:20:44 | dansmith | I was forgetting how this worked, that we create the mapping early and then update it later | |
| 18:21:34 | mriedem | mnaser: yes that's what i was thinking | |
| 18:21:37 | mriedem | dansmith: the create happens in the api | |
| 18:21:45 | mriedem | atomically with the build request and request spec | |
| 18:21:55 | mriedem | yeah that (sorry was catching up) | |
| 18:22:36 | mriedem | mnaser: so i'd move the code that gets the instance mapping earlier before we create the instance record in cell0, check if instance_mapping.cell_mapping is not None and if not log an error or something and return | |
| 18:22:57 | mriedem | note we still have a "remove after ocata" thing in that method :) | |
| 18:23:30 | dansmith | mriedem: move the getting of the mapping? I would think you'd want it very tight like it is now so you had the best opportunity to see that it has been set by a competitor | |
| 18:37:35 | mriedem | dansmith: as long as it's before we create the instance record in cell0 | |
| 18:38:13 | mriedem | b/c otherwise you have to roll that back | |
| 18:38:24 | mriedem | anyway, i think the shed should be bronw | |
| 18:38:26 | mriedem | *brown | |
| 18:43:01 | dansmith | maybe we're talking about different things | |
| 18:47:17 | mriedem | mnaser: were you going to push a patch for this sanity check? | |
| 18:47:24 | mriedem | it'd be easier to discuss in review | |
| 19:18:32 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Poison netifaces.interfaces() in tests https://review.opendev.org/671773 | |
| 19:37:45 | mnaser | mriedem: i was *if* i was talking about the right thing | |
| 19:38:05 | mnaser | even if i pushed the initial patch tbh i dont know if i have enough bandwidth to drive it all the way through review and all right now | |
| 19:38:11 | mnaser | but i can do the initial if statement before the update there.. | |
| 19:38:17 | mnaser | with a test | |
| 19:40:11 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Rename Claims resources to compute_node https://review.opendev.org/679470 | |
| 19:45:37 | mriedem | travis --com, travis --org, what a mess | |
| 19:46:31 | mriedem | mnaser: that's good enough to start, it shouldn't be a difficult change, i think the complexity is mostly in the test since we want to assert that (1) we fail if the instance mapping already has a cell mapping set in _bury_in_cell0 and (2) we check that before calling instance.create | |
| 20:24:45 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Clear instance.launched_on when build fails https://review.opendev.org/683725 | |
| 21:01:44 | mriedem | maybe we should finally fix that bug where nova-compute creates volumes but doesn't name them... | |
| 21:10:44 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Sanity check instance mapping in _bury_in_cell0 https://review.opendev.org/683730 | |
| 21:11:04 | mriedem | mnaser: dansmith: ^ this is what i was thinking | |
| 21:11:10 | mriedem | it's a bit fuglier than i expected | |
| 21:12:15 | dansmith | mriedem: okay I thought you meant adding it quite a bit earlier | |
| 21:12:19 | dansmith | er, moving | |
| 21:12:23 | mriedem | so did i, | |
| 21:12:26 | mriedem | but realized we have a list here | |
| 21:12:31 | mriedem | so it has to go in the loop | |
| 21:12:42 | dansmith | mriedem: the other thing is, we should probably do the same sanity check when we set it for a non cell0 cell, | |
| 21:12:51 | dansmith | because if cell0 is faster (likely) then we'll create it there and map it, | |
| 21:12:56 | dansmith | then remap it to cell1 | |
| 21:13:05 | dansmith | and we should at least warn that that happened for the forensic value | |
| 21:13:08 | mriedem | yeah - can you leave a comment so i don't forget? | |
| 21:13:12 | dansmith | I mean...if we think this is what is happening | |
| 21:14:05 | dansmith | done | |
| 21:14:09 | mriedem | danke | |
| 21:18:37 | mriedem | replied with a question, | |
| 21:18:50 | mriedem | mostly about leaving 2 copies in different dbs which could mess up listing, | |
| 21:19:16 | mriedem | i guess the warning could just be "uh oh spaghettios this is in cell0 and now it's in cell1 too - we're going to trust the cell1 version but you'll need to cleanup cell0" | |
| 21:20:11 | dansmith | it's going to be in both places anyway, apparently, so mapped appropriately and logged is minimal best case I think | |
| 21:20:28 | dansmith | you could also nuke the cell0 version, but I kinda want to see evidence that this is really happening first | |
| 21:21:28 | mriedem | right i don't want to overengineer this | |
| 21:21:35 | mriedem | anyway, not going to happen today, i'm taking off | |
| 21:21:55 | dansmith | ack | |
| 22:07:12 | mlavalle | I am working on a devstack built with stable rockey. I modified and restarted the nova api with the following changes: https://review.opendev.org/#/c/674038 and https://review.opendev.org/#/c/645452. I also modified https://github.com/openstack/nova/blob/master/nova/policies/servers.py#L58 to base.RULE_ADMIN_API or base.SYSTEM_READER | |
| 22:08:37 | mlavalle | granted alt_demo user the reader role with system(all) scope. alt_demo is still unbale to do list server --all-users. what am I missing? | |
| 22:09:18 | mlavalle | granted alt_demo user the reader role with system(all) scope. alt_demo is still unbale to do list server --all-projects. what am I missing? | |
| 22:09:57 | mlavalle | lbragstad: ^^^^ any advice? | |
| 22:14:31 | lbragstad | mlavalle what kind of token are you using to make the request to nova? | |
| 22:19:05 | lbragstad | mlavalle you could put something like https://pasted.tech/pastes/7d06348fdea072ad4784fa75940c142dc3d63f86.raw in your clouds.yaml and then export OS_CLOUD=devstack-alt-system to ensure you're using a system-scoped token in your request | |
| 22:19:36 | lbragstad | i assume you're using openstackclient | |
| 22:19:37 | mlavalle | lbragstad: duuh. that was the problem. I was getting a project scoped token. I just tested with a system scoped token and it worked | |
| 22:19:47 | lbragstad | mlavalle nice | |
| 22:19:51 | mlavalle | thanks! | |
| 22:19:59 | mlavalle | have a nice weekend | |
| 22:20:00 | lbragstad | no problem | |
| 22:20:02 | lbragstad | you, too | |
| #openstack-nova - 2019-09-21 | |||
| 00:47:44 | openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP: Provider Config File: Merge provider configs to provider tree https://review.opendev.org/676522 | |
| 01:41:41 | openstackgerrit | Merged openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571 | |
| 01:41:49 | openstackgerrit | Merged openstack/nova master: tests: Additional functional tests for pinned instances https://review.opendev.org/681750 | |
| 01:53:01 | openstackgerrit | Merged openstack/nova master: Include both VCPU and PCPU in core quota count https://review.opendev.org/681374 | |
| 02:07:11 | openstackgerrit | Merged openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 02:07:19 | openstackgerrit | Merged openstack/nova master: fakelibvirt: Make 'Connection.getHostname' unique https://review.opendev.org/681060 | |
| 02:07:29 | openstackgerrit | Merged openstack/nova master: libvirt: Mock 'libvirt_utils.file_open' properly https://review.opendev.org/681061 | |
| 02:07:38 | openstackgerrit | Merged openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 03:40:34 | openstackgerrit | Merged openstack/nova master: libvirt: Enable driver configuring PMEM namespaces https://review.opendev.org/679640 | |
| 03:40:42 | openstackgerrit | Merged openstack/nova master: libvirt: Correctly handle non-CPU flag traits https://review.opendev.org/681932 | |
| 05:15:29 | openstackgerrit | Merged openstack/nova master: libvirt: Get the CPU model, not 'arch' from get_capabilities() https://review.opendev.org/682267 | |
| 10:39:44 | stephenfin | About damn time ^^^ | |
| 11:06:30 | openstackgerrit | Merged openstack/nova master: trivial: Use sane indent https://review.opendev.org/680229 | |
| 13:43:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: Stop using volume_api.initialize/terminate_connection https://review.opendev.org/682508 | |