| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-06-26 | |||
| 20:25:06 | mriedem | dansmith: does that in any way solve that problem? | |
| 20:26:18 | mriedem | i think there is also a minimum version of oslo.db needed for this... | |
| 20:27:40 | dansmith | mriedem: hmm, I don't think so, and that syntax may break this | |
| 20:27:44 | dansmith | mriedem: a minimum oslo_db for what? | |
| 20:27:48 | mriedem | sec | |
| 20:28:06 | mriedem | https://review.openstack.org/#/c/539035/ | |
| 20:28:14 | mriedem | i assume ^ is related | |
| 20:28:59 | dansmith | mriedem: that's part of their workaround | |
| 20:29:08 | dansmith | mriedem: that isn't needed if we have this, regardless of the oslo_db version | |
| 20:33:18 | dansmith | so, that bug says, I think, that oslo.msg does a split(',') on the netloc part of the url, yeah? | |
| 20:33:39 | dansmith | that'll break my extraction of the username and password,at least, and probably the hostname | |
| 20:36:33 | dansmith | ugh, yeah | |
| 20:36:53 | dansmith | I dunno why that was done instead of just multiple full urls.. | |
| 20:56:13 | mriedem | dansmith: ok comments inline | |
| 20:56:40 | mriedem | reading mike's comment now, i suppose i should have read that first | |
| 20:57:17 | melwitt | does anyone know off the top of their head how instances can end up with no InstanceMapping? | |
| 20:58:11 | melwitt | chatting with someone from rdo cloud about it, it's happened to them a couple of times | |
| 20:58:52 | mriedem | no instance mapping or no cell mapping in the instance mapping? | |
| 20:58:53 | melwitt | it seems like something that could only happen during an upgrade, if instance(s) were somehow missed upon nova-manage cell_v2 map_instances | |
| 21:01:16 | melwitt | I'm not sure yet, will be looking at the details tomorrow (they're in EU time zone). but just wanted to ask in case this was a common thing other people have seen | |
| 21:01:36 | openstackgerrit | Merged openstack/nova master: Remove support for /os-fixed-ips REST API https://review.openstack.org/568516 | |
| 21:02:04 | melwitt | we know CERN has hit the no cell mapping in the instance mapping, and I still don't understand how that occurs | |
| 21:07:27 | mriedem | i do | |
| 21:07:38 | mriedem | well i know one way you can hit it | |
| 21:07:42 | mriedem | https://review.openstack.org/#/c/576161/ | |
| 21:11:21 | melwitt | I won't get to see what the situation is in their db until tomorrow. I doubt they're doing the ip filter though | |
| 21:13:41 | mriedem | well i also started https://review.openstack.org/#/c/575556/ to try and debug when this does happen | |
| 21:14:11 | mriedem | only need to determine if i should lookup the build request to see if we somehow missed the build request and went straight to listing instances with a marker that is on an instance not yet mapped to a cell | |
| 21:14:54 | mriedem | there is a window of time between when we create the instance in a cell and when we actually update the instance mapping in conductor (and then we delete the build request) | |
| 21:16:09 | melwitt | I think last time this happened, it was that there were a few instances in the cell0 db but they had no instance mappings, so they showed up in a 'server list' but could not be deleted because no instance mapping or no cell mapping in instance mapping | |
| 21:16:13 | mriedem | dansmith: yeah so if i create my cell mapping with multiple rabbit hosts using , it would mess up that urlparse - or transport_url for that matter right? | |
| 21:16:28 | mriedem | and transport_url is a stropt which is comma-delimited | |
| 21:16:29 | dansmith | mriedem: yes, but I've got a solution | |
| 21:16:38 | mriedem | [oslo_messaging_rabbit]/rabbit_hosts is a listopt | |
| 21:16:40 | dansmith | the weird chars in db thing is more annoying | |
| 21:16:48 | mriedem | yeah... | |
| 21:16:51 | openstackgerrit | Dan Smith proposed openstack/nova master: Allow templated cell_mapping URLs https://review.openstack.org/578163 | |
| 21:16:52 | dansmith | mriedem: checkityo ^ | |
| 21:23:37 | dansmith | mriedem: I guess I'm missing something | |
| 21:23:57 | dansmith | surely ? in the url anywhere other than before the query string is not valid right? | |
| 21:25:16 | melwitt | it could be part of the password also, right? | |
| 21:25:45 | dansmith | that's the assertion, but I'm just surprised most url parsing implementations would allow it | |
| 21:25:50 | melwitt | unless that doesn't work in general, connecting to mysql db separately | |
| 21:25:59 | dansmith | clearly python's does not | |
| 21:29:07 | dansmith | yeah, password should be url encoded | |
| 21:29:49 | dansmith | https://bugs.python.org/issue18140 | |
| 21:29:54 | dansmith | so I call BS on that | |
| 21:30:48 | dansmith | I would take the three years of no answer to that last question as "nobody has a good reason" | |
| 21:31:06 | melwitt | hm, yeah. I was thinking this same question, "Where do such unencoded URLs come from?" | |
| 21:31:30 | dansmith | same would go for having "@" or ":" in the password.. you'd *have* to urlencode it for sanity | |
| 21:31:38 | dansmith | or / or any of the other reserved characters | |
| 21:31:42 | melwitt | yeah | |
| 21:35:24 | mriedem | while on the subject, i saw this today https://bugs.launchpad.net/nova/+bug/1778515 | |
| 21:35:24 | openstack | Launchpad bug 1778515 in OpenStack Compute (nova) "nova-manage list_cells doesn't work is some special characters are in the passwords" [Undecided,New] - Assigned to Surya Seetharaman (tssurya) | |
| 21:37:25 | melwitt | oh yeah, similar deal | |
| 21:37:29 | mriedem | so apparently CERN has [ or ] but not both, or in the wrong order, in their db/mq passwords | |
| 21:37:44 | mriedem | urlparse thinks they are invalid ipv6 urls | |
| 21:39:28 | mriedem | this is the thing we use in map_cell0 https://github.com/zzzeek/sqlalchemy/blob/15ea75981305fdad8286f6803671b864ccda13f2/lib/sqlalchemy/engine/url.py#L203 | |
| 21:53:25 | openstackgerrit | Dan Smith proposed openstack/nova master: Allow templated cell_mapping URLs https://review.openstack.org/578163 | |
| 21:53:36 | dansmith | I think I got all the actionable feedback ^ but I gotta run off for a bit now | |
| 21:57:21 | mriedem | just noticed something weird, but surely i can't be the first... | |
| 21:57:29 | mriedem | we move instance groups and instance group members from the cell to api db yeah | |
| 21:57:47 | mriedem | and when we delete an instance, the instance_destroy db api code used to also remove the instance group member table entry for the deleted instance | |
| 21:57:59 | melwitt | yeah instance group stuff moved to api db | |
| 21:58:03 | mriedem | we don't have anything like that with deleting an instance, from what i can tell, when deleting an instance and the members are now in the api db | |
| 21:58:37 | mriedem | so if i create an instance in a group, then delete the instance, and show members on the group, will it show the deleted instance uuid? | |
| 21:58:57 | melwitt | hm | |
| 21:59:13 | mriedem | ah _get_not_deleted | |
| 21:59:36 | mriedem | that seems pretty inefficient | |
| 22:00:05 | mriedem | but, | |
| 22:00:20 | mriedem | if you delete the group membership in the api when deleting the instance, if we fail to delete the instance in the cell then we're hosed | |
| 22:00:28 | mriedem | similar to deleting request specs and instance mappings | |
| 22:00:45 | mriedem | seems we should clean up old instance group members during archive like we do for mappings and request specs | |
| 22:01:15 | melwitt | ah, yeah | |
| 22:01:45 | mriedem | although _get_not_deleted also relies on instance mappings, so if we're correctly deleting instance mappings for deleted / purged instances, maybe it's not a problem | |
| 22:01:57 | melwitt | I was trying to think why we didn't go ahead and delete the group member at instance_destroy time but it must be like you said | |
| 22:04:04 | mriedem | looks like we'd add something here if we wanted to https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L553 | |
| 22:05:20 | mriedem | this is where the api does the filtering https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/server_groups.py#L48 | |
| 22:05:54 | mriedem | so, we could potentially avoid passing more uuids into the cell db query https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/server_groups.py#L75 | |
| 22:09:40 | mriedem | ah well, can track it as low priority https://bugs.launchpad.net/nova/+bug/1778804 | |
| 22:09:40 | openstack | Launchpad bug 1778804 in OpenStack Compute (nova) "instance_group_member records not purged from nova_api DB for deleted instances" [Low,Triaged] | |
| 22:21:48 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove unused DB API instance_group_delete method https://review.openstack.org/578223 | |
| #openstack-nova - 2018-06-27 | |||
| 00:08:18 | openstackgerrit | karim proposed openstack/nova master: Handle rebuild of instances with image traits https://review.openstack.org/569498 | |
| 00:39:45 | openstackgerrit | karim proposed openstack/nova master: Update scheduler to use image-traits https://review.openstack.org/576054 | |
| 02:12:09 | openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Use ThreadPoolExecutor for max_concurrent_live_migrations https://review.openstack.org/563505 | |
| 02:55:32 | openstackgerrit | Merged openstack/nova master: [placement] Add test demonstrating bug 1778743 https://review.openstack.org/578126 | |
| 02:55:33 | openstack | bug 1778743 in OpenStack Compute (nova) "When POSTing to /allocations with multiple consumers it is possible violate inventory capacity constraints" [Medium,In progress] https://launchpad.net/bugs/1778743 - Assigned to Chris Dent (cdent) | |
| 03:01:00 | openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Compute: add support to abort queued live migration https://review.openstack.org/568542 | |
| 03:08:36 | deepak_mourya | efried: Hi, what should be the good string to use here according to you https://review.openstack.org/#/c/577726/6/nova/api/openstack/placement/handlers/resource_provider.py | |
| 03:09:11 | efried | deepak_mourya: Yeah, sorry, I didn't leave a lot of detail on that. Are you familiar with the framework for defining the error codes? | |
| 03:10:23 | deepak_mourya | efried: It's ok, any example for the same in the nova before ? | |
| 03:10:32 | efried | deepak_mourya: You want to define it in nova/api/openstack/placement/errors.py, and then look at how the other consts therein are being used. | |
| 03:10:46 | efried | deepak_mourya: But you can do that in a followon patch. The -1 is just for the failing functional test. | |
| 03:16:43 | deepak_mourya | efried: ok, i didn't understand much from here nova/api/openstack/placement/errors.py | |
| 03:16:43 | deepak_mourya | ok so what exactly these things are | |
| 03:16:43 | deepak_mourya | DEFAULT = 'placement.undefined_code' | |
| 03:16:43 | deepak_mourya | INVENTORY_INUSE = 'placement.inventory.inuse' | |
| 03:16:43 | deepak_mourya | CONCURRENT_UPDATE = 'placement.concurrent_update' | |
| 03:17:02 | deepak_mourya | Do i need to create another consts for the same? | |
| 03:17:40 | efried | deepak_mourya: Yes. It's explained in this spec: http://specs.openstack.org/openstack/nova-specs/specs/rocky/approved/placement-api-error-handling.html | |