| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-01 | |||
| 20:56:31 | temka | "Allocation for resource provider 'e7fbbec5-7f95-4f4e-895c-a14b691bca6e' that does not exist" | |
| 20:56:40 | temka | Looks like I'm missing a placement-y fixture or something | |
| 20:57:34 | mnaser | does nova cache neutron port info somewhere? | |
| 20:57:50 | mriedem | mnaser: instance_info_caches table | |
| 20:57:51 | mnaser | i had a misconfigured n-ovs-agent which lead to resizes failing with binding_failed on the port | |
| 20:58:00 | mnaser | i've reset the info, but its still grabbing the cached value | |
| 20:58:12 | mnaser | ok, i'll dig there, thanks mriedem | |
| 20:58:47 | mriedem | mnaser: see https://review.openstack.org/#/c/591607/ | |
| 20:59:20 | mriedem | the instance_info_caches.network_info is a json blob string so if you're looking to mess with that manually.... | |
| 20:59:38 | mnaser | mriedem: neat. i assume its not a real cache as in i cant really just drop that row | |
| 20:59:56 | mriedem | ummm | |
| 21:01:35 | mriedem | i don't think that will work | |
| 21:02:04 | mriedem | because anything that tries to refresh the cache from neutron will just rely on the existing cache, which will now be empty... | |
| 21:02:12 | mriedem | which is what https://review.openstack.org/#/c/591607/ was trying to address | |
| 21:03:39 | mriedem | wholesale deleting of the cache would also make nova forget which ports nova created and which were pre-created, so on server delete nova will also delete any pre-existing ports | |
| 21:03:46 | mriedem | which is probably not something your users want | |
| 21:04:50 | mriedem | related change here https://review.openstack.org/#/c/603844/ | |
| 21:05:19 | mriedem | the issue is kind of this code right here https://github.com/openstack/nova/blob/c57d532a306303412d6253f424bee7d665865279/nova/network/neutronv2/api.py#L3055 | |
| 21:05:38 | mriedem | if you wipe out that record in the db, that will result in port_ids being empty | |
| 21:05:46 | mriedem | so https://github.com/openstack/nova/blob/c57d532a306303412d6253f424bee7d665865279/nova/network/neutronv2/api.py#L3061 won't do anything | |
| 21:06:20 | mriedem | it's kind of dumb - if port_ids was empty but current_neutron_port_map is not, we should probably realize "oh we're fully rebuilding the cache b/c the cache was wiped out" | |
| 21:07:08 | mnaser | mriedem: yeah so cache is probably not really cache i guess | |
| 21:07:25 | mnaser | it is tracking state | |
| 21:07:42 | mriedem | well it's a cache in the sense that you don't have to go to neutron every time you want to find out what ports are attached to a server, | |
| 21:07:48 | mriedem | similar to the block_device_mappings table is a "cache" | |
| 21:07:52 | mriedem | of volumes attached to the server | |
| 21:08:02 | mnaser | yeah | |
| 21:08:27 | mriedem | but yeah it's got some state information in there too about the ports nova knows it created | |
| 21:08:45 | mnaser | (ps i totally thought this whole detach => delete port thing is silly but thats another topic) | |
| 21:09:00 | mnaser | this whole thing would be so much easier if i just detached the port and attached it again | |
| 21:09:04 | mnaser | but because it was done on boot, detaching the port = deleting it | |
| 21:09:47 | mriedem | oh there was a spec for that... | |
| 21:09:53 | mriedem | i.e. delete_on_termination for ports | |
| 21:09:58 | mriedem | like volumes | |
| 21:11:09 | mriedem | anyway like i said if you delete the cache via the db, and port_ids is empty but current_neutron_port_map is not https://github.com/openstack/nova/blob/c57d532a306303412d6253f424bee7d665865279/nova/network/neutronv2/api.py#L3061 you could probably just force a cache refresh | |
| 21:13:52 | cfriesen | can someone take a look at http://logs.openstack.org/63/631363/15/check/openstack-tox-py27/6b108d3/testr_results.html.gz and suggest why test_executable_exists_in_path is failing? It works perfectly on my local box. | |
| 21:16:54 | mriedem | mnaser: like this ^ | |
| 21:16:54 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Force refresh nw info cache if cache is empty but ports attached https://review.openstack.org/640516 | |
| 21:27:15 | mriedem | dansmith: here is another fun side effect of the mox->mock test removal - besides making backports a nightmare, it also makes git blame a real chore | |
| 21:27:18 | mriedem | that is all | |
| 21:27:36 | dansmith | yeah | |
| 21:27:48 | mriedem | plus most parts of nova you can't git blame in github anymore b/c it times out | |
| 21:28:25 | dansmith | ...really? | |
| 21:28:33 | dansmith | with git or github? | |
| 21:28:37 | mriedem | github | |
| 21:28:40 | mriedem | not cli | |
| 21:28:40 | mnaser | github | |
| 21:28:40 | mnaser | lol yeah | |
| 21:28:40 | dansmith | heh nice | |
| 21:28:41 | mnaser | mriedem: i noticed if you refresh a few times | |
| 21:28:44 | mnaser | it eventually goes through | |
| 21:28:49 | mnaser | i assume that it caches portions of the git blame page | |
| 21:28:50 | mriedem | git blame nova.compute.manager | |
| 21:28:54 | mriedem | or nova.virt.libvirt.driver | |
| 21:29:03 | mnaser | and then eventually caches enough so the request doesnt timeout | |
| 21:29:38 | openstackgerrit | Merged openstack/nova master: Add "links" in the response of "nova show" for a down-cell instance https://review.openstack.org/640302 | |
| 21:31:04 | mriedem | well i see this has come up before https://review.openstack.org/#/q/I3b9f60a2c4aea904f39d6914c8f35c4d682a6b3a | |
| 21:32:16 | mriedem | followed by https://review.openstack.org/#/q/I53d5284907d44ae8b5546993f8fd461b385c39e6 | |
| 21:32:20 | mriedem | we can't seem to make up our mind | |
| 21:40:03 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Force refresh nw info cache if cache is empty but ports attached https://review.openstack.org/640516 | |
| 21:41:24 | mriedem | melwitt: well i'm kind of stuck on how to proceed with your counting quotas from placement stuff | |
| 21:44:44 | openstackgerrit | Tim Rozet proposed openstack/nova master: Fixes race condition with privsep utime https://review.openstack.org/625741 | |
| 21:49:53 | mriedem | stephenfin: in case you didn't notice it looks like mikal would like you to remove the -2 from https://review.openstack.org/#/c/554438/ | |
| 21:56:48 | openstackgerrit | Merged openstack/nova master: conf: Deprecate 'disable_libvirt_livesnapshot' option https://review.openstack.org/626932 | |
| 21:57:12 | openstackgerrit | Merged openstack/nova master: Stop using "nova" in API samples when creating a server https://review.openstack.org/639874 | |
| 21:57:29 | openstackgerrit | Merged openstack/nova master: Make Claim._claim_test handle SchedulerLimits object https://review.openstack.org/636410 | |
| 22:16:50 | mriedem | is InstanceNUMACell.id a db primary key or something from the actual host...? | |
| 22:17:49 | mriedem | i guess the latter | |
| 22:32:15 | melwitt | mriedem: reading through your comments | |
| 22:37:22 | melwitt | mriedem: why stuck? do you and dansmith think the quota counting should try to parse user_id out of request specs? I assumed not | |
| 22:37:40 | mriedem | you mean the InstanceMapping._from_db_object? | |
| 22:37:52 | mriedem | stuck on whether or not InstanceMapping.user_id should be nullable, yes | |
| 22:39:00 | melwitt | oh | |
| 22:39:25 | melwitt | so the desire is to get it from the request spec json blob in _from_db_object | |
| 22:40:19 | mriedem | well we have 3 places to look | |
| 22:40:27 | mriedem | request spec, build request, instance in the cell | |
| 22:40:39 | mriedem | if the cell is down, the last one is going to blow up | |
| 22:40:40 | melwitt | build request doesn't have it, AFAICT | |
| 22:40:46 | mriedem | BuildRequest.instance.user_id | |
| 22:40:52 | mriedem | BuildRequest.instance is a json blob | |
| 22:40:58 | melwitt | oh, ok | |
| 22:41:11 | melwitt | I didn't know that was persisted in the db | |
| 22:41:39 | mriedem | yup | |
| 22:42:23 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L934 | |
| 22:44:10 | mriedem | elbragstad: i waxed policy rule names just for you https://review.openstack.org/#/c/621476/33/nova/policies/server_topology.py@20 | |
| 22:46:14 | cfriesen | any chance either one of you could take a quick look at https://review.openstack.org/#/c/620706/ ? It's on a runway and nobody has looked at it since Matt and Alex on the 27th. | |
| 22:46:35 | mriedem | cfriesen: i'll be honest i'm avoiding that one | |
| 22:46:39 | cfriesen | heh | |
| 22:46:50 | cfriesen | good to know | |
| 22:46:59 | mriedem | it's just low priority for me | |
| 22:47:42 | mriedem | can the hardware stuff be split out of that? | |
| 22:47:52 | mriedem | you windriver folk seem to enjoy the larger single patches... | |
| 22:49:11 | mriedem | that would drop 1/3 of the size (and context to load) for the api part of it | |
| 22:49:36 | cfriesen | yeah, I can break out the hardware bit. you're thinking do that part first? | |
| 22:49:50 | cfriesen | or second | |
| 22:49:50 | mriedem | it's a dependency for the api stuff it looks like | |
| 22:49:56 | mriedem | idk honestly | |
| 22:50:15 | cfriesen | I think it could go either way, a bunch of the api stuff is just calling existing checks | |
| 22:50:27 | mriedem | the new hardware stuff is "Added the following validations to _validate_flavor_image(): cpu_policy and cpu_thread_policy values, serial ports, realtime mask, cpu topology." right? | |
| 22:51:07 | cfriesen | no, much less than that | |