| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-02-06 | |||
| 15:23:54 | stephenfin | In hindsight, it should have been a separate patch :( | |
| 15:24:09 | ralonsoh | stephenfin, dansmith those ones with exception message | |
| 15:24:11 | ralonsoh | "<class 'nova.exception.PortNotFound'>" | |
| 15:24:28 | dansmith | ralonsoh: okay so some specifically for that api, but also some general ones it looks like | |
| 15:24:54 | stephenfin | dansmith: I assume you're suggesting that the general ones shouldn't be using the proxy APIs? | |
| 15:24:55 | spatel | sean-k-mooney: morning | |
| 15:25:09 | stephenfin | i.e. test_server_basic_ops | |
| 15:25:27 | sean-k-mooney | stephenfin: ya i think the secnairo test shoudl be updated | |
| 15:25:40 | spatel | Could you check this CPU policy for vm - http://paste.openstack.org/show/789225/ | |
| 15:25:46 | dansmith | stephenfin: well, maybe, but it's also maybe interesting that if the alternative is very onerous ... that tells us something | |
| 15:26:18 | sean-k-mooney | the other way is not really that hard. its just more api requests | |
| 15:26:23 | stephenfin | dansmith: that everyone is probably ignoring us and using these deprecated APIs anyway? :) | |
| 15:26:31 | dansmith | stephenfin: like, if the alternative is 20 calls to neutron and a bunch of local mapping, then that may be an indication that we kinda messed up | |
| 15:26:38 | dansmith | stephenfin: indeed | |
| 15:27:20 | stephenfin | less users = less bugs = more time for stephenfin to eat popcorn | |
| 15:29:15 | sean-k-mooney | the old way is just neutron port list device-id=<vm uuid> then you loop over that calling /v2.0/floatingips?port_id=<port uuid>) | |
| 15:30:25 | sean-k-mooney | so you get the port detalis form the orginal call to list all the ports on a vm and then map it to the fip from the second call | |
| 15:30:47 | dansmith | yeah, this is exactly what I'm talking about :) | |
| 15:30:55 | sean-k-mooney | if you know the fip you jsut wrok backward first do a fip show get the port and then do a port show | |
| 15:31:23 | sean-k-mooney | port_details on the fip is a proxy api in neutorn to get info form a different resouce | |
| 15:31:38 | sean-k-mooney | the added it as an optional exteion to make it 1 call | |
| 15:32:02 | sean-k-mooney | but each ml2 driver has to advertise support and ovn did not | |
| 15:32:18 | kashyap | stephenfin: Hehe, figured as much. (And I like the expression) | |
| 15:43:46 | Sundar | sean-k-mooney: Re. your message "that was the latest set that were up at that point", it got updated on Jan 10 to remove the specific exception that you posted: https://review.opendev.org/#/c/698846/2/cyborg/common/nova_client.py. Please rebase if your env is older. Sorry for the trouble. | |
| 15:46:16 | openstackgerrit | Sylvain Bauza proposed openstack/nova-specs master: Proposes NUMA topology with RPs https://review.opendev.org/552924 | |
| 15:46:26 | sean-k-mooney | Sundar: i used the local.conf i posted | |
| 15:46:52 | sean-k-mooney | which was pulling in form a new branch | |
| 15:47:02 | sean-k-mooney | ill double check | |
| 15:47:36 | openstackgerrit | Merged openstack/nova master: libvirt: Rename _is_storage_shared_with to _is_path_shared_with https://review.opendev.org/693337 | |
| 15:48:29 | sean-k-mooney | strang it seam to have checkout out master... | |
| 15:49:00 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Merge unnecessary 'NovaProxyRequestHandlerBase' separation https://review.opendev.org/705655 | |
| 15:49:08 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove 'run_once' helper https://review.opendev.org/705656 | |
| 15:49:14 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tox: Integrate mypy https://review.opendev.org/676208 | |
| 15:49:24 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add type annotations to 'nova.pci' https://review.opendev.org/676209 | |
| 15:49:31 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add nova.cmd, nova.conf, nova.console https://review.opendev.org/705657 | |
| 15:49:43 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: mypy: Add type annotations to top-level modules https://review.opendev.org/705658 | |
| 15:50:04 | sean-k-mooney | Sundar: im using enable_plugin cyborg https://review.opendev.org/openstack/cyborg refs/changes/61/703261/1 | |
| 15:50:17 | sean-k-mooney | that should have checkout the specific version | |
| 15:52:43 | stephenfin | bauzas: Care to do me the honour? https://review.opendev.org/#/c/703796/ | |
| 15:52:51 | bauzas | sure | |
| 15:53:16 | bauzas | sean-k-mooney: efried: others, I'm glad to say I'm eventually done with https://review.opendev.org/552924 | |
| 15:53:18 | sean-k-mooney | Sundar: it looks like there is either a devstack bug or the cybog plugin is nit support the gitref | |
| 15:53:25 | sean-k-mooney | Sundar: https://github.com/openstack/devstack/blob/ae73b23373f8673722fac2fbd349fa2075fda8d6/doc/source/plugins.rst#plugin-interface | |
| 15:53:31 | efried | bauzas: that's what YOU think :P | |
| 15:53:53 | bauzas | honestly, thanks you all | |
| 15:54:16 | bauzas | but I'm dead with this spec :p | |
| 15:54:26 | bauzas | no battery left. | |
| 15:55:39 | Sundar | sean-k-mooney: I think it is the latter: https://opendev.org/openstack/cyborg/src/branch/master/devstack/lib/cyborg#L85 | |
| 15:55:39 | sean-k-mooney | bauzas: https://www.deathwishcoffee.com/ i have a friend that quite likes it | |
| 15:56:03 | bauzas | sean-k-mooney: I've been told that an english tea is *always* black with milk | |
| 15:56:26 | bauzas | can't tell what an english coffee is then | |
| 15:56:44 | sean-k-mooney | Sundar: ok i can just define CYBORG_BRANCH in my local.conf | |
| 15:57:04 | sean-k-mooney | but you actully dont need to and should not clone the git repo in the plugin | |
| 15:57:12 | sean-k-mooney | that is done for you by devstack | |
| 15:58:43 | dansmith | sean-k-mooney: devstack or devstack-gate? | |
| 15:59:23 | sean-k-mooney | bauzas: are you confusing me with an english man becasue as a irish eu citizen i coudl be offended by that. also irish <anything> usally meens add good wiskey | |
| 15:59:47 | sean-k-mooney | *irish <beverage or food item> | |
| 16:00:00 | sean-k-mooney | dansmith: devstack | |
| 16:00:15 | dansmith | sean-k-mooney: the other projects have git_clone calls in their modules | |
| 16:00:55 | sean-k-mooney | so in the gate they are disabled via a diffenert parmater | |
| 16:01:24 | mnaser | ok, i'm absolutely hitting a wall. nova list --all-tenants returns only 11 vms, but i have 1297 vm_state='ACTIVE' instances. instance_mappings are correct in nova_api | |
| 16:01:30 | sean-k-mooney | but the plugin interface specific was designed for devstack to do the clone of the pugin repo | |
| 16:01:34 | mnaser | i double checked the db creds inside nova_api.cell_mappings | |
| 16:01:49 | sean-k-mooney | anyway its not a big deal | |
| 16:02:08 | mnaser | the crazier thing is running something like placement heal allocations only returns 11 instances only, which that is doing objects.InstaceList.... | |
| 16:02:51 | mnaser | so its note even an API level problem, it's in the db layer (i assume objects.* API inside nova-manage commands talks directly to db, not via conductor?) | |
| 16:03:53 | dansmith | mnaser: nova-manage does not talk to conductor | |
| 16:04:08 | mnaser | ok, so that's what i thought as well | |
| 16:04:30 | sean-k-mooney | nova manage will try and get the db infor form your nova.conf | |
| 16:04:45 | sean-k-mooney | is it perhaps geting the wrong db info | |
| 16:04:46 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Configure max-line-length to 88 https://review.opendev.org/706311 | |
| 16:05:26 | mnaser | i used the mysql client to use the same credentials in nova.conf and i see all the instances | |
| 16:05:48 | dansmith | mnaser: one non-cell0 cell? | |
| 16:05:53 | stephenfin | bauzas: thanks :) | |
| 16:05:58 | mnaser | yes, only one non-cell0 cell | |
| 16:06:11 | dansmith | mnaser: and the 11 instances you do see, are they from that cell or in cell0? | |
| 16:06:20 | bauzas | sean-k-mooney: I've been told that by an irish man in Grenoblre | |
| 16:06:29 | mnaser | i ddint check all of them but one of them is from that cell dansmith (aka inside `nova`) | |
| 16:07:00 | dansmith | mnaser: is it complete? meaning, it's not returned as a shell instance with no real state information? | |
| 16:07:09 | mnaser | oh good call | |
| 16:07:27 | mnaser | nova show brings even the user_data | |
| 16:07:36 | mnaser | so it sounds like its pulling it in from nova and not nova_api | |
| 16:07:42 | dansmith | that's not the same | |
| 16:07:53 | dansmith | when you see it in nova list, does it have vm_state? | |
| 16:07:57 | mnaser | yes | |
| 16:08:02 | dansmith | of active/ | |
| 16:08:05 | mnaser | yep | |
| 16:08:24 | mnaser | and those instances were all created post upgrade so its almost like the db just like.. only started reading specific records | |
| 16:08:37 | mnaser | now to note this is a s=>t upgrade, but those have been pretty smooth : | |
| 16:09:12 | dansmith | so I think you can rule out cells, cell_mappings, instance_mapping related issues | |
| 16:09:44 | mnaser | well to me the weird thing is how objects.InstanceList which is called inside heal placement allocation.. only sees those 11 instances | |
| 16:10:03 | mnaser | yet hypervisors are happily complaining like "66 vms on host but 65 in db" and things like that still happen | |
| 16:10:10 | dansmith | yep, another reason I think you can rule out the cell infrastructure | |
| 16:10:33 | mnaser | and if i hit a "Show" on a specific instance, i can retrieve it (just found out) | |
| 16:10:40 | mnaser | even if i dont see it listed | |
| 16:11:31 | dansmith | show finds instances in a pretty different way, | |
| 16:11:31 | mnaser | oh wth | |
| 16:11:32 | dansmith | but it also means that instance_mappings is correct | |
| 16:11:37 | mnaser | it looks like a db migration has failed | |
| 16:11:46 | mnaser | ..but it somehow KEPT going with the migrations? | |