| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-13 | |||
| 12:39:22 | ohwhyosa | and thanks a lot, I'm learning a ton | |
| 12:40:06 | sean-k-mooney | do you have osc-placement installed? and if so what version | |
| 12:41:27 | ohwhyosa | I believe I don't, I don't seem to be able to use any of its commands | |
| 12:42:34 | sean-k-mooney | ok you could install it but since we are using curl anyway this will fined the RP for you | |
| 12:42:36 | sean-k-mooney | curl -s -H "X-Auth-Token: ${OS_TOKEN}" "http://192.168.100.253/placement/resource_providers/?name=${old_hostname}" | python -mjson.tool | |
| 12:43:00 | sean-k-mooney | you need to replace 192.168.100.253/placement with your placemetn endpoint | |
| 12:45:13 | ohwhyosa | Not found (neither with the new nor the old hostname) | |
| 12:45:23 | sean-k-mooney | ya i had a typo | |
| 12:45:25 | sean-k-mooney | curl -s -H "X-Auth-Token: ${OS_TOKEN}" "http://192.168.100.253/placement/resource_providers?name=${old_hostname}" | python -mjson.tool | |
| 12:45:42 | sean-k-mooney | you do not want the / at the end of the url i think that was the error | |
| 12:46:25 | ohwhyosa | Yup! | |
| 12:46:34 | ohwhyosa | Indeed I have both the old name and the new one | |
| 12:47:22 | sean-k-mooney | cool so if you grab the uuid for the old name and append <uuid>/allocations to that url | |
| 12:47:34 | sean-k-mooney | then it should not be empty on the old host name correct? | |
| 12:47:54 | sean-k-mooney | e.g. curl -s -H "X-Auth-Token: ${OS_TOKEN}" "http://192.168.100.253/placement/resource_providers/${uuid}/allocations" | python -mjson.tool | |
| 12:48:33 | ohwhyosa | { | |
| 12:48:33 | ohwhyosa | "resource_provider_generation": 2, | |
| 12:48:33 | ohwhyosa | "allocations": {} | |
| 12:48:33 | ohwhyosa | } | |
| 12:48:46 | sean-k-mooney | huh i was expecting that to be non empty | |
| 12:48:57 | sean-k-mooney | well the issue is that that RP was not deleted | |
| 12:49:40 | sean-k-mooney | i guess the lxc container destory playbook may not have actully deleted it properly | |
| 12:49:57 | ohwhyosa | It has self link, inventories link and usages link when getting the uuid | |
| 12:50:58 | ohwhyosa | Hmmm could be, should I just delete it? | |
| 12:51:16 | ohwhyosa | And should that fix the novnc console also? | |
| 12:51:28 | sean-k-mooney | you have changed the hostname so it should not be an issue. | |
| 12:51:40 | sean-k-mooney | i think the novnc console issue is seperate | |
| 12:52:21 | sean-k-mooney | the fact the RP exsited is why you could not originally recreate the invenoties when the compute agent started up | |
| 12:52:30 | ohwhyosa | It does have a populated resource inventory though | |
| 12:52:35 | ohwhyosa | Oooooh get it! | |
| 12:52:37 | ohwhyosa | Thanks a ton | |
| 12:53:21 | sean-k-mooney | ya the invetory and RP are just left over form the last time the compute node ran update_provider_tree before you redeploy | |
| 12:54:04 | sean-k-mooney | you should clean up the RPs so that the schduler does not try to select them but the nova filters shoudl eliminate them anyway | |
| 12:54:32 | sean-k-mooney | before you try to delete them can you first check if there is a compute service with the old name still | |
| 12:54:51 | ohwhyosa | sorry, i deleted it before reading this | |
| 12:55:02 | ohwhyosa | How can I check if there is a service with the old name? | |
| 12:56:06 | sean-k-mooney | openstack compute service list --host ${old_hostname} --service nova-compute | |
| 12:56:38 | ohwhyosa | Yep, there is one! enabled and down | |
| 12:56:58 | sean-k-mooney | ya its down because it has not recived a heartbeat since you redeployed. | |
| 12:57:28 | ohwhyosa | I should disable and delete it right? | |
| 12:57:48 | ohwhyosa | So, only two issues left with nova for now hahaha the novnc proxy and the err 32 broken pipe | |
| 12:57:49 | sean-k-mooney | if you do openstack compute service delete <service uuid> it will clean up the old placmenet RP too | |
| 12:58:08 | ohwhyosa | Oh! So it does already what I did via the api! | |
| 12:58:25 | sean-k-mooney | ya | |
| 12:58:38 | sean-k-mooney | the playbook you ran jsut nuked the containers | |
| 12:58:51 | sean-k-mooney | it did not also clean up the compute resouces in nova or placmenet | |
| 12:59:25 | sean-k-mooney | normally if it reused the same host name it should have reused the same compute service but im guessing somthing went wrong | |
| 13:00:49 | ohwhyosa | Good, good, note to self then, delete all the resources before nuking containers | |
| 13:01:33 | sean-k-mooney | well i would check with the openstack-ansible people on how they expect it to be done | |
| 13:01:52 | sean-k-mooney | e.g. im sure they have a playbook or docs for reporviioning node somewhere | |
| 13:02:10 | sean-k-mooney | or they will be able to tell you what the workflow shoud be with there tooling | |
| 13:02:30 | sean-k-mooney | if they dont you can file a bug and im sure they would be happy to add some | |
| 13:03:03 | ohwhyosa | Yup! I'll ask them, to be fair the whole #openstack-* ircs are full of helpful people, thanks a couple tons | |
| 13:03:05 | sean-k-mooney | ohwhyosa: mnaser is really approachable although he is also quite busy but im sure he and the other osa folk will be able to help too | |
| 13:03:47 | mnaser | yeah i don't think we have a 'off-board compute' playbook | |
| 13:03:55 | mnaser | whichgoes and cleans up placement and what not | |
| 13:04:21 | sean-k-mooney | mnaser: do you have a playbook that will reconfigure exisitng nodes? | |
| 13:04:29 | ohwhyosa | Thanks! mnaser so you recommend deleting the service via cli? | |
| 13:05:11 | mnaser | i mean usually the idea is that we run things and they automatically converge again.. in theory | |
| 13:13:26 | ohwhyosa | mnaser: Hmmm converge as in work again right? So, even if I hadn't changed the hostnamed, should it have started working again? | |
| 13:15:24 | ohwhyosa | Also, https://pastebin.com/vnmSEKDW --> This is the broken pipe error, in case you want to take a look, and this | |
| 13:18:28 | ohwhyosa | And this https://pastebin.com/A3VrPvy9 is the noVNC issue | |
| 13:18:40 | ohwhyosa | The problem, though, is that the "token" var is an empty string | |
| 13:18:51 | mriedem | amorin: do you plan on backporting this? https://review.opendev.org/#/c/660761/ | |
| 13:19:04 | ohwhyosa | I don't really know where is it querying from, it does mention something in the docs about the token being in the cookies | |
| 13:19:14 | ohwhyosa | But it is inthe url | |
| 13:19:50 | mriedem | need some stable cores on these +2ed rocky changes https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:stable/rocky+label:Code-Review=2 | |
| 13:20:04 | lyarwood | ack, I can look shortly | |
| 13:20:31 | lyarwood | mriedem: https://review.opendev.org/#/c/663677/ would you mind looking at that in return | |
| 13:21:34 | mriedem | sure | |
| 13:22:32 | amorin | mriedem: yes | |
| 13:25:22 | amorin | mriedem: however, I think I will abandon this one: https://review.opendev.org/#/c/660982/ | |
| 13:25:37 | amorin | we dont need to force refresh, nova DB is always correct in my tests | |
| 13:25:44 | amorin | we dont need to ask neutron | |
| 13:25:48 | amorin | what do you think? | |
| 13:26:21 | mriedem | i'd say we don't need it until we do :) | |
| 13:26:38 | mriedem | the force refresh is more for when the cache gets corrupted | |
| 13:29:32 | openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.74: Support Specifying AZ to unshelve https://review.opendev.org/665136 | |
| 13:31:18 | openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.74: Support Specifying AZ to unshelve https://review.opendev.org/665136 | |
| 13:32:43 | openstackgerrit | Brin Zhang proposed openstack/nova master: Specify availability_zone to unshelve https://review.opendev.org/663851 | |
| 13:35:14 | openstackgerrit | Leehom Li proposed openstack/nova master: Make sure update resource usage when revert_resize https://review.opendev.org/665138 | |
| 13:40:55 | mnaser | ohwhyosa: better to move the convo over to #openstack-ansible to avoid causing noise here :) | |
| 13:42:02 | openstackgerrit | Merged openstack/nova-specs master: Spec to pre-filter disabled computes with placement https://review.opendev.org/657884 | |
| 13:43:42 | ohwhyosa | mnaser:Nice! Going there, I though since it was nova related here would be better! | |
| 13:44:39 | sean-k-mooney | ohwhyosa: it depends on the issue. it sounds liek this might be more related to the confgiration of the service rather then the code | |
| 13:44:58 | sean-k-mooney | ohwhyosa: so #openstack-ansible might be able to help more | |
| 13:47:07 | ohwhyosa | Great! And thanks for all the help, really, I feel like an openstack jedi already (not even a padawan really, but the feeling is there) | |
| 13:53:06 | openstackgerrit | sean mooney proposed openstack/nova-specs master: Libvirt: add vPMU spec for train https://review.opendev.org/651269 | |
| 13:53:36 | sean-k-mooney | now ^ in the correct folder and some typos fixed :) | |
| 13:58:16 | sean-k-mooney | stephenfin: can you take a look at these quickly https://review.opendev.org/#/q/topic:bug/1832652+(status:open+OR+status:merged) | |
| 13:58:25 | stephenfin | ;sure | |
| 14:00:17 | efried | nova meeting now in #openstack-meeting | |
| 14:00:47 | bhagyashris | stephenfin, sean-k-mooney: Hi, I just want your opinion about http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007084.html http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007109.html | |
| 14:01:36 | sean-k-mooney | bhagyashris i replied to both this morning | |
| 14:02:19 | sean-k-mooney | bhagyashris: http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007118.html and http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007119.html | |
| 14:02:44 | bhagyashris | sean-k-mooney; Thank you! i would also like more opinions about the same so that i can proceed | |
| 14:03:44 | sean-k-mooney | the important thing to remember is vcpu_pin_set does not and has never ment that the host was use for pinnned instances | |
| 14:04:34 | sean-k-mooney | its related but it not a direct mapping | |
| 14:07:26 | openstackgerrit | Arnaud Morin proposed openstack/nova stable/stein: Refresh instance network info on deletion https://review.opendev.org/665143 | |
| 14:08:27 | amorin | mriedem and others see ^, cherry pick for stein | |