| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-09-24 | |||
| 14:20:21 | gibi | ahh I see what is needed | |
| 14:20:35 | priteau | I've updated the wiki page | |
| 14:21:59 | gibi | hm, do we need both -a and -l ? | |
| 14:23:07 | gibi | is -l a shortcut for --last ? | |
| 14:23:11 | gibi | (I'm on debian :D)_ | |
| 14:23:16 | sean-k-mooney | we proably should jsut remove it from the template | |
| 14:23:44 | sean-k-mooney | i think for deb packages -l is for list | |
| 14:23:47 | sean-k-mooney | i can check | |
| 14:24:00 | priteau | Just -qa | |
| 14:24:07 | priteau | -ql is for listing files inside a package | |
| 14:24:15 | priteau | -l, --list | |
| 14:24:15 | priteau | List files in package. | |
| 14:24:29 | gibi | ohh OK then -a it is | |
| 14:24:49 | gibi | fixed the LP | |
| 14:24:56 | gibi | priteau: thanks for reporting it | |
| 14:24:58 | sean-k-mooney | yes -qa | |
| 14:25:16 | sean-k-mooney | but really we dont care about the package version most of the time we just care about the openstack version | |
| 14:25:59 | priteau | This is an easy way to find the version when using binary packages | |
| 14:26:01 | sean-k-mooney | we care for libvirt and qemu sometimes but we porably could make it more generic | |
| 14:26:16 | sean-k-mooney | priteau: right but upstream that normally not useful | |
| 14:26:29 | sean-k-mooney | since we cant easially map it to the source code | |
| 14:27:02 | sean-k-mooney | even downstream the pacakge version is not very useful since that mapping is hard to do | |
| 14:27:32 | sean-k-mooney | we often have to pull the srouce rpm and check if a patch is in it which is a pain | |
| 14:29:01 | sean-k-mooney | priteau: by the way the template also tells you to run udo sosreport -o openstack_nova --batch | |
| 14:29:12 | sean-k-mooney | which i dont think i have ever seen peopl actully do | |
| 14:29:20 | priteau | Who follows instructions? :) | |
| 14:29:34 | priteau | Would you like people to use `pip3 list | grep nova` as an alternative? | |
| 14:30:08 | sean-k-mooney | not nessisarly but its helpful if they clearly state that they used train or the serise name | |
| 14:30:16 | opendevreview | Lucian Petrut proposed openstack/nova master: api: enable oslo.reports when using uwsgi https://review.opendev.org/c/openstack/nova/+/810922 | |
| 14:30:27 | sean-k-mooney | knowing the disto and or package version is nice too | |
| 14:32:05 | priteau | I imagine knowing the release tag can be quite useful | |
| 14:32:15 | opendevreview | Lucian Petrut proposed openstack/nova master: api: enable oslo.reports when using uwsgi https://review.opendev.org/c/openstack/nova/+/810922 | |
| 14:32:44 | gibi | just yesterday I troubleshooted a deployment with nova_compute version 22.2.3 :D | |
| 14:33:08 | lpetrut | hi, I'm hitting some nova api deadlocks and noticed that oslo.reports isn't enabled when using uwsgi so I've submitted a small commit: https://review.opendev.org/c/openstack/nova/+/810922 | |
| 14:33:16 | gibi | (note that we only released 22.2.2 upstream) | |
| 14:33:48 | sean-k-mooney | gibi: ya i was going to say was this in the gate :) | |
| 14:34:19 | sean-k-mooney | cause otherwise they are going to have fun when we do the next stable release | |
| 14:34:50 | gibi | sean-k-mooney: it was downstream. I think what they did is they took what was unreleased from stable/victoria and created 22.2.3 out of it downstream | |
| 14:35:10 | sean-k-mooney | i see | |
| 14:35:26 | gibi | which is problematic as you said | |
| 14:35:32 | gibi | lpetrut: seems useful | |
| 14:35:59 | gibi | lpetrut: thanks | |
| 14:36:33 | sean-k-mooney | huh | |
| 14:37:19 | sean-k-mooney | maybe that is why the GMR were not working instead of what we tought with the signal being intercpted by uwsgi/mod_wsgi | |
| 14:37:42 | gibi | sean-k-mooney: or we need both :) | |
| 14:37:52 | sean-k-mooney | lpetrut: i assume you tested this and it logs the GMR to the log properly on kill -usr2 | |
| 14:38:24 | sean-k-mooney | gibi: ya i was wonderign if that would only work if you set the signal to the python interpreter instnace | |
| 14:39:17 | gibi | unfortunately this is pretty complicated to test upstream. | |
| 14:39:22 | lpetrut | sean-k-mooney: the signal still gets intercepted but I'm using a file listener | |
| 14:39:47 | gibi | I mean automatically testing it | |
| 14:39:48 | sean-k-mooney | lpetrut: intercepted by uwsgi and not passed to nova right | |
| 14:40:04 | sean-k-mooney | lpetrut: oh are you poking a file to trigger it | |
| 14:40:14 | sean-k-mooney | instead of sig_usr2 or soemthing | |
| 14:40:38 | lpetrut | yep, I'm setting something like oslo_reports.file_event_handler = /opt/stack/logs/trigger | |
| 14:40:48 | sean-k-mooney | gibi: we could proably add a func test but we would have to expand the test scope | |
| 14:40:57 | sean-k-mooney | lpetrut: ok i dont think we technially support that in nova | |
| 14:41:07 | sean-k-mooney | but it certenly works around the issue | |
| 14:41:27 | lpetrut | it already works with most nova services, they key is to pass the config when calling the gmr hook | |
| 14:41:30 | sean-k-mooney | so this is really a mini feature rather then a bug | |
| 14:41:34 | gibi | sean-k-mooney: do we run nova-api in uwsgi in func test? | |
| 14:41:56 | sean-k-mooney | gibi: no but we coudl do somehting like neutorn fullstack tests | |
| 14:42:03 | gibi | sean-k-mooney: ack, that is a possibility | |
| 14:42:04 | sean-k-mooney | it would be a different type of test | |
| 14:42:20 | gibi | sean-k-mooney: or add this to nova-next post test hook | |
| 14:42:33 | sean-k-mooney | gibi: ya that too | |
| 14:43:00 | gibi | bauzas, sean-k-mooney: btw I confirm that neutron also leaks inventories if the bw or pps config is removed | |
| 14:43:16 | sean-k-mooney | lpetrut: did you want to backport udo sosreport -o openstack_nova --batch | |
| 14:43:24 | sean-k-mooney | * https://review.opendev.org/c/openstack/nova/+/810922/2/nova/api/openstack/wsgi_app.py | |
| 14:43:52 | sean-k-mooney | to me this is really a specless blueprint | |
| 14:43:58 | sean-k-mooney | so not something we woudl backport | |
| 14:44:18 | sean-k-mooney | i think its a nice change to merge so no real objection to the patch | |
| 14:45:02 | sean-k-mooney | just not sure this is a bug and a spec is way to heavy weight so not sure how to track this | |
| 14:45:23 | sean-k-mooney | to me its really just a trivial fix but it proably should have a release note | |
| 14:46:45 | lpetrut | yeah, it's hard to label it as a bug in order to allow backports but that's ok. a release note makes sense, I can also mention the fact that uwsgi may intercept SIGUSR2, in which case a file trigger may be configured | |
| 14:47:35 | sean-k-mooney | lpetrut: ya if you add a release note and maybe add a doc for the intercept i would be +1 on it | |
| 14:48:07 | lpetrut | awesome, thanks. is there a specific doc that you have in mind? | |
| 14:48:28 | sean-k-mooney | we have a doc for GMR i think in the contibutor section | |
| 14:49:17 | sean-k-mooney | https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/doc/source/reference/gmr.rst | |
| 14:49:20 | sean-k-mooney | ah its in refernce | |
| 14:49:44 | sean-k-mooney | can you update https://github.com/openstack/nova/blob/master/doc/source/reference/gmr.rst#generating-a-gmr | |
| 14:50:07 | sean-k-mooney | with the exmaple of the file trigger | |
| 14:50:14 | lpetrut | definitely, thanks for the link | |
| 14:51:00 | lpetrut | the "TextGuruMeditation.setup_autorun(version)" hook sample should also be updated. if we don't pass the config, gmr will not be aware of the [oslo_reports] config opts | |
| 14:56:32 | sean-k-mooney | ya i dont think we have really updated it since it was added | |
| 14:57:26 | sean-k-mooney | i would suggest updating the existig singal based exmaple to use nova-compute and then adding the file example for nova-api and makeign any other changes that you think are needed | |
| 14:59:54 | lpetrut | sounds good | |
| 15:15:07 | bauzas | gibi: ack, so we need to discuss this during the PTG | |
| 15:15:52 | gibi | yepp | |
| 15:15:57 | gibi | added notes to the bad | |
| 15:16:00 | gibi | pad | |
| 15:16:29 | lpetrut | one minor nit: gmr.setup_autorun takes a "service_name" parameter which is used when constructing the report filename. when missing, it's trying to retrieve it from the stack trace but it seems to always end up with "thread.py", so the reports are named something like "thread.py_gurumeditation_20210924141722". since none of the other service pass this parameter, I'm thinking about doing the same for nova-uwsgi for consistency reasons. | |
| 15:19:54 | sean-k-mooney | lpetrut: or you could fix them all | |
| 15:20:05 | sean-k-mooney | lpetrut: you should be able to use the service binary name | |
| 15:20:29 | sean-k-mooney | so service_obj.binary | |
| 15:20:33 | lpetrut | that works as well | |
| 15:21:16 | sean-k-mooney | that way if you have multiple service on the same host using the same file tirrger they wont overright | |
| 15:21:31 | sean-k-mooney | although the timestamp is unlikely to collide in anycase | |
| 15:24:03 | lpetrut | right, it's also more user friendly since it's easier to tell which is the originating service | |
| 15:25:24 | sean-k-mooney | well when not using the file backend it dumps to the service log so that not been an issue before but for dumping the GMR to a file its something we shoudl definetly address | |
| 15:41:31 | gibi | sean-k-mooney: btw, I tried your echo 0 > numa_node trick and it works like a charm. I can now confirm that live migration with SRIOV + NUMA works and the numa topology is properly recalculated | |