| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-23 | |||
| 14:55:40 | mnaser | nope, even if you set retry=0 it fails :( | |
| 14:55:51 | sean-k-mooney | oh ok then that is a bug | |
| 14:55:51 | mnaser | see what gibi mentioned above, it gets stuck on trying to connect and that has no retries | |
| 14:55:56 | mnaser | it loops forever on trying to connect | |
| 14:56:04 | sean-k-mooney | ah right | |
| 14:56:27 | sean-k-mooney | because its the amqp connection that is down its not an issue with sendign to a queue or exchange | |
| 14:56:32 | mnaser | i captured GMR and updated info here - https://bugs.launchpad.net/nova/+bug/1917645 | |
| 14:57:00 | sean-k-mooney | mnaser: well that proably should be converted to an oslo messaging bug at this point | |
| 14:57:22 | mnaser | sean-k-mooney: good point actually | |
| 14:57:47 | sean-k-mooney | nova might be able to work around this but eventually we will want to adress this there i suspect | |
| 15:01:11 | opendevreview | Dmitrii Shcherbakov proposed openstack/os-traits master: Add a trait for remote_managed port-capable nodes https://review.opendev.org/c/openstack/os-traits/+/818514 | |
| 15:03:42 | gibi | sean-k-mooney: sure oslo.messaging lacks a configuration possibility to stop retrying the connect, but I don't think it is purelya bug | |
| 15:03:58 | sean-k-mooney | pro tip dont put a folder with 2412 sub directories in your test path if you want test discovery to happen before the heat death of the universe | |
| 15:04:00 | gibi | sean-k-mooney: on nova side it is sure a bug if the instances stuck in some tranisitional state due to hanging | |
| 15:04:45 | sean-k-mooney | gibi: yes but i dont think im sold that we should intoduction config driven api behavior | |
| 15:04:53 | gibi | sean-k-mooney: not in nova no | |
| 15:05:00 | gibi | sean-k-mooney: we need oslo.messaging configurability | |
| 15:05:12 | sean-k-mooney | right so we cant block opertaion because we failed to send a notification | |
| 15:05:35 | sean-k-mooney | gibi: well not i think not at all nova need to proceed even if the notificaiton cant be sent | |
| 15:05:55 | sean-k-mooney | we can log an error ro report it via a healthcheck or similar | |
| 15:06:11 | gibi | sean-k-mooney: I don't agree. If notifications are used for billing then I can imagine that the operator want the instance boot to fail if the notification bus is down | |
| 15:06:28 | sean-k-mooney | gibi: its a grey area i guess | |
| 15:06:35 | gibi | sure, don't stuck, but fail cleanly | |
| 15:06:53 | sean-k-mooney | to me unless we can report a 500 to the user im not sure we should block the operation | |
| 15:07:22 | bauzas | gibi: sean-k-mooney: thoughts abotu https://bugs.launchpad.net/nova/+bug/1951623 be a nova or neutron issue ? | |
| 15:08:04 | sean-k-mooney | ..... both | |
| 15:08:05 | bauzas | nova wouldn't know whether the port is disabled | |
| 15:08:10 | gibi | bauzas: nova, as nova waits unconditionally | |
| 15:08:15 | bauzas | yeah | |
| 15:08:15 | sean-k-mooney | nova can check that | |
| 15:08:16 | gibi | bauzas: nova can query the port sate | |
| 15:08:18 | gibi | state | |
| 15:08:32 | sean-k-mooney | nova because this is not checked today | |
| 15:08:40 | sean-k-mooney | and neutron because this change depending on the backend | |
| 15:08:53 | bauzas | gibi: mmm, that means that we would call neutron to get the port state | |
| 15:09:15 | sean-k-mooney | bauzas: i think we have that in the network info cache but yes we have a check for this in some of the code already | |
| 15:09:39 | bauzas | sean-k-mooney: ok, then it's a nova bug if we already call it | |
| 15:09:50 | sean-k-mooney | we just dont in that part of the code | |
| 15:10:19 | sean-k-mooney | but part of me is wondering if we would be better off just not using those event until we actully define a contract with neutorn on when they will be sent | |
| 15:10:59 | gibi | sean-k-mooney: but thing might never happen based on past experience ^^ :) | |
| 15:11:07 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L7221-L7228 | |
| 15:11:21 | bauzas | gibi: sean-k-mooney: I just triaged the bug https://bugs.launchpad.net/nova/+bug/1951623 | |
| 15:11:28 | sean-k-mooney | so we check if its active here but we dont for the migration events | |
| 15:12:31 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/network/model.py#L488-L508 or here https://github.com/openstack/nova/blob/master/nova/network/model.py#L567-L586 | |
| 15:12:59 | sean-k-mooney | bauzas: gibi so we just need to update the model fucntion to skip disbaled ports | |
| 15:13:15 | bauzas | ack | |
| 15:13:22 | bauzas | bug confirmed anyway | |
| 15:13:37 | sean-k-mooney | the issue is that in some cases we will get event if the port is disabled | |
| 15:13:49 | sean-k-mooney | which we will now be ignorign but thats is likely ok | |
| 15:13:53 | gibi | yeah | |
| 15:20:29 | bauzas | elodilles: I hadn't time to update the meeting agenda, but add your points if you want | |
| 15:20:50 | bauzas | reminder : nova meeting in 40 mins here in #openstack-nova | |
| 15:23:38 | elodilles | bauzas: adding it now then | |
| 15:23:47 | DK4 | when creating instances they are all stuck in creating state. ive combed thrtough logs and did not find anything special. any advice on debugging that? | |
| 15:25:49 | elodilles | bauzas: done, just added a short info update, quickly | |
| 15:26:17 | elodilles | bauzas: you can edit now the page if you want | |
| 15:31:06 | sean-k-mooney | bauzas: by the way unless youhave actully repoduced the bug or root caused it you should be setting it to triaged not confirmed | |
| 15:32:12 | sean-k-mooney | we treat them more or less the same so i guess it does not matter but confrimed imples we coudl repoduces it triage does not | |
| 15:37:20 | opendevreview | Dan Smith proposed openstack/nova master: Revert project-specific APIs for servers https://review.opendev.org/c/openstack/nova/+/816206 | |
| 15:51:06 | bauzas | last reminder: nova meeting starts in 10 mins here at #openstack-nova IRC chan | |
| 15:51:40 | bauzas | sean-k-mooney: yes and no, we could put in Triaged if we don't know the direction | |
| 15:51:53 | bauzas | sean-k-mooney: we don't ask bug triagers to test all the issues | |
| 15:52:00 | bauzas | (even if Launchpad says it) | |
| 15:52:15 | bauzas | at least that's the census we have | |
| 15:52:24 | sean-k-mooney | yes i know which is why i am saying confirmed is a larger statement | |
| 15:53:40 | sean-k-mooney | triaged just means we think its valid and have some understading of why it happens, confrimed implies you ahve actully tried to repoduce it an have confirmed the behavior is as reported in the bug | |
| 15:53:48 | sean-k-mooney | or at least that is how i treat it | |
| 16:00:15 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:00:15 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:00:15 | opendevmeet | Meeting started Tue Nov 23 16:00:15 2021 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:00:15 | bauzas | #startmeeting nova | |
| 16:00:25 | gibi | o/ | |
| 16:00:29 | elodilles | o/ | |
| 16:00:32 | bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | |
| 16:00:37 | lyarwood | \o | |
| 16:00:37 | bauzas | good day, 'vyone | |
| 16:01:37 | bauzas | ok, let's start | |
| 16:02:04 | bauzas | a few folks could be off b/c of some holiday they have on Thursday :) | |
| 16:02:16 | bauzas | not French , tho | |
| 16:02:28 | bauzas | #topic Bugs (stuck/critical) | |
| 16:02:33 | bauzas | #info No Critical bug | |
| 16:02:39 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 29 new untriaged bugs (+1 since the last meeting) | |
| 16:02:50 | bauzas | I did a bit of triage and will continue on it tomorrow | |
| 16:03:17 | bauzas | thanks for the people who did this too | |
| 16:03:22 | bauzas | #help Nova bug triage help is appreciated https://wiki.openstack.org/wiki/Nova/BugTriage | |
| 16:03:35 | bauzas | #link https://storyboard.openstack.org/#!/project/openstack/placement 33 open stories (+0 since the last meeting) in Storyboard for Placement | |
| 16:03:42 | bauzas | any bug to discuss by now ? | |
| 16:04:21 | bauzas | #topic Gate status | |
| 16:04:29 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | |
| 16:04:45 | bauzas | no new gate bug AFAICS | |
| 16:05:03 | bauzas | btw. we could triage three of them that are NEW :) | |
| 16:05:13 | bauzas | (even 4) | |
| 16:05:30 | bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status | |
| 16:05:42 | bauzas | we have an issue | |
| 16:05:57 | bauzas | #info tempest-integrated-placement job had a CI issue | |
| 16:06:13 | bauzas | #link https://zuul.openstack.org/build/43f380b91bd14579806d0b9be0d702c8 the job itself | |
| 16:06:44 | bauzas | looks like an unrelated package issue | |
| 16:06:49 | gibi | it is a known issue https://zuul.openstack.org/build/43f380b91bd14579806d0b9be0d702c8/log/job-output.txt#4206 | |
| 16:07:02 | gibi | the pmlogger service did not start | |
| 16:07:09 | kashyap | What does the service even do? | |