| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-07-25 | |||
| 09:14:20 | sean-k-mooney | but we could also set it to unsafe ... that would definetly be faster | |
| 09:15:17 | bauzas | fwiw, I only see this pattern on nova-next | |
| 09:15:55 | bauzas | or, I'd rather say, this failure incidence is way higher on this job than any other one in the nova check and gate pipelines | |
| 09:16:09 | sean-k-mooney | [libvirt]/disk_cachemode=file=none,block=unsafe,network=writeback | |
| 09:17:22 | sean-k-mooney | im goign to add a patch to nova-next to configre that and lets see if it fails | |
| 09:23:42 | opendevreview | sean mooney proposed openstack/nova master: [WIP] use disk caching to hide slow cinder performance https://review.opendev.org/c/openstack/nova/+/889383 | |
| 09:25:47 | sean-k-mooney | bauzas: ^ that might work although in production you would not use unsafe but for ci it should be fine | |
| 09:37:49 | dvo-plv | Hello, nova folks. Maybe you will have a chance to review this commit, thank you https://review.opendev.org/c/openstack/nova/+/876075 | |
| 09:39:47 | sean-k-mooney | i started that review twice already so i proably should go finish it :) | |
| 09:47:27 | sean-k-mooney | dvo-plv: looks good to me | |
| 11:35:44 | kashyap | While I'm off the next 3 days, anyone willing to get this over the line? It's just failing on some timeouts: https://review.opendev.org/c/openstack/nova/+/887255 | |
| 11:36:51 | kashyap | (I see sean-k-mooney already did a recheck on the dep patch; thanks!) | |
| 11:37:41 | sean-k-mooney | it still needs review form other but i set RP+2 on it as well | |
| 11:38:39 | kashyap | Yeah, saw that, thank you! | |
| 12:53:10 | auniyal | sean-k-mooney, and others, how can I set debug=true for nova-scheduler; in devstack nova.conf for [DEFAULT] its already set there is another group [scheduler], I tried there as well but no difference | |
| 12:53:18 | auniyal | I am trying to see https://github.com/openstack/nova/blob/815683ea86492d3ed77b04cc56f3db87e2b8c47d/nova/weights.py#L136 | |
| 12:53:33 | auniyal | in scheduler logs while launching VM | |
| 12:55:19 | auniyal | is there any other conf I should look into ? | |
| 13:06:25 | sean-k-mooney | devstack runs in debug mode by default | |
| 13:07:42 | auniyal | yes, still these logs are not coming in "jouranlctl -u devstack@n-sch" | |
| 13:09:34 | sean-k-mooney | stack@upstream-devstack:~$ sudo journalctl -u devstack@n-sch | grep debug | wc | |
| 13:09:36 | sean-k-mooney | 10 172 2932 | |
| 13:11:54 | auniyal | yes debugs logs are coming in n-sch but not these logs https://github.com/openstack/nova/blob/815683ea86492d3ed77b04cc56f3db87e2b8c47d/nova/weights.py#L136 | |
| 13:12:16 | auniyal | so I think they should come whie VM create | |
| 13:12:32 | auniyal | right ? | |
| 13:12:50 | sean-k-mooney | im seeign the log in the ci | |
| 13:13:17 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/fd53404ef23341828ae15f8ccf596e6c/log/controller/logs/screen-n-sch.txt#1018 | |
| 13:15:04 | auniyal | yes, here they are coming, I have single-node devstack env localy | |
| 13:15:57 | auniyal | it should be right ? | |
| 13:23:51 | sean-k-mooney | /etc/nova/nova.conf as rendered by devstack has debug=true set in the default section | |
| 13:24:08 | sean-k-mooney | that is what the scheduler uses | |
| 13:30:11 | auniyal | ack, thanks | |
| 13:33:25 | sean-k-mooney | the backport does not seam to be working properly | |
| 13:34:41 | sean-k-mooney | wait why are you proposing https://review.opendev.org/c/openstack/nova/+/889311 directly to yoga | |
| 13:34:55 | sean-k-mooney | oh your are not | |
| 13:35:03 | sean-k-mooney | this is incorrect | |
| 14:01:43 | auniyal | ack | |
| 14:10:58 | opendevreview | alecorps proposed openstack/nova master: Workaround for issues with ephemeral disk named disk.local during resize https://review.opendev.org/c/openstack/nova/+/888220 | |
| 14:14:56 | opendevreview | Amit Uniyal proposed openstack/nova master: Added context manager for instance lock https://review.opendev.org/c/openstack/nova/+/873648 | |
| 14:14:57 | opendevreview | Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446 | |
| 14:37:34 | bauzas | elodilles: gibi: I'll need to stop leading the nova meeting after 20-25 mins | |
| 14:37:53 | bauzas | Uggla and me are going to visit artom who's in the surroundings | |
| 14:39:47 | opendevreview | Maxim Monin proposed openstack/nova master: Server Rescue leads to Server ERROR state if original image is deleted https://review.opendev.org/c/openstack/nova/+/872385 | |
| 14:42:36 | elodilles | bauzas: hmmm... i might be late like 20-25 mins, actually :S | |
| 14:43:06 | elodilles | (though i hope not...) | |
| 14:47:01 | auniyal | hey melwitt, I heard PS[number] from you few times, but not sure what it is, like PS1 or PS10 | |
| 14:47:30 | bauzas | auniyal: PS = patchset | |
| 14:47:40 | bauzas | the revision number in gerrit | |
| 14:47:59 | auniyal | so in one change patchset 10 | |
| 14:48:04 | auniyal | ack, thanks | |
| 14:56:59 | gibi | bauzas: I probably need to skip today's meeting (or I will be on and off during it) | |
| 14:57:12 | bauzas | np | |
| 14:57:29 | bauzas | honestly, given this and that, I don't want to skip but I'll say that we'll just check a few things | |
| 14:59:02 | gibi | ack | |
| 15:52:07 | auniyal | gibi, bauzas, dansmith, sean-k-mooney, melwitt can you please review these stable branch patches - https://etherpad.opendev.org/p/release-liaison-PatchesToReview | |
| 15:52:40 | auniyal | most of them are clean cherry-pick and already have 1 +2 and good to merge | |
| 15:54:15 | sean-k-mooney | the gates on stabel shoudl be unblocked | |
| 15:54:28 | sean-k-mooney | the nova-lvm and ceph issues shoudl be fixed on all branches | |
| 15:55:58 | bauzas | auniyal: ack, my main prio before leaving for 3 weeks is about features reviews but I'll try | |
| 15:56:16 | elodilles | yepp, they are unblocked, thanks sean-k-mooney for the patches \o/ | |
| 16:00:23 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:00:23 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:00:23 | opendevmeet | Meeting started Tue Jul 25 16:00:23 2023 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:00:23 | bauzas | #startmeeting nova | |
| 16:00:39 | bauzas | hey, let's try to have a 10-min meeting if we can | |
| 16:00:43 | dansmith | o/ | |
| 16:00:51 | bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | |
| 16:00:52 | gmann | o/ | |
| 16:01:01 | auniyal | o/ | |
| 16:01:22 | elodilles | o/ | |
| 16:01:24 | bauzas | ok, starting | |
| 16:01:34 | bauzas | if someone wants to continue discussing, I can pass the chair | |
| 16:01:40 | bauzas | #topic Bugs (stuck/critical) | |
| 16:01:44 | bauzas | #info No Critical bug | |
| 16:01:48 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 40 new untriaged bugs (+0 since the last meeting) | |
| 16:01:58 | bauzas | Uggla made a good effort on triaging this week, thanks | |
| 16:02:11 | bauzas | he shared to me https://etherpad.opendev.org/p/nova-bug-triage-20230725 | |
| 16:02:27 | bauzas | #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster | |
| 16:02:39 | bauzas | elodilles: can you be the baton owner for this week ? | |
| 16:02:44 | elodilles | yepp | |
| 16:02:49 | bauzas | cool thanks ! | |
| 16:02:53 | bauzas | #info bug baton is being passed to elodilles | |
| 16:02:57 | bauzas | any bug to discuss ? | |
| 16:02:58 | elodilles | i'll be off on Friday, but otherwise fine! | |
| 16:03:09 | bauzas | elodilles: I'll be off next week so :) | |
| 16:03:18 | elodilles | touche ;) | |
| 16:03:18 | bauzas | #topic Gate status | |
| 16:03:22 | bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | |
| 16:03:27 | bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status | |
| 16:03:31 | bauzas | #info nova-emulation back to work | |
| 16:03:35 | bauzas | #link https://zuul.openstack.org/build/2eb47b5dcb944b7eb03224e6ee599a57 | |
| 16:03:41 | dansmith | gmann and I have been working on a bunch of patches to address job timeout issues | |
| 16:03:46 | bauzas | now we have a failure on tempest-integrated-placement | |
| 16:03:54 | dansmith | still lots in flight, but we're hopeful that it will improve those issues at least | |
| 16:03:56 | bauzas | but we'll see next week | |
| 16:04:01 | dansmith | but there are still lots of other spurious fails | |
| 16:04:10 | bauzas | dansmith: I was about to mention the other pipelines but the periodic-weekly | |
| 16:04:21 | bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | |
| 16:04:26 | dansmith | really need all hands on deck to improve this situation | |
| 16:04:32 | gmann | yeah, timeout now a days is due to multiple reasons and we are trying to improve a few of them | |