| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-07-24 | |||
| 21:09:30 | dansmith | gmann: do you know if anyone is looking at this? https://54345f3748db63fac266-f6d343514834eb14212ec1823288b4de.ssl.cf5.rackcdn.com/889196/2/check/tempest-slow-py3/298c784/testr_results.html | |
| 21:09:39 | dansmith | seems like maybe we need to skip it as it's failing a *lot* | |
| 21:10:40 | gmann | dansmith: humm, i saw it earlier when I made this to run in parallel and ykarel mentioned parallel run can be one of the reason. i made it to run serially 2-3 days back but it is still failing | |
| 21:11:00 | dansmith | yeah and it's been failing since before that I think | |
| 21:11:05 | gmann | confirming if it run serially | |
| 21:11:33 | dansmith | ugh, sorry, wrong channel | |
| 21:11:38 | dansmith | meant to be in -qa | |
| 23:24:15 | opendevreview | Merged openstack/nova master: Add compute_id columns to instances, migrations https://review.opendev.org/c/openstack/nova/+/879499 | |
| #openstack-nova - 2023-07-25 | |||
| 01:56:43 | sapd1 | Hi everyone, I'm running openstack train. I have a problem when resize an instance with PCI devices(GPUPassthrough) to a flavor without PCI devices. | |
| 01:56:54 | sapd1 | Does anyone know how to fix this problem. | |
| 01:57:06 | sapd1 | When I perform resize: No valid host was found. No valid host found for resize (HTTP 400) (Request-ID: req-9d6cf3f9-8f6f-482f-b0b1-95ec286a0fc3) | |
| 05:33:40 | auniyal | https://bugs.launchpad.net/nova | |
| 05:33:40 | auniyal | but if you are sure this is not correct and still looking for reason, can you add --debug in your resize cmd to find out exactly which call failed and why, then once you find out that look for req-ID in compute logs. it will give you better idea of fail. if there are any traceback's in logs - you can file a bug here: | |
| 05:33:40 | auniyal | sapd1 - as told No Valid Host. | |
| 07:07:38 | sapd1 | auniyal: this bug: https://bugs.launchpad.net/nova/+bug/1941005 | |
| 07:09:04 | auniyal | it seems fixed | |
| 07:09:55 | sapd1 | auniyal: but it's not merged yet | |
| 07:10:31 | auniyal | oh yes, in train its not merged | |
| 07:19:58 | sapd1 | auniyal: I patched it in my system. | |
| 07:40:43 | opendevreview | Konrad Gube proposed openstack/nova master: Use Cinder's os-extend_volume_completion volume action. https://review.opendev.org/c/openstack/nova/+/873560 | |
| 08:32:18 | bauzas | dansmith: gmann: it occurs to me that nova-next is almost failing on volume detach due to some kind of filesystem check (that's a guess) and kinda related to what you told this night (for me) | |
| 08:32:26 | bauzas | https://6d9b97dc35f887a99105-3214406b4544fce2f9d807df6ea4fe3f.ssl.cf5.rackcdn.com/886232/4/check/nova-next/31d3775/testr_results.html | |
| 09:02:57 | sean-k-mooney | is that using cinder lvm? | |
| 09:03:05 | sean-k-mooney | i.e. not ceph | |
| 09:04:54 | sean-k-mooney | its not quite the saem as what https://github.com/openstack/devstack/commit/58c80b2424623096e4a1f7a901f424be0ce6cb3f adressed and actully that should help there in any case | |
| 09:05:57 | sean-k-mooney | i think https://review.opendev.org/c/openstack/tempest/+/886991 could still help | |
| 09:06:26 | sean-k-mooney | bauzas: i think its https://bugs.launchpad.net/tempest/+bug/2024859 | |
| 09:07:25 | bauzas | sean-k-mooney: -ish yeah | |
| 09:08:42 | sean-k-mooney | i just rebased melwitt patch to see if it will pass ci | |
| 09:09:01 | sean-k-mooney | we have made some other improvment so it might help but im not sure | |
| 09:09:08 | sean-k-mooney | we could also enabel caching in qemu | |
| 09:09:19 | sean-k-mooney | so trun on teh writeback cache | |
| 09:09:51 | sean-k-mooney | and see if that can hide the slow storage performance | |
| 09:13:52 | sean-k-mooney | hehe i was tinkign of setting https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.disk_cachemodes to writeback | |
| 09:14:20 | sean-k-mooney | but we could also set it to unsafe ... that would definetly be faster | |
| 09:15:17 | bauzas | fwiw, I only see this pattern on nova-next | |
| 09:15:55 | bauzas | or, I'd rather say, this failure incidence is way higher on this job than any other one in the nova check and gate pipelines | |
| 09:16:09 | sean-k-mooney | [libvirt]/disk_cachemode=file=none,block=unsafe,network=writeback | |
| 09:17:22 | sean-k-mooney | im goign to add a patch to nova-next to configre that and lets see if it fails | |
| 09:23:42 | opendevreview | sean mooney proposed openstack/nova master: [WIP] use disk caching to hide slow cinder performance https://review.opendev.org/c/openstack/nova/+/889383 | |
| 09:25:47 | sean-k-mooney | bauzas: ^ that might work although in production you would not use unsafe but for ci it should be fine | |
| 09:37:49 | dvo-plv | Hello, nova folks. Maybe you will have a chance to review this commit, thank you https://review.opendev.org/c/openstack/nova/+/876075 | |
| 09:39:47 | sean-k-mooney | i started that review twice already so i proably should go finish it :) | |
| 09:47:27 | sean-k-mooney | dvo-plv: looks good to me | |
| 11:35:44 | kashyap | While I'm off the next 3 days, anyone willing to get this over the line? It's just failing on some timeouts: https://review.opendev.org/c/openstack/nova/+/887255 | |
| 11:36:51 | kashyap | (I see sean-k-mooney already did a recheck on the dep patch; thanks!) | |
| 11:37:41 | sean-k-mooney | it still needs review form other but i set RP+2 on it as well | |
| 11:38:39 | kashyap | Yeah, saw that, thank you! | |
| 12:53:10 | auniyal | sean-k-mooney, and others, how can I set debug=true for nova-scheduler; in devstack nova.conf for [DEFAULT] its already set there is another group [scheduler], I tried there as well but no difference | |
| 12:53:18 | auniyal | I am trying to see https://github.com/openstack/nova/blob/815683ea86492d3ed77b04cc56f3db87e2b8c47d/nova/weights.py#L136 | |
| 12:53:33 | auniyal | in scheduler logs while launching VM | |
| 12:55:19 | auniyal | is there any other conf I should look into ? | |
| 13:06:25 | sean-k-mooney | devstack runs in debug mode by default | |
| 13:07:42 | auniyal | yes, still these logs are not coming in "jouranlctl -u devstack@n-sch" | |
| 13:09:34 | sean-k-mooney | stack@upstream-devstack:~$ sudo journalctl -u devstack@n-sch | grep debug | wc | |
| 13:09:36 | sean-k-mooney | 10 172 2932 | |
| 13:11:54 | auniyal | yes debugs logs are coming in n-sch but not these logs https://github.com/openstack/nova/blob/815683ea86492d3ed77b04cc56f3db87e2b8c47d/nova/weights.py#L136 | |
| 13:12:16 | auniyal | so I think they should come whie VM create | |
| 13:12:32 | auniyal | right ? | |
| 13:12:50 | sean-k-mooney | im seeign the log in the ci | |
| 13:13:17 | sean-k-mooney | https://zuul.opendev.org/t/openstack/build/fd53404ef23341828ae15f8ccf596e6c/log/controller/logs/screen-n-sch.txt#1018 | |
| 13:15:04 | auniyal | yes, here they are coming, I have single-node devstack env localy | |
| 13:15:57 | auniyal | it should be right ? | |
| 13:23:51 | sean-k-mooney | /etc/nova/nova.conf as rendered by devstack has debug=true set in the default section | |
| 13:24:08 | sean-k-mooney | that is what the scheduler uses | |
| 13:30:11 | auniyal | ack, thanks | |
| 13:33:25 | sean-k-mooney | the backport does not seam to be working properly | |
| 13:34:41 | sean-k-mooney | wait why are you proposing https://review.opendev.org/c/openstack/nova/+/889311 directly to yoga | |
| 13:34:55 | sean-k-mooney | oh your are not | |
| 13:35:03 | sean-k-mooney | this is incorrect | |
| 14:01:43 | auniyal | ack | |
| 14:10:58 | opendevreview | alecorps proposed openstack/nova master: Workaround for issues with ephemeral disk named disk.local during resize https://review.opendev.org/c/openstack/nova/+/888220 | |
| 14:14:56 | opendevreview | Amit Uniyal proposed openstack/nova master: Added context manager for instance lock https://review.opendev.org/c/openstack/nova/+/873648 | |
| 14:14:57 | opendevreview | Amit Uniyal proposed openstack/nova master: Disconnecting volume from the compute host https://review.opendev.org/c/openstack/nova/+/877446 | |
| 14:37:34 | bauzas | elodilles: gibi: I'll need to stop leading the nova meeting after 20-25 mins | |
| 14:37:53 | bauzas | Uggla and me are going to visit artom who's in the surroundings | |
| 14:39:47 | opendevreview | Maxim Monin proposed openstack/nova master: Server Rescue leads to Server ERROR state if original image is deleted https://review.opendev.org/c/openstack/nova/+/872385 | |
| 14:42:36 | elodilles | bauzas: hmmm... i might be late like 20-25 mins, actually :S | |
| 14:43:06 | elodilles | (though i hope not...) | |
| 14:47:01 | auniyal | hey melwitt, I heard PS[number] from you few times, but not sure what it is, like PS1 or PS10 | |
| 14:47:30 | bauzas | auniyal: PS = patchset | |
| 14:47:40 | bauzas | the revision number in gerrit | |
| 14:47:59 | auniyal | so in one change patchset 10 | |
| 14:48:04 | auniyal | ack, thanks | |
| 14:56:59 | gibi | bauzas: I probably need to skip today's meeting (or I will be on and off during it) | |
| 14:57:12 | bauzas | np | |
| 14:57:29 | bauzas | honestly, given this and that, I don't want to skip but I'll say that we'll just check a few things | |
| 14:59:02 | gibi | ack | |
| 15:52:07 | auniyal | gibi, bauzas, dansmith, sean-k-mooney, melwitt can you please review these stable branch patches - https://etherpad.opendev.org/p/release-liaison-PatchesToReview | |
| 15:52:40 | auniyal | most of them are clean cherry-pick and already have 1 +2 and good to merge | |
| 15:54:15 | sean-k-mooney | the gates on stabel shoudl be unblocked | |
| 15:54:28 | sean-k-mooney | the nova-lvm and ceph issues shoudl be fixed on all branches | |
| 15:55:58 | bauzas | auniyal: ack, my main prio before leaving for 3 weeks is about features reviews but I'll try | |
| 15:56:16 | elodilles | yepp, they are unblocked, thanks sean-k-mooney for the patches \o/ | |
| 16:00:23 | opendevmeet | The meeting name has been set to 'nova' | |
| 16:00:23 | opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | |
| 16:00:23 | opendevmeet | Meeting started Tue Jul 25 16:00:23 2023 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | |
| 16:00:23 | bauzas | #startmeeting nova | |
| 16:00:39 | bauzas | hey, let's try to have a 10-min meeting if we can | |
| 16:00:43 | dansmith | o/ | |