| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-27 | |||
| 09:16:49 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add type annotations to top-level modules https://review.opendev.org/c/openstack/nova/+/705658 | |
| 09:16:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Clean manager.Manager, service.Service signatures https://review.opendev.org/c/openstack/nova/+/764806 | |
| 10:03:48 | openstackgerrit | Xinran WANG proposed openstack/nova-specs master: Repropose smartnic support spec https://review.opendev.org/c/openstack/nova-specs/+/783632 | |
| 10:20:14 | openstackgerrit | Daniel Bengtsson proposed openstack/nova master: Use the new type HostDomainOpt. https://review.opendev.org/c/openstack/nova/+/788240 | |
| 10:54:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Allow provider re-parenting in placement https://review.opendev.org/c/openstack/nova-specs/+/788243 | |
| 10:55:18 | gibi | bauzas: ^^ I've moved the proposal from the storyboard to a nova spec for review. I've added the saferty mechanism altenratives to the spec. If you have time please check it. I have problems with the PATCH way too | |
| 10:55:29 | gibi | bauzas: ^^ I've moved the proposal from the storyboard to a nova spec for review. I've added the saferty mechanism altenratives to the spec. If you have time please check it. I have problems with the PATCH way too | |
| 12:10:29 | openstackgerrit | Balazs Gibizer proposed openstack/placement master: Add support for RP re-parenting and orphaning https://review.opendev.org/c/openstack/placement/+/784020 | |
| 12:10:30 | openstackgerrit | Balazs Gibizer proposed openstack/placement master: Add support for RP re-parenting and orphaning https://review.opendev.org/c/openstack/placement/+/784020 | |
| 12:30:16 | bauzas | gibi: ack, will look | |
| 12:30:31 | gibi | bauzas: thansk | |
| 12:30:32 | gibi | bauzas: thansk | |
| 12:45:03 | frickler | has anyone seen the situation where nova-compute logs "Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0)" and then goes on with that for days? usually these byte counts should be > 0, possibly some libvirt bug? | |
| 12:45:03 | frickler | has anyone seen the situation where nova-compute logs "Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0)" and then goes on with that for days? usually these byte counts should be > 0, possibly some libvirt bug? | |
| 12:53:54 | sean-k-mooney | frickler: we have seen that in the past | |
| 12:53:54 | sean-k-mooney | frickler: we have seen that in the past | |
| 12:55:27 | sean-k-mooney | i cant recall what was the cause but i know the env had rabbitmq issuues and network issues | |
| 12:55:28 | sean-k-mooney | i cant recall what was the cause but i know the env had rabbitmq issuues and network issues | |
| 12:58:08 | sean-k-mooney | frickler: the first message by the way is normally (bytes processed=0, remaining=0, total=0) | |
| 12:58:08 | sean-k-mooney | frickler: the first message by the way is normally (bytes processed=0, remaining=0, total=0) | |
| 12:58:33 | sean-k-mooney | then it get updated when the migration actully starts | |
| 12:58:33 | sean-k-mooney | then it get updated when the migration actully starts | |
| 12:59:00 | sean-k-mooney | 2018-04-16 20:46:31.747 767495 INFO nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) | |
| 12:59:00 | sean-k-mooney | 2018-04-16 20:46:31.747 767495 INFO nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) | |
| 12:59:02 | sean-k-mooney | 2018-04-16 20:46:37.422 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 5 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) _live_migration_monitor | |
| 12:59:02 | sean-k-mooney | 2018-04-16 20:46:37.422 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 5 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) _live_migration_monitor | |
| 12:59:04 | sean-k-mooney | /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428 | |
| 12:59:06 | sean-k-mooney | 2018-04-16 20:46:43.713 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 10 secs, memory 0% remaining; (bytes processed=387830565, remaining=0, total=4312604672) _live_migration_monitor | |
| 12:59:06 | sean-k-mooney | 2018-04-16 20:46:43.713 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 10 secs, memory 0% remaining; (bytes processed=387830565, remaining=0, total=4312604672) _live_migration_monitor | |
| 12:59:08 | sean-k-mooney | /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428 | |
| 12:59:08 | sean-k-mooney | /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428 | |
| 12:59:10 | sean-k-mooney | like htat | |
| 12:59:11 | sean-k-mooney | like htat | |
| 13:00:43 | frickler | in my case, it stays the same for 80000 seconds or so now, migration not finishing at all | |
| 13:00:43 | frickler | in my case, it stays the same for 80000 seconds or so now, migration not finishing at all | |
| 13:01:15 | sean-k-mooney | frickler: do you happen to have ceph? | |
| 13:01:15 | sean-k-mooney | frickler: do you happen to have ceph? | |
| 13:01:38 | frickler | sean-k-mooney: for volume storage, yes. root disk is local | |
| 13:01:38 | frickler | sean-k-mooney: for volume storage, yes. root disk is local | |
| 13:01:54 | sean-k-mooney | that was form https://bugzilla.redhat.com/show_bug.cgi?id=1566723 | |
| 13:01:55 | openstack | bugzilla.redhat.com bug 1566723 in RBD "Instance live migration times out after <x> seconds with nova - doesn't copy instance memory" [High,Closed: errata] - Assigned to jdillama | |
| 13:01:55 | sean-k-mooney | that was form https://bugzilla.redhat.com/show_bug.cgi?id=1566723 | |
| 13:01:55 | openstack | bugzilla.redhat.com bug 1566723 in RBD "Instance live migration times out after <x> seconds with nova - doesn't copy instance memory" [High,Closed: errata] - Assigned to jdillama | |
| 13:03:08 | sean-k-mooney | frickler: apparently restarting OSDs durign the migration could cause locking issues in librbd in the past | |
| 13:03:08 | sean-k-mooney | frickler: apparently restarting OSDs durign the migration could cause locking issues in librbd in the past | |
| 13:03:33 | frickler | oh, I can check whether this is actually a boot-from-volume instance | |
| 13:03:33 | frickler | oh, I can check whether this is actually a boot-from-volume instance | |
| 13:04:20 | sean-k-mooney | frickler: there is also https://bugs.launchpad.net/nova/+bug/1583145 | |
| 13:04:21 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "duplicate for #1583145 Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:04:21 | sean-k-mooney | frickler: there is also https://bugs.launchpad.net/nova/+bug/1583145 | |
| 13:04:21 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "duplicate for #1583145 Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:04:44 | sean-k-mooney | oh the other one is fixed | |
| 13:04:44 | sean-k-mooney | oh the other one is fixed | |
| 13:04:49 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1644248 | |
| 13:04:50 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:04:50 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1644248 | |
| 13:04:50 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:18:13 | sean-k-mooney | bauzas: gibi lyarwood if we were to impleemnte the generic nova manage command for updating image metadata | |
| 13:18:13 | sean-k-mooney | bauzas: gibi lyarwood if we were to impleemnte the generic nova manage command for updating image metadata | |
| 13:18:26 | sean-k-mooney | how would ye feel about backpoarting hat | |
| 13:18:26 | sean-k-mooney | how would ye feel about backpoarting hat | |
| 13:18:28 | sean-k-mooney | *that | |
| 13:18:28 | sean-k-mooney | *that | |
| 13:18:57 | sean-k-mooney | elod: any tought form a stable core perspective | |
| 13:18:57 | sean-k-mooney | elod: any tought form a stable core perspective | |
| 13:19:44 | sean-k-mooney | context is we have a custoemr downstream that really want to fix a few instance by adding os-type=windows to the inatnce that were booted before they fixed there windows images | |
| 13:19:45 | sean-k-mooney | context is we have a custoemr downstream that really want to fix a few instance by adding os-type=windows to the inatnce that were booted before they fixed there windows images | |
| 13:20:18 | sean-k-mooney | and today they will have to do a manual db update to do that | |
| 13:20:18 | sean-k-mooney | and today they will have to do a manual db update to do that | |
| 13:20:38 | sean-k-mooney | then live/cold migrate teh inststances so they land up in the correct host aggrate after | |
| 13:20:38 | sean-k-mooney | then live/cold migrate teh inststances so they land up in the correct host aggrate after | |
| 13:21:38 | bauzas | sean-k-mooney: in general, we don't accept to backport features like this | |
| 13:21:39 | bauzas | sean-k-mooney: in general, we don't accept to backport features like this | |
| 13:21:42 | bauzas | (upstream) | |
| 13:21:42 | bauzas | (upstream) | |
| 13:22:23 | sean-k-mooney | bauzas: in general but we have backported nova-manage command on the bases it resolved operator pain in the past | |
| 13:22:24 | sean-k-mooney | bauzas: in general but we have backported nova-manage command on the bases it resolved operator pain in the past | |
| 13:22:35 | sean-k-mooney | like the heal allocations ones | |
| 13:22:35 | sean-k-mooney | like the heal allocations ones | |
| 13:28:33 | bauzas | sean-k-mooney: yup, because it's a bugfix | |
| 13:28:33 | bauzas | sean-k-mooney: yup, because it's a bugfix | |
| 13:29:49 | bauzas | but fwiw, we abandoned the backports for the placement audit https://review.opendev.org/q/I537ed74503d208957f0a97af3ab754a6750dac20 | |
| 13:29:49 | bauzas | but fwiw, we abandoned the backports for the placement audit https://review.opendev.org/q/I537ed74503d208957f0a97af3ab754a6750dac20 | |
| 13:41:45 | elod | sean-k-mooney: bauzas is right: in general, we don't accept to backport features like this | |
| 13:41:46 | elod | sean-k-mooney: bauzas is right: in general, we don't accept to backport features like this | |
| 13:43:18 | frickler | sean-k-mooney: turns out this migration was in status "queued", being held up by another one in progress that is having trouble getting the disk synced, seems I didn't look close enough there. thanks for your help anyway | |
| 13:43:18 | frickler | sean-k-mooney: turns out this migration was in status "queued", being held up by another one in progress that is having trouble getting the disk synced, seems I didn't look close enough there. thanks for your help anyway | |
| 13:43:59 | sean-k-mooney | frickler: glad you figured it out | |
| 13:43:59 | sean-k-mooney | frickler: glad you figured it out | |
| 13:45:16 | sean-k-mooney | elod: bauzas yep i know and in general we do not allow custoemr to do db udate eitehr. i dont partically want to do a downstream only backport of that feature. i think it would be generic useful but if we do not want to backport it that is fine | |
| 13:45:16 | sean-k-mooney | elod: bauzas yep i know and in general we do not allow custoemr to do db udate eitehr. i dont partically want to do a downstream only backport of that feature. i think it would be generic useful but if we do not want to backport it that is fine | |
| 14:09:50 | elod | sean-k-mooney: well, according to policy, there could be exceptions. I still tend to say no for this backport (though I haven't seen the implementation) | |
| 14:09:50 | elod | sean-k-mooney: well, according to policy, there could be exceptions. I still tend to say no for this backport (though I haven't seen the implementation) | |
| 14:10:22 | elod | exception* -- if stable cores agree... | |
| 14:10:22 | elod | exception* -- if stable cores agree... | |
| 14:10:51 | sean-k-mooney | elod: its not written yet :) | |
| 14:10:51 | sean-k-mooney | elod: its not written yet :) | |
| 14:12:49 | sean-k-mooney | elod: it would be https://review.opendev.org/c/openstack/nova/+/774896/8/nova/cmd/manage.py excpet instead of setting just machine_type you could set any image metatadata value that does not chagne resouce usage | |
| 14:12:49 | sean-k-mooney | elod: it would be https://review.opendev.org/c/openstack/nova/+/774896/8/nova/cmd/manage.py excpet instead of setting just machine_type you could set any image metatadata value that does not chagne resouce usage | |
| 14:13:57 | sean-k-mooney | elod: anyway its fine i was going to say if it was short to implement and contovertial then maybe instead of deliving a custom sql srcript we could implement this and backport it for our customer to use | |