| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-04-27 | |||
| 02:42:06 | __ministry | Anybody can help me to fix this? | |
| 06:43:26 | openstackgerrit | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List/Update Servers APIs https://review.opendev.org/c/openstack/nova/+/764292 | |
| 06:43:27 | openstackgerrit | Brin Zhang proposed openstack/nova master: Replace all_tenants with all_projects in List Server APIs https://review.opendev.org/c/openstack/nova/+/765311 | |
| 07:24:26 | gibi | __ministry: hi! have you followed the pointers / questions in the comment of that bug report? | |
| 07:24:26 | gibi | __ministry: hi! have you followed the pointers / questions in the comment of that bug report? | |
| 07:27:11 | __ministry | oh, this bug not reported by me. I will follow log to see more, thank you. | |
| 07:27:11 | __ministry | oh, this bug not reported by me. I will follow log to see more, thank you. | |
| 07:27:35 | gibi | __ministry: those questions there might help you figuring out what went wrong | |
| 07:27:35 | gibi | __ministry: those questions there might help you figuring out what went wrong | |
| 07:30:35 | __ministry | gibi: yes, I was trying config neutron-api with rpc_response_time options, because bug not appear always, I doing monitor this bug. | |
| 07:30:35 | __ministry | gibi: yes, I was trying config neutron-api with rpc_response_time options, because bug not appear always, I doing monitor this bug. | |
| 07:52:21 | sean-k-mooney | lbragstad: it uses ksa directly also because we were trying to move to it before the openstack sdk was really a thing | |
| 07:52:22 | sean-k-mooney | lbragstad: it uses ksa directly also because we were trying to move to it before the openstack sdk was really a thing | |
| 07:53:15 | sean-k-mooney | lbragstad: we can move to neutorn client now and stephenfin has patches to move it i belive. i would prefer us to move to the sdk but i dont really see an issue in using ksa directly | |
| 07:53:15 | sean-k-mooney | lbragstad: we can move to neutorn client now and stephenfin has patches to move it i belive. i would prefer us to move to the sdk but i dont really see an issue in using ksa directly | |
| 08:40:56 | openstackgerrit | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from Rebuild Server API https://review.opendev.org/c/openstack/nova/+/766380 | |
| 09:16:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add type annotations to 'nova.pci' https://review.opendev.org/c/openstack/nova/+/676209 | |
| 09:16:42 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add type annotations to 'nova.pci' https://review.opendev.org/c/openstack/nova/+/676209 | |
| 09:16:44 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add nova.cmd, nova.conf, nova.console https://review.opendev.org/c/openstack/nova/+/705657 | |
| 09:16:49 | openstackgerrit | Stephen Finucane proposed openstack/nova master: mypy: Add type annotations to top-level modules https://review.opendev.org/c/openstack/nova/+/705658 | |
| 09:16:53 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Clean manager.Manager, service.Service signatures https://review.opendev.org/c/openstack/nova/+/764806 | |
| 10:03:48 | openstackgerrit | Xinran WANG proposed openstack/nova-specs master: Repropose smartnic support spec https://review.opendev.org/c/openstack/nova-specs/+/783632 | |
| 10:20:14 | openstackgerrit | Daniel Bengtsson proposed openstack/nova master: Use the new type HostDomainOpt. https://review.opendev.org/c/openstack/nova/+/788240 | |
| 10:54:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Allow provider re-parenting in placement https://review.opendev.org/c/openstack/nova-specs/+/788243 | |
| 10:55:18 | gibi | bauzas: ^^ I've moved the proposal from the storyboard to a nova spec for review. I've added the saferty mechanism altenratives to the spec. If you have time please check it. I have problems with the PATCH way too | |
| 10:55:29 | gibi | bauzas: ^^ I've moved the proposal from the storyboard to a nova spec for review. I've added the saferty mechanism altenratives to the spec. If you have time please check it. I have problems with the PATCH way too | |
| 12:10:29 | openstackgerrit | Balazs Gibizer proposed openstack/placement master: Add support for RP re-parenting and orphaning https://review.opendev.org/c/openstack/placement/+/784020 | |
| 12:10:30 | openstackgerrit | Balazs Gibizer proposed openstack/placement master: Add support for RP re-parenting and orphaning https://review.opendev.org/c/openstack/placement/+/784020 | |
| 12:30:16 | bauzas | gibi: ack, will look | |
| 12:30:31 | gibi | bauzas: thansk | |
| 12:30:32 | gibi | bauzas: thansk | |
| 12:45:03 | frickler | has anyone seen the situation where nova-compute logs "Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0)" and then goes on with that for days? usually these byte counts should be > 0, possibly some libvirt bug? | |
| 12:45:03 | frickler | has anyone seen the situation where nova-compute logs "Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0)" and then goes on with that for days? usually these byte counts should be > 0, possibly some libvirt bug? | |
| 12:53:54 | sean-k-mooney | frickler: we have seen that in the past | |
| 12:53:54 | sean-k-mooney | frickler: we have seen that in the past | |
| 12:55:27 | sean-k-mooney | i cant recall what was the cause but i know the env had rabbitmq issuues and network issues | |
| 12:55:28 | sean-k-mooney | i cant recall what was the cause but i know the env had rabbitmq issuues and network issues | |
| 12:58:08 | sean-k-mooney | frickler: the first message by the way is normally (bytes processed=0, remaining=0, total=0) | |
| 12:58:08 | sean-k-mooney | frickler: the first message by the way is normally (bytes processed=0, remaining=0, total=0) | |
| 12:58:33 | sean-k-mooney | then it get updated when the migration actully starts | |
| 12:58:33 | sean-k-mooney | then it get updated when the migration actully starts | |
| 12:59:00 | sean-k-mooney | 2018-04-16 20:46:31.747 767495 INFO nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) | |
| 12:59:00 | sean-k-mooney | 2018-04-16 20:46:31.747 767495 INFO nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 0 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) | |
| 12:59:02 | sean-k-mooney | 2018-04-16 20:46:37.422 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 5 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) _live_migration_monitor | |
| 12:59:02 | sean-k-mooney | 2018-04-16 20:46:37.422 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 5 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0) _live_migration_monitor | |
| 12:59:04 | sean-k-mooney | /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428 | |
| 12:59:06 | sean-k-mooney | 2018-04-16 20:46:43.713 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 10 secs, memory 0% remaining; (bytes processed=387830565, remaining=0, total=4312604672) _live_migration_monitor | |
| 12:59:06 | sean-k-mooney | 2018-04-16 20:46:43.713 767495 DEBUG nova.virt.libvirt.driver [req-7af65a7d-9d61-4b4b-955d-07d7791b18ac 13515e05a63e48e0b9adb991b250c5f7 8bc196f362b442a4891986517b60389c - - -] [instance: e9c1de25-daa8-43f6-b9b8-a909c19255d9] Migration running for 10 secs, memory 0% remaining; (bytes processed=387830565, remaining=0, total=4312604672) _live_migration_monitor | |
| 12:59:08 | sean-k-mooney | /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428 | |
| 12:59:08 | sean-k-mooney | /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:6428 | |
| 12:59:10 | sean-k-mooney | like htat | |
| 12:59:11 | sean-k-mooney | like htat | |
| 13:00:43 | frickler | in my case, it stays the same for 80000 seconds or so now, migration not finishing at all | |
| 13:00:43 | frickler | in my case, it stays the same for 80000 seconds or so now, migration not finishing at all | |
| 13:01:15 | sean-k-mooney | frickler: do you happen to have ceph? | |
| 13:01:15 | sean-k-mooney | frickler: do you happen to have ceph? | |
| 13:01:38 | frickler | sean-k-mooney: for volume storage, yes. root disk is local | |
| 13:01:38 | frickler | sean-k-mooney: for volume storage, yes. root disk is local | |
| 13:01:54 | sean-k-mooney | that was form https://bugzilla.redhat.com/show_bug.cgi?id=1566723 | |
| 13:01:55 | openstack | bugzilla.redhat.com bug 1566723 in RBD "Instance live migration times out after <x> seconds with nova - doesn't copy instance memory" [High,Closed: errata] - Assigned to jdillama | |
| 13:01:55 | sean-k-mooney | that was form https://bugzilla.redhat.com/show_bug.cgi?id=1566723 | |
| 13:01:55 | openstack | bugzilla.redhat.com bug 1566723 in RBD "Instance live migration times out after <x> seconds with nova - doesn't copy instance memory" [High,Closed: errata] - Assigned to jdillama | |
| 13:03:08 | sean-k-mooney | frickler: apparently restarting OSDs durign the migration could cause locking issues in librbd in the past | |
| 13:03:08 | sean-k-mooney | frickler: apparently restarting OSDs durign the migration could cause locking issues in librbd in the past | |
| 13:03:33 | frickler | oh, I can check whether this is actually a boot-from-volume instance | |
| 13:03:33 | frickler | oh, I can check whether this is actually a boot-from-volume instance | |
| 13:04:20 | sean-k-mooney | frickler: there is also https://bugs.launchpad.net/nova/+bug/1583145 | |
| 13:04:21 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "duplicate for #1583145 Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:04:21 | sean-k-mooney | frickler: there is also https://bugs.launchpad.net/nova/+bug/1583145 | |
| 13:04:21 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "duplicate for #1583145 Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:04:44 | sean-k-mooney | oh the other one is fixed | |
| 13:04:44 | sean-k-mooney | oh the other one is fixed | |
| 13:04:49 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1644248 | |
| 13:04:50 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:04:50 | sean-k-mooney | https://bugs.launchpad.net/nova/+bug/1644248 | |
| 13:04:50 | openstack | Launchpad bug 1644248 in OpenStack Compute (nova) "Nova incorrectly tracks live migration progress" [High,Fix released] | |
| 13:18:13 | sean-k-mooney | bauzas: gibi lyarwood if we were to impleemnte the generic nova manage command for updating image metadata | |
| 13:18:13 | sean-k-mooney | bauzas: gibi lyarwood if we were to impleemnte the generic nova manage command for updating image metadata | |
| 13:18:26 | sean-k-mooney | how would ye feel about backpoarting hat | |
| 13:18:26 | sean-k-mooney | how would ye feel about backpoarting hat | |
| 13:18:28 | sean-k-mooney | *that | |
| 13:18:28 | sean-k-mooney | *that | |
| 13:18:57 | sean-k-mooney | elod: any tought form a stable core perspective | |
| 13:18:57 | sean-k-mooney | elod: any tought form a stable core perspective | |
| 13:19:44 | sean-k-mooney | context is we have a custoemr downstream that really want to fix a few instance by adding os-type=windows to the inatnce that were booted before they fixed there windows images | |
| 13:19:45 | sean-k-mooney | context is we have a custoemr downstream that really want to fix a few instance by adding os-type=windows to the inatnce that were booted before they fixed there windows images | |
| 13:20:18 | sean-k-mooney | and today they will have to do a manual db update to do that | |
| 13:20:18 | sean-k-mooney | and today they will have to do a manual db update to do that | |
| 13:20:38 | sean-k-mooney | then live/cold migrate teh inststances so they land up in the correct host aggrate after | |
| 13:20:38 | sean-k-mooney | then live/cold migrate teh inststances so they land up in the correct host aggrate after | |
| 13:21:38 | bauzas | sean-k-mooney: in general, we don't accept to backport features like this | |
| 13:21:39 | bauzas | sean-k-mooney: in general, we don't accept to backport features like this | |
| 13:21:42 | bauzas | (upstream) | |
| 13:21:42 | bauzas | (upstream) | |
| 13:22:23 | sean-k-mooney | bauzas: in general but we have backported nova-manage command on the bases it resolved operator pain in the past | |
| 13:22:24 | sean-k-mooney | bauzas: in general but we have backported nova-manage command on the bases it resolved operator pain in the past | |
| 13:22:35 | sean-k-mooney | like the heal allocations ones | |
| 13:22:35 | sean-k-mooney | like the heal allocations ones | |
| 13:28:33 | bauzas | sean-k-mooney: yup, because it's a bugfix | |
| 13:28:33 | bauzas | sean-k-mooney: yup, because it's a bugfix | |