| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-05-22 | |||
| 12:01:58 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1736, in attach_volume | |
| 12:02:00 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] guest.attach_device(conf, persistent=True, live=live) | |
| 12:02:02 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/virt/libvirt/guest.py", line 293, in attach_device | |
| 12:02:04 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] self._domain.attachDeviceFlags(device_xml, flags=flags) | |
| 12:02:06 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit | |
| 12:02:08 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] result = proxy_call(self._autowrap, f, *args, **kwargs) | |
| 12:02:10 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call | |
| 12:02:12 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] rv = execute(f, *args, **kwargs) | |
| 12:02:14 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute | |
| 12:02:16 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] six.reraise(c, e, tb) | |
| 12:02:18 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/usr/lib/python3.6/site-packages/six.py", line 703, in reraise | |
| 12:02:20 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] raise value | |
| 12:02:22 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/tpool.py", line 83, in tworker | |
| 12:02:24 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] rv = meth(*args, **kwargs) | |
| 12:02:26 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] File "/usr/lib64/python3.6/site-packages/libvirt.py", line 614, in attachDeviceFlags | |
| 12:02:28 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) | |
| 12:02:30 | sean-k-mooney | 2020-05-22 00:28:20.355 6 ERROR nova.compute.manager [instance: 8cfaf7ce-13ae-4b50-9bd8-20c957a8dfb5] libvirt.libvirtError: internal error: child reported (status=125): unable to open /dev/sdg: No such device or address | |
| 12:02:32 | sean-k-mooney | ops | |
| 12:02:34 | sean-k-mooney | http://paste.openstack.org/show/793881/ | |
| 12:02:36 | sean-k-mooney | ment to past the link | |
| 12:02:38 | sean-k-mooney | lyarwood: are you still working on stable rescue? | |
| 12:02:50 | gibi | I think stable rescue was finished in ussuri | |
| 12:03:06 | sean-k-mooney | this might be related or might not. i have different disk_bus and cdrom_bus image proterties set in my orginal image and the rescue image | |
| 12:03:51 | sean-k-mooney | i might do an upgrade of my cloud and see if the issue still exists. i have a random commit off master deployed | |
| 12:06:12 | lyarwood | sean-k-mooney: finished in U, what's up? | |
| 12:06:51 | sean-k-mooney | if we use q35 i have found at least two cases where attach volume is broken | |
| 12:06:58 | sean-k-mooney | one is when you boot form an iso | |
| 12:07:10 | sean-k-mooney | the other is if you try to use an iso for rescue | |
| 12:07:29 | lyarwood | sean-k-mooney: in both cases is it trying to use an IDE bus? | |
| 12:07:40 | sean-k-mooney | no | |
| 12:08:05 | sean-k-mooney | the iso image is using sata | |
| 12:08:18 | sean-k-mooney | i set cdrom_bus=sata in the image meta | |
| 12:09:19 | openstackgerrit | Lee Yarwood proposed openstack/nova master: DNM: Test m1.nano flavor with 96MB to avoid tmpfs exhaustion https://review.opendev.org/730308 | |
| 12:09:30 | sean-k-mooney | lyarwood: i do not have rescue_bus set in any of the cases | |
| 12:10:18 | lyarwood | sean-k-mooney: kk can you write up some bugs for that? | |
| 12:10:44 | lyarwood | I've had my head in cirros sysvinit scripts all morning | |
| 12:10:45 | sean-k-mooney | yep i found it last night after trying to create a windows vm from an iso | |
| 12:13:51 | sean-k-mooney | the code is now different on master https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1416-L1425 | |
| 12:14:01 | sean-k-mooney | but it might stil have the bug | |
| 12:14:18 | openstackgerrit | Merged openstack/nova stable/stein: Update scheduler instance info at confirm resize https://review.opendev.org/729538 | |
| 12:14:37 | sean-k-mooney | max([]) | |
| 12:14:39 | sean-k-mooney | Traceback (most recent call last): | |
| 12:14:41 | sean-k-mooney | File "<stdin>", line 1, in <module> | |
| 12:14:43 | sean-k-mooney | ValueError: max() arg is an empty sequence | |
| 12:14:45 | sean-k-mooney | yep it does | |
| 12:14:55 | sean-k-mooney | oh no its fixed | |
| 12:15:02 | sean-k-mooney | max(ret) + 1 if ret else 0 | |
| 12:15:15 | sean-k-mooney | the if will prevent the ValueError | |
| 12:20:30 | sean-k-mooney | https://github.com/openstack/nova/commit/c8d6767cf8baaf3cc81496c83db10c8ae72fce06 is what im missing | |
| 12:21:56 | sean-k-mooney | so we should backport https://review.opendev.org/#/c/712607/ | |
| 12:24:55 | gibi | sean-k-mooney: +1 on that to backport | |
| 12:25:39 | sean-k-mooney | ill check how far back we had the old logic and then start propsoing them | |
| 12:26:58 | gibi | cool | |
| 12:33:50 | sean-k-mooney | apparently forever or at last to ocata. ill backport to queens | |
| 12:35:42 | openstackgerrit | sean mooney proposed openstack/nova stable/train: fix scsi disk unit number of the attaching volume when cdrom bus is scsi https://review.opendev.org/730310 | |
| 12:36:33 | openstackgerrit | sean mooney proposed openstack/nova stable/stein: fix scsi disk unit number of the attaching volume when cdrom bus is scsi https://review.opendev.org/730311 | |
| 12:36:55 | openstackgerrit | sean mooney proposed openstack/nova stable/rocky: fix scsi disk unit number of the attaching volume when cdrom bus is scsi https://review.opendev.org/730312 | |
| 12:37:15 | openstackgerrit | sean mooney proposed openstack/nova stable/queens: fix scsi disk unit number of the attaching volume when cdrom bus is scsi https://review.opendev.org/730313 | |
| 12:51:46 | openstackgerrit | Elod Illes proposed openstack/nova stable/ussuri: remove support of oslo.messaging 9.8.0 warning message https://review.opendev.org/728055 | |
| 12:56:26 | jsuchome | lyarwood: hey, just a humble ping with a request for a new review... | |
| 12:59:36 | lyarwood | jsuchome: ack looking | |
| 13:08:20 | openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Make barbican-simple-crypto-devstack-tempest a non-voting check job https://review.opendev.org/730319 | |
| 13:08:20 | openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Make devstack-plugin-ceph-tempest-py3 a voting check job again https://review.opendev.org/730320 | |
| 13:10:40 | sean-k-mooney | gibi: dansmith added https://github.com/openstack/nova/commit/04e79b5b7e18f033053d4057edc768281b91a4d7 to reserve migration for ussuri but we also merged https://github.com/openstack/nova/commit/75a2d600fc26beda78bdafd209e92f0bc9802927 | |
| 13:10:55 | sean-k-mooney | https://github.com/openstack/nova/commit/75a2d600fc26beda78bdafd209e92f0bc9802927 is not on stable/ussuri however | |
| 13:11:12 | sean-k-mooney | do you know why we have 2 set of placeholder migration for ussuri | |
| 13:11:22 | sean-k-mooney | and why only one of them is in stable ussuri | |
| 13:12:12 | sean-k-mooney | is the commit message in https://github.com/openstack/nova/commit/75a2d600fc26beda78bdafd209e92f0bc9802927 just wrong and those are for victoria | |
| 13:14:29 | gibi | sean-k-mooney: good question | |
| 13:15:14 | sean-k-mooney | it add ussuri_placeholders = list(range(408, 413)) to the migration tests | |
| 13:15:40 | sean-k-mooney | but i would expect that to get run on stable/ussuri but i guess i dont really follow how this is ment to work | |
| 13:16:19 | gibi | I think only the commit message is missleading both patches are correct, they reserve db migration for bugfixes | |
| 13:16:43 | sean-k-mooney | right the patches are more or less idential | |
| 13:17:10 | sean-k-mooney | im just wondering what we expect to see on stable/ussuri | |
| 13:17:35 | sean-k-mooney | i see the place holder up to 072 https://github.com/openstack/nova/tree/stable/ussuri/nova/db/sqlalchemy/api_migrations/migrate_repo/versions | |
| 13:17:44 | sean-k-mooney | which were intoduced by dansmith | |
| 13:17:51 | sean-k-mooney | so i think that is correct | |
| 13:18:22 | sean-k-mooney | but if i look at his patch those are listed as train_placeholders = list(range(68, 73)) | |
| 13:18:29 | gibi | when you want to backport a migraiton to ussuri you have to use one of the reserved migration from https://review.opendev.org/#/c/722546 if you want to add a new migration that is not backported then you have to use 078 | |
| 13:19:25 | sean-k-mooney | right but what is confutin me is we are calling them train_placeholders in the tests | |
| 13:19:32 | gibi | train_placeholders means for me that such versions are reserved for migration that might need to be backported to train | |
| 13:20:06 | sean-k-mooney | ok that a different way to look at it | |
| 13:20:17 | gibi | when we started ussuri we reserved migrations for stable/train when we started V we reserved migration for stable/ussur | |
| 13:20:20 | gibi | i | |
| 13:20:45 | sean-k-mooney | but see that does not fully make sense to me | |
| 13:20:58 | sean-k-mooney | oh well maybe | |
| 13:21:13 | sean-k-mooney | when we do the backprot to stable usue it would start with 73? | |
| 13:21:40 | sean-k-mooney | which is hte first migration we reseved in the vicrotia cycle | |
| 13:21:50 | sean-k-mooney | for backport to ussuri | |
| 13:22:30 | sean-k-mooney | ok i think i understand how this work now | |
| 13:23:51 | gibi | cool | |
| 13:24:40 | sean-k-mooney | i still find it slightly backwards but i am now following at least | |
| 13:25:45 | sean-k-mooney | ii was expecteing each release to add backport slot for backport to that release but i was off by 1 | |
| 13:33:15 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/train: Reserve DISK_GB resource for the image cache https://review.opendev.org/730324 | |
| 13:34:26 | openstackgerrit | Balazs Gibizer proposed openstack/nova stable/train: Add admin doc information about image cache resource accounting https://review.opendev.org/730326 | |
| 13:47:43 | lyarwood | sean-k-mooney: https://github.com/cirros-dev/cirros/issues/53 - something to be aware of in the future with cirros 0.5.0 and q35 | |
| 13:47:48 | mnaser | https://review.opendev.org/#/c/729088/ if anyone wants to finish backporting it all the way back :) | |
| 13:48:21 | sean-k-mooney | lyarwood: aw you mean we cant keep using or normal 64mb nano images | |
| 13:48:49 | sean-k-mooney | lyarwood: is that memory the guest memory or host memoery | |
| 13:49:07 | lyarwood | sean-k-mooney: guest AFAICT | |