| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-08-09 | |||
| 13:06:17 | lyarwood | ack will do | |
| 13:54:13 | gmann | stephenfin: ack, will check and comment, thanks | |
| 14:04:18 | lyarwood | stephenfin: https://zuul.opendev.org/t/openstack/build/6cedd9d4547c4f608553ace4701d4709 - did you see that docs failure on your https://review.opendev.org/c/openstack/nova/+/799523/ change btw? | |
| 14:21:37 | stephenfin | lyarwood: No, I did not | |
| 14:32:04 | stephenfin | lyarwood: Sigh, that failed 8 times in a row. I missed it because there were quite a few random failures elsewhere and I got lazy and stopped checking the logs :-( | |
| 14:32:18 | stephenfin | lyarwood: The issue is that this change should not be here but rather in the next patch in the series https://review.opendev.org/c/openstack/nova/+/799523/1/etc/nova/nova-config-generator.conf | |
| 14:33:00 | stephenfin | afraid so | |
| 14:33:07 | lyarwood | RIP the gate | |
| 14:33:21 | lyarwood | but cool at least it's fixed then | |
| 14:33:22 | stephenfin | Much maligned but ultimately missed | |
| 14:34:02 | lyarwood | the series looks good overall btw, I went through to the end but wanted to look at a few things in more detail before voting either way | |
| 14:34:30 | lyarwood | mostly with the automatic migrations at the end tbh | |
| 14:35:23 | stephenfin | anything else obvious you want fixed before I push this rebase? | |
| 14:35:40 | lyarwood | nothing at the moment no | |
| 14:35:42 | stephenfin | I dragged the follow-up into the series after the patch it fixed | |
| 14:35:47 | lyarwood | cool | |
| 14:38:06 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Unify 'nova.db.api', 'nova.db.sqlalchemy.api' https://review.opendev.org/c/openstack/nova/+/799524 | |
| 14:38:06 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Register database config options ourselves https://review.opendev.org/c/openstack/nova/+/799523 | |
| 14:38:07 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Move remaining 'nova.db.sqlalchemy' modules https://review.opendev.org/c/openstack/nova/+/799525 | |
| 14:38:07 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Exclude the 'use_db_reconnect' option https://review.opendev.org/c/openstack/nova/+/803778 | |
| 14:38:08 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Avoid use of ALTER in initial migration https://review.opendev.org/c/openstack/nova/+/800076 | |
| 14:38:08 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Post reshuffle cleanup https://review.opendev.org/c/openstack/nova/+/799526 | |
| 14:38:09 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Add initial alembic migration for API DB https://review.opendev.org/c/openstack/nova/+/799528 | |
| 14:38:09 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Add initial alembic migration for main DB https://review.opendev.org/c/openstack/nova/+/799527 | |
| 14:38:10 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Trivial style changes https://review.opendev.org/c/openstack/nova/+/799529 | |
| 14:38:11 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Integrate alembic https://review.opendev.org/c/openstack/nova/+/799530 | |
| 14:38:11 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Normalize migrations tests https://review.opendev.org/c/openstack/nova/+/799684 | |
| 14:38:13 | opendevreview | Stephen Finucane proposed openstack/nova master: docs: Add documentation on database migrations https://review.opendev.org/c/openstack/nova/+/800078 | |
| 14:38:13 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Enable auto-generation of migrations https://review.opendev.org/c/openstack/nova/+/800077 | |
| 14:38:15 | opendevreview | Stephen Finucane proposed openstack/nova master: db: Final cleanups https://review.opendev.org/c/openstack/nova/+/800484 | |
| 14:39:41 | stephenfin | okay, https://review.opendev.org/c/openstack/nova/+/799523/ now excludes the one line change to etc/nova/nova-config-generator.conf. https://review.opendev.org/c/openstack/nova/+/799524 is the opposite, in that it now includes this. Both otherwise unchanged | |
| 15:04:50 | opendevreview | Merged openstack/nova stable/rocky: [neutron] Get only ID and name of the SGs from Neutron https://review.opendev.org/c/openstack/nova/+/791420 | |
| 15:05:36 | opendevreview | Slawek Kaplonski proposed openstack/nova stable/queens: [neutron] Get only ID and name of the SGs from Neutron https://review.opendev.org/c/openstack/nova/+/791421 | |
| 16:55:22 | lyarwood | kashyap: https://zuul.opendev.org/t/openstack/build/3c4856a7e156479e8b7bda3135389b64/log/controller/logs/screen-n-cpu.txt?severity=4 - one for tomorrow but I'm seeing `qemu unexpectedly closed the monitor: Could not allocate dynamic translator buffer` errors in the CentOS job, grepping around I think this is SELinux related but could use a hand tracking it down. | |
| 18:19:44 | opendevreview | Merged openstack/nova master: db: Register database config options ourselves https://review.opendev.org/c/openstack/nova/+/799523 | |
| #openstack-nova - 2021-08-10 | |||
| 00:03:00 | opendevreview | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from Rebuild Server API https://review.opendev.org/c/openstack/nova/+/766380 | |
| 00:04:07 | opendevreview | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List SG API https://review.opendev.org/c/openstack/nova/+/766726 | |
| 00:52:58 | opendevreview | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from Flavor Access APIs https://review.opendev.org/c/openstack/nova/+/767704 | |
| 01:08:17 | opendevreview | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List/Show usage APIs https://review.opendev.org/c/openstack/nova/+/768509 | |
| 02:02:26 | opendevreview | Brin Zhang proposed openstack/nova master: Replace tenants* with projects* of policies https://review.opendev.org/c/openstack/nova/+/765315 | |
| 02:02:27 | opendevreview | Brin Zhang proposed openstack/nova master: Replace tenant_id with project_id in os-quota-sets path https://review.opendev.org/c/openstack/nova/+/768851 | |
| 02:02:27 | opendevreview | Brin Zhang proposed openstack/nova master: Replace os-simple-tenant-usage with os-simple-project-usage https://review.opendev.org/c/openstack/nova/+/768852 | |
| 02:02:28 | opendevreview | Brin Zhang proposed openstack/nova master: Replace tenant* with project* in codes https://review.opendev.org/c/openstack/nova/+/769329 | |
| 02:02:28 | opendevreview | Brin Zhang proposed openstack/nova master: Replace tenant_id with project_id in Limits API https://review.opendev.org/c/openstack/nova/+/768862 | |
| 02:09:16 | opendevreview | Takashi Kajinami proposed openstack/nova master: tests: Add test for bug #1936278 https://review.opendev.org/c/openstack/nova/+/802893 | |
| 07:01:29 | opendevreview | Slawek Kaplonski proposed openstack/nova stable/queens: [neutron] Get only ID and name of the SGs from Neutron https://review.opendev.org/c/openstack/nova/+/791421 | |
| 08:36:08 | kashyap | lyarwood: Looking | |
| 08:54:08 | kashyap | lyarwood: Okay, I see that error was already discussed upstream QEMU by Debian folks last year ... looking | |
| 09:05:47 | kashyap | lyarwood: Okay, talking to upstream QEMU maint: | |
| 09:06:38 | kashyap | - That terrible error is thrown becuase QEMU is unable execute code from the memory QEMU is executing from — IOW, it requires the allocated chunk of memory to be executable. | |
| 09:07:36 | kashyap | - Peter (from QEMU) had three possible guesses off-hand: | |
| 09:07:47 | kashyap | (a) host OS is enforcing no-rwx like OpenBSD can, or (b) if the user's set a very aggressive 'ulimit'on virtual memory usage, or (c) perhaps a system with no swap and not enough RAM either | |
| 09:11:33 | kashyap | - From QEMU 6.0 or higher, they fixed that usless error to report a better error including the error code: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=6bc144237a857b | |
| 09:11:51 | kashyap | So, our possible options are: | |
| 09:12:38 | kashyap | (1) Check if this is repetable at all -- and rule out 'ulimit' and "not enough RAM" options | |
| 09:12:58 | kashyap | (2) Try QEMU 6.0 or higher, to identify the precise error - that may give better info to the QEMU folks to debug further | |
| 09:13:57 | kashyap | (3) In case we have access to the host, get `strace` output, which should tell us where the mmap() call is failing. | |
| 09:15:30 | kashyap | I don't think (2) and (3) are not possible because our hands are tied, and no access to the machine? :-( | |
| 09:16:32 | kashyap | Oh, correction: Peter tells me that the 'strace' will give the same info as trying with QEMU 6.0 or higher. So we can try whatever is eaiser: option two or (2) or (3) from the above. | |
| 09:19:50 | kashyap | s/two or// | |
| 09:27:05 | lyarwood | kashyap: kk, we don't have virt-preview for centos stream so I'm not sure what we can do tbh | |
| 09:27:20 | lyarwood | oh wait we do | |
| 09:27:26 | lyarwood | TIL | |
| 09:27:35 | lyarwood | cool I'll enable that now | |
| 09:27:40 | kashyap | lyarwood: Yeah, CentOS does have it, last I checked | |
| 09:27:49 | kashyap | lyarwood: Via a CI job? | |
| 09:27:59 | lyarwood | yeah | |
| 09:30:23 | kashyap | lyarwood: Ah, cool. In case this is not resolved by tomm afternoon (I'll be on PTO from day after), you can follow-up with 'pm215' (Peter Maydell) on #qemu, OFTC. He helped the initial diagnosis from QEMU | |
| 09:30:47 | kashyap | (If he's not around, then 'davidgiluk') | |
| 09:31:13 | kashyap | s/helped the/helped with the/ | |
| 09:33:29 | lyarwood | kashyap: ack thanks | |
| 09:33:34 | lyarwood | kashyap: https://review.opendev.org/c/openstack/tempest/+/804055 will enable the repo | |
| 09:58:45 | kashyap | lyarwood: Excellent; thank you! Do you know what version of QEMU it'll bring? | |
| 09:59:32 | lyarwood | kashyap: 6.0.0-12 appears to be in the repo | |
| 09:59:43 | kashyap | lyarwood: Ah, was just looking at the repo | |
| 10:00:05 | lyarwood | https://copr.fedorainfracloud.org/coprs/g/virtmaint-sig/virt-preview/build/2353031/ | |
| 10:01:08 | kashyap | lyarwood: Isn't that -12 for fc34? - 6.0.0-12.fc34 - maybe it's the same for CentOS 8 too | |
| 10:01:24 | kashyap | Yes, it is - https://download.copr.fedorainfracloud.org/results/@virtmaint-sig/virt-preview/centos-stream-8-x86_64/02353031-qemu/ | |
| 10:01:26 | lyarwood | yeah I think it's cross built | |
| 10:01:34 | kashyap | Yep, you're right | |
| 10:02:24 | kashyap | lyarwood: BTW, I double-checked that version of QEMU has the commit we want: | |
| 10:02:27 | kashyap | $> git tag --contains 6bc144237a857b | head -1 | |
| 10:02:30 | kashyap | v6.0.0 | |
| 10:07:10 | kashyap | Gave my paltry ACK on the Tempest change, with a note | |
| 10:37:06 | sean-k-mooney1 | gmann: gibi_pto ye might find https://review.opendev.org/c/openstack/keystonemiddleware/+/787822 interesting its the implementation of X-Project-Id passthru that allows you to use project ids with system scoped tokens | |
| 11:11:38 | opendevreview | Lee Yarwood proposed openstack/nova master: compute: Query the service group API within check_instance_host https://review.opendev.org/c/openstack/nova/+/802317 | |
| 14:38:07 | artom | Am I high? Why am I seeing api_models.py in Newton? I thought cells v2 was way after that... | |
| 14:43:07 | artom | I guess we'd already started by then... | |
| 14:44:25 | sean-k-mooney | i dont think that was new to cells v2 | |
| 14:45:03 | sean-k-mooney | even with v1 i think we had a seperate top level db that we then had sync tasks runing btween the cells | |
| 14:45:41 | sean-k-mooney | i try to block out cell_v1's existance form my mind as much as i can | |
| 14:57:45 | stephenfin | It was new to cells v2 | |
| 14:58:03 | stephenfin | we only turned it on in Pike, but there was ground work way before that | |
| 15:58:39 | gmann | sean-k-mooney: ack, thanks | |
| 18:49:48 | b1tsh1ft3r | Hey all, had a compute node hardware failure. Ive removed the compute node and re-added according to the docs here: https://docs.openstack.org/openstack-ansible/train/admin/scale-environment.html#add-compute-host | |
| 18:50:35 | b1tsh1ft3r | i'm having issues however being able to live migrate instances to the compute node. I'm not seeing any real errors in logs or the UI. It looks as if it works, but doesnt. | |
| 18:54:27 | sean-k-mooney | i would check that ssh is workign between the compute nodes and that the host name resolve correctly | |
| 18:55:02 | sean-k-mooney | you need passworad less ssh to work as the use nova is running as and yuou will need to have exchanged keys | |
| 18:55:41 | sean-k-mooney | assuming you are getting that far | |
| 18:56:24 | sean-k-mooney | b1tsh1ft3r: in general thougfh you might be better of asking the osa developers | |