| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-07-07 | |||
| 16:02:25 | opendevreview | Balazs Gibizer proposed openstack/nova master: Transfer RequestLevelParams from ports to scheduling https://review.opendev.org/c/openstack/nova/+/791506 | |
| 16:03:47 | opendevreview | Balazs Gibizer proposed openstack/nova master: [func test] ports with both bw and pps resources https://review.opendev.org/c/openstack/nova/+/792394 | |
| 16:05:15 | opendevreview | Balazs Gibizer proposed openstack/nova master: [func test] move unshelve test to the proper place https://review.opendev.org/c/openstack/nova/+/793621 | |
| 16:15:10 | opendevreview | Lee Yarwood proposed openstack/nova-specs master: Add nova-manage commands to show and refresh connection_info https://review.opendev.org/c/openstack/nova-specs/+/799624 | |
| 16:45:42 | stephenfin | zzzeek: If you've any spare time whatsoever this week, I'd appreciate your attention on an issue I'm having during the sqla-migrate -> alembic switch. Basically, my migration code to create shadow tables doesn't seem to be working with in-memory SQLite databases, but I can't figure out how to fix it | |
| 16:45:58 | stephenfin | zzzeek: I have a pretty minimal reproducer here https://github.com/stephenfin/alembic-sqlite-memory-issue | |
| 16:46:28 | stephenfin | zzzeek: and the series to do the conversion for nova ends here, fwiw https://review.opendev.org/c/openstack/nova/+/799530/ | |
| 16:47:16 | stephenfin | zzzeek: I _think_ I'm seeing an issue like the one in the cookbook, but I don't understand the issue well enough yet to say /o\ Any input would be much appreciated https://alembic.sqlalchemy.org/en/latest/cookbook.html#sharing-a-connection-with-a-series-of-migration-commands-and-environments | |
| 17:06:55 | opendevreview | Lee Yarwood proposed openstack/nova-specs master: Store and allow libvirt instance device buses and models to be updated https://review.opendev.org/c/openstack/nova-specs/+/799811 | |
| 17:07:54 | lyarwood | stephenfin: hope you're happy, I pretty much always forget to manually run pep8 in projects outside of Nova now thanks to pre-commit ;) | |
| 17:20:09 | sean-k-mooney | hehe same | |
| 17:27:35 | opendevreview | Merged openstack/nova master: api: Handle invalid volume UUIDs during spawn https://review.opendev.org/c/openstack/nova/+/794006 | |
| 18:26:55 | zzzeek | stephenfin: the issue is a combination of using NullPool with create_engine() combined with the usage of a different connection here: https://github.com/stephenfin/alembic-sqlite-memory-issue/blob/main/foo/migrations/versions/8f2f1571d55b_initial_version.py#L98 | |
| 18:27:18 | zzzeek | stephenfin: when you're in a migration, there's a method op.get_bind(). that gives you the current Connection. use that everywhere for SQL, table reflection, etc | |
| 18:34:07 | zzzeek | stephenfin: I marked it up quite a bit to get you started :) | |
| 19:28:56 | rm_work | anyone seen issues with this error on starting up nova-compute (I'm on wallaby): `greenlet.error: cannot switch to a different thread` | |
| 19:29:01 | rm_work | full traceback: http://paste.openstack.org/show/807240/ | |
| 19:29:19 | sean-k-mooney | is that in the api | |
| 19:29:22 | rm_work | log just shows it trying to do heartbeats and then ... this, and the process locks up | |
| 19:29:28 | rm_work | nova-compute service | |
| 19:29:40 | sean-k-mooney | that should not happen in the compute service | |
| 19:30:02 | sean-k-mooney | we do use real threas but only to interact with a select number of things liek libvirt | |
| 19:30:18 | rm_work | hmm, no VMs even in this system yet, it's essentially a clean install | |
| 19:30:34 | rm_work | just trying to start it up for the first time | |
| 19:30:54 | sean-k-mooney | so i woudl assume this is somehow related to the perodic taks | |
| 19:31:00 | sean-k-mooney | *tasks | |
| 19:31:43 | sean-k-mooney | do you have anything custom set in you nova.conf | |
| 19:31:48 | rm_work | hmmm... what periodics does nova-compute run? can I disable them to check? | |
| 19:31:57 | sean-k-mooney | e.g. are you trying to use any atypical features | |
| 19:32:03 | sean-k-mooney | it runs quite a few | |
| 19:32:19 | sean-k-mooney | i think all or most of them have config options | |
| 19:32:23 | rm_work | It's hard for me to answer that question -- yes, the conf is probably custom, but I don't know enough about nova's features to know which are atypical | |
| 19:32:29 | rm_work | I've tried to trim it down as much as possible | |
| 19:33:12 | sean-k-mooney | i was wondering if you were tweakign any oslo concurancy setting related to threading for example | |
| 19:33:22 | sean-k-mooney | you coudl try setting https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.run_external_periodic_tasks to false | |
| 19:33:27 | rm_work | ah this is a nova-compute configured for ironic | |
| 19:33:42 | rm_work | ooo, will try that for testing | |
| 19:34:29 | rm_work | no config in oslo_concurrency besides setting the lock_path | |
| 19:34:55 | sean-k-mooney | oh its ironic hum | |
| 19:35:03 | sean-k-mooney | i dont think that shoudl really change things | |
| 19:35:06 | sean-k-mooney | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.sync_power_state_interval | |
| 19:35:24 | sean-k-mooney | there are several "*_interval" | |
| 19:35:34 | sean-k-mooney | config option which contol the perodic tasks | |
| 19:35:40 | rm_work | hmm, added that `run_external_periodic_tasks` to false, didn't help | |
| 19:35:44 | sean-k-mooney | do you have a more complete errror message | |
| 19:35:54 | sean-k-mooney | so we can see what it was doing right before the error | |
| 19:35:58 | rm_work | yeah secx | |
| 19:38:56 | rm_work | http://paste.openstack.org/show/807242/ | |
| 19:39:26 | sean-k-mooney | oh | |
| 19:39:34 | sean-k-mooney | its right after the heartbeat | |
| 19:39:37 | rm_work | yeah | |
| 19:39:39 | sean-k-mooney | one sec | |
| 19:40:05 | sean-k-mooney | try setting https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/impl_rabbit.py#L90-L100 | |
| 19:40:08 | sean-k-mooney | to false | |
| 19:40:16 | rm_work | kk | |
| 19:40:49 | sean-k-mooney | [oslo_messaging_rabbit]/heartbeat_in_pthread=false | |
| 19:40:56 | sean-k-mooney | in the nova.conf | |
| 19:41:12 | rm_work | well that's ... different | |
| 19:41:31 | sean-k-mooney | no crash? | |
| 19:41:34 | rm_work | oh derp sorry sec | |
| 19:41:38 | rm_work | it crashed but ... lol | |
| 19:41:44 | rm_work | fucking macbook keyboard | |
| 19:42:16 | sean-k-mooney | i recently got a ipad pro and have been trying to use it as an ssh clinet | |
| 19:42:22 | rm_work | yeah looks A++ | |
| 19:42:22 | sean-k-mooney | its ... interesting | |
| 19:42:35 | rm_work | seems to have connected up successfully, starts and shows up in a compute service list | |
| 19:42:36 | rm_work | thanks! | |
| 19:42:55 | rm_work | though, that opt is deprecated it seems ;D | |
| 19:43:02 | sean-k-mooney | well that option is deprecated for removal and planned to be remvoed this cycle | |
| 19:43:11 | sean-k-mooney | so we shoudl file a bug and get oslo invovled | |
| 19:43:16 | sean-k-mooney | to see if they can repoduce this | |
| 19:44:47 | sean-k-mooney | in theory you should not need to run the heatbeat in a real greenthread by the way | |
| 19:44:59 | sean-k-mooney | that was to workaround an issue in the api when service are run under uwsgi | |
| 19:45:11 | sean-k-mooney | since the wsgi server would kill the heatbeat | |
| 19:46:34 | rm_work | hmm | |
| 19:46:57 | opendevreview | melanie witt proposed openstack/nova master: Add functional regression test for bug 1853009 https://review.opendev.org/c/openstack/nova/+/695012 | |
| 19:46:57 | opendevreview | melanie witt proposed openstack/nova master: Clear rebalanced compute nodes from resource tracker https://review.opendev.org/c/openstack/nova/+/695187 | |
| 19:46:58 | opendevreview | melanie witt proposed openstack/nova master: Invalidate provider tree when compute node disappears https://review.opendev.org/c/openstack/nova/+/695188 | |
| 19:47:00 | opendevreview | melanie witt proposed openstack/nova master: Prevent deletion of a compute node belonging to another host https://review.opendev.org/c/openstack/nova/+/694802 | |
| 19:47:00 | opendevreview | melanie witt proposed openstack/nova master: Fix inactive session error in compute node creation https://review.opendev.org/c/openstack/nova/+/695189 | |
| 19:49:06 | sean-k-mooney | rm_work: you should be able to use that as a workaround for now | |
| 19:49:23 | sean-k-mooney | but i mentioned it in #openstack-oslo | |
| 19:49:26 | rm_work | yeah, will do. thanks for the help :) | |
| 19:50:27 | sean-k-mooney | rm_work: if you can file a bug or provide more detail e.g. ubutu20.04 vs centos8 that kind of thing it might help nail this down | |
| 19:51:03 | rm_work | kk | |
| 19:51:03 | sean-k-mooney | it should be filed against oslo.messaging but you can also add nova if you like althoguh this is not in the nova code | |
| 19:54:59 | rm_work | ah k | |
| 20:40:58 | rm_work | posted here: https://bugs.launchpad.net/oslo.messaging/+bug/1934937 | |
| 20:41:03 | rm_work | I am ... not great at launchpad | |
| 20:41:11 | rm_work | hopefully that's enough to start with | |
| 20:41:49 | rm_work | can add more details if someone tells me what would be good to add | |
| 21:21:26 | opendevreview | melanie witt proposed openstack/nova master: Add functional regression test for bug 1853009 https://review.opendev.org/c/openstack/nova/+/695012 | |
| 21:21:26 | opendevreview | melanie witt proposed openstack/nova master: Clear rebalanced compute nodes from resource tracker https://review.opendev.org/c/openstack/nova/+/695187 | |
| 21:21:27 | opendevreview | melanie witt proposed openstack/nova master: Invalidate provider tree when compute node disappears https://review.opendev.org/c/openstack/nova/+/695188 | |
| 21:21:28 | opendevreview | melanie witt proposed openstack/nova master: Prevent deletion of a compute node belonging to another host https://review.opendev.org/c/openstack/nova/+/694802 | |
| 21:21:29 | opendevreview | melanie witt proposed openstack/nova master: Fix inactive session error in compute node creation https://review.opendev.org/c/openstack/nova/+/695189 | |
| #openstack-nova - 2021-07-08 | |||
| 00:49:37 | opendevreview | melanie witt proposed openstack/nova master: Prevent deletion of a compute node belonging to another host https://review.opendev.org/c/openstack/nova/+/694802 | |
| 00:49:38 | opendevreview | melanie witt proposed openstack/nova master: Fix inactive session error in compute node creation https://review.opendev.org/c/openstack/nova/+/695189 | |
| 06:56:55 | opendevreview | ruishawn proposed openstack/nova master: Add armv6l to arch list https://review.opendev.org/c/openstack/nova/+/799964 | |
| 07:00:04 | opendevreview | ruishawn proposed openstack/nova master: Add armv6l to arch list https://review.opendev.org/c/openstack/nova/+/799964 | |