| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2023-04-11 | |||
| 19:41:19 | sean-k-mooney | we should have merged that | |
| 19:41:23 | sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/863910/1 | |
| 19:41:29 | sean-k-mooney | and that in Antielope | |
| 19:41:36 | sean-k-mooney | i knew stephen had patches for this | |
| 19:42:12 | sean-k-mooney | bauzas: ^ we missed inclucing those | |
| 19:42:46 | sean-k-mooney | gmann: i think we should consider backporting those to stable/antelope | |
| 19:42:54 | gmann | i see | |
| 19:43:48 | gmann | sean-k-mooney: this 'not tested' is good to backport but for HyperV case we need to mark deprecated instead of experimental | |
| 19:44:45 | sean-k-mooney | i would prefer to do that as a third patch just to keep that discussion sperate form addign the warning | |
| 19:45:17 | sean-k-mooney | gmann: from a release note point of view both of these we going to be deprecations | |
| 19:45:32 | gmann | sure, that works fine. and we can backport this 'note tested' warning to reflect the actual things | |
| 19:45:43 | sean-k-mooney | we just didnt want to use that trem in the warning | |
| 19:46:00 | sean-k-mooney | basically we wanted to have the ablity to remove in b or C if needed | |
| #openstack-nova - 2023-04-12 | |||
| 06:48:11 | opendevreview | Sam Morrison proposed openstack/nova master: Filter out deleted instances when looking for build timouts https://review.opendev.org/c/openstack/nova/+/880125 | |
| 08:17:20 | bauzas | dansmith: gmann: sean-k-mooney: I added my thoughts on the deprecation patches but I'm afraid of the fact a backport wouldn't help operators to notice | |
| 08:48:13 | opendevreview | Jorge San Emeterio proposed openstack/nova master: WIP: Testing whether tests on bug#1998148 still fail. https://review.opendev.org/c/openstack/nova/+/880135 | |
| 09:05:44 | Uggla | bauzas , hi, if you have some time, can you have a look at the virtiofs series and especially the interfaces ? | |
| 09:06:10 | bauzas | sure I've seen your updates | |
| 09:06:30 | bauzas | but I need a jar of coffee first | |
| 09:34:25 | sean-k-mooney | Uggla: have you moved the libvirt driver changes for creating the xml config to the start fo the series | |
| 09:34:52 | sean-k-mooney | that should be the first set of patches in the serise so that can be merged to allow scaphanda work to start in parallel | |
| 09:35:08 | sean-k-mooney | while we wait for manialla to add the lock api | |
| 09:36:32 | manuvakery1 | nova.boot_servers: 18.803 sec (Avg) | |
| 09:36:32 | manuvakery1 | make the server count to 5 added a huge difference in delete server | |
| 09:36:32 | manuvakery1 | nova.delete_servers: 14.675 sec (Avg) | |
| 09:36:32 | manuvakery1 | nova.boot_servers : 11.941 sec (Avg) | |
| 09:36:32 | manuvakery1 | Hi, I have a 2 node cluster and I am running a rally scenario against it to create and delete servers. When i set the server count to 2 I see the response time as follows | |
| 09:36:33 | manuvakery1 | is this expected? | |
| 09:36:33 | manuvakery1 | nova.delete_servers: 42.988 sec (Avg) | |
| 09:37:14 | manuvakery1 | the compute nodes are not under any load | |
| 09:37:49 | Uggla | sean-k-mooney, hum, not really. | |
| 09:40:45 | sean-k-mooney | ok im not sure when we disucssed that | |
| 09:41:05 | kashyap | bauzas: Do we have a conclusion on this? -- https://review.opendev.org/c/openstack/nova/+/879021 | |
| 09:41:28 | sean-k-mooney | but if we want to merge any code before the manial api is implmented the libvirt dirver chage for the xml generation is the lowest risk | |
| 09:42:09 | bauzas | sean-k-mooney: I haven't went more than the 4th change, but the Manila API should be the last change in the series | |
| 09:42:10 | noonedeadpunk | hey folks! We're reviewing our nova role, as we haven't touched in for a while, and now a bit /o\ about logic we have there. so would be glad if you can share your prespective on some things :) | |
| 09:42:34 | sean-k-mooney | bauzas: yes the api should be last and the xml change sin the drive rhsould be first | |
| 09:42:51 | sean-k-mooney | driver then db then rpc finaly api | |
| 09:42:55 | bauzas | sean-k-mooney: for the libvirt driver, meh to me, since I think we should be quite okay for the 4 first patches (DB and objects) | |
| 09:43:14 | bauzas | noonedeadpunk: shoot | |
| 09:43:16 | noonedeadpunk | So first question - when `discover_hosts_in_cells_interval` is set to -1, I assume you need to run `nova-manage cell_v2 discover_hosts` before compute will appear in `openstack compute service list`? | |
| 09:43:42 | noonedeadpunk | As I can recall that compute itself does report back on startup as well | |
| 09:43:47 | sean-k-mooney | noonedeadpunk: no they will show up in the list | |
| 09:43:54 | noonedeadpunk | and this option is for scheduler | |
| 09:43:54 | sean-k-mooney | but they wont be in the cell mappings | |
| 09:43:58 | bauzas | noonedeadpunk: no, but they won't be scheduled | |
| 09:44:09 | bauzas | since they won't have a cell mapping | |
| 09:44:16 | bauzas | haha | |
| 09:44:16 | noonedeadpunk | aha, gotcha | |
| 09:44:47 | sean-k-mooney | discover_hosts tell use which rabbit to use to talk to the compute and what cell db its in | |
| 09:44:52 | noonedeadpunk | ok, then we have this set correctly :D | |
| 09:45:14 | noonedeadpunk | but it discovers... through API? | |
| 09:45:21 | noonedeadpunk | not through conductor then? | |
| 09:45:21 | sean-k-mooney | no | |
| 09:45:28 | noonedeadpunk | ah, right | |
| 09:45:29 | sean-k-mooney | it discovers by connecting to all the cell dbs | |
| 09:45:34 | noonedeadpunk | ok-ok, yes, I recalled :) | |
| 09:46:38 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L1043 | |
| 09:46:40 | sean-k-mooney | calls https://github.com/openstack/nova/blob/ae42400b7663bc58d5562de99e976c95131b77a9/nova/objects/host_mapping.py#L247-L285 | |
| 09:47:02 | noonedeadpunk | and in order for compute to be discovered by discover_hosts it must already be in service list? | |
| 09:47:12 | opendevreview | Merged openstack/placement master: Db: Drop redundant indexes for columns already having unique constraint https://review.opendev.org/c/openstack/placement/+/856770 | |
| 09:47:14 | sean-k-mooney | correct | |
| 09:47:26 | sean-k-mooney | so the compute agent connects to the conducotr on startup | |
| 09:47:47 | sean-k-mooney | and then it ask the conductor to register a service in the db | |
| 09:48:01 | sean-k-mooney | "the cell db" | |
| 09:48:23 | sean-k-mooney | the conducotor may or may not have access to the api db | |
| 09:48:32 | sean-k-mooney | so we cant assume it can add the cell mapping | |
| 09:50:11 | noonedeadpunk | yeah, ok, thanks for the explanation | |
| 09:50:33 | noonedeadpunk | As I was mixing up discovery flow | |
| 09:50:45 | bauzas | noonedeadpunk: maybe this would help you :p https://docs.openstack.org/nova/latest/admin/cells.html#faqs | |
| 09:51:31 | bauzas | but yeah, discover_hosts is idempotent so you can call it anytime you want | |
| 09:52:14 | bauzas | this is explained at the end of https://docs.openstack.org/nova/latest/admin/cells.html#configuring-a-new-deployment | |
| 09:52:38 | noonedeadpunk | well, I was unsure about logic, as I could not fully understand the reason why we're waiting for compute to be present in service list when nova_discover_hosts_in_cells_interval=-1 but run cell_v2 discover_hosts only after that | |
| 09:53:05 | bauzas | noonedeadpunk: we have two different databases now as a reminder | |
| 09:53:17 | bauzas | so we need to make sure we synchronize them | |
| 09:53:24 | noonedeadpunk | As for some reason I thought that discover_hosts also will add to service list, but now I know it;s not how it works) | |
| 09:54:30 | sean-k-mooney | bauzas: technially we have 3. we alway have api + cell 0 and cell one in any real deployment :) | |
| 09:57:21 | bauzas | sean-k-mooney: we have two different table schemas, but yeah :) | |
| 09:57:36 | bauzas | cell0 is logically identically to any cell | |
| 09:57:40 | bauzas | identical* | |
| 09:57:54 | bauzas | only the usage of this cell is different | |
| 09:58:05 | bauzas | but we're bikeshedding :) | |
| 09:59:58 | noonedeadpunk | Another thing I am unsure about - db online_data_migrations and nova-status upgrade check and when they can/should be run. | |
| 10:00:12 | noonedeadpunk | As far as I got - db online_data_migrations should be done only during upgrades | |
| 10:00:34 | noonedeadpunk | But is it recommended to run `nova-status upgrade check` before that? | |
| 10:01:26 | noonedeadpunk | As it feels like upgrade check should be run once all computes are upgraded, while online_data_migrations requires only conductor to be done? | |
| 10:05:16 | bauzas | noonedeadpunk: this is correct | |
| 10:05:44 | bauzas | you should run nova-status upgrade check *before* the db upgrade as a pre-flight check | |
| 10:06:24 | bauzas | nova-status is a controller script, checking a few APIs and DBs | |
| 10:06:33 | bauzas | to see whether you're safe to upgrade | |
| 10:07:02 | bauzas | like, say we introduced a thing but wasn't the default | |
| 10:07:27 | bauzas | before we flip the default and remove the old bits, we propose you to check that you opted in before | |
| 10:07:45 | noonedeadpunk | aha, yes, exactly what I was unsure about. As currently we run upgrade check basically after cell_v2 discover_hosts at the very end | |
| 10:08:18 | noonedeadpunk | which didn't make much sense to me | |
| 10:08:43 | noonedeadpunk | is there any way to tell if online_data_migrations are needed at all? Like by some exit code of smth? | |
| 10:09:12 | bauzas | noonedeadpunk: see the upgrade docs, this explains it way better than me now https://docs.openstack.org/nova/latest/admin/upgrades.html#rolling-upgrade-process | |
| 10:10:06 | bauzas | noonedeadpunk: wait, I said something wrong | |
| 10:10:29 | bauzas | noonedeadpunk: https://docs.openstack.org/nova/latest/cli/nova-status.html#upgrade | |
| 10:11:25 | bauzas | you need to do DB schema expands/contracts before with the migrations | |
| 10:11:38 | bauzas | then, you run the status check | |