| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2022-08-31 | |||
| 14:32:32 | sean-k-mooney | dansmith: did you say this just started failing recently? | |
| 14:32:42 | sean-k-mooney | because we merged unshelve to host a while ago | |
| 14:32:43 | dansmith | sean-k-mooney: it just recently merged I think gmann said | |
| 14:32:50 | sean-k-mooney | the tempest test? | |
| 14:33:16 | sean-k-mooney | if so i guess we dont run the multi-cell job on tempest | |
| 14:33:44 | gmann | dansmith: sean-k-mooney gibi that passed in tempest-multinode-full-py3 job and I do not think we checked multi-cell job run for this | |
| 14:33:56 | dansmith | the base nova conf doesn't have a db connection pointing at cell0, which I assume is what the apis are using | |
| 14:34:02 | gmann | yeah we do not run multi-cell job there and only checked tempest-multinode-full-py3 job passing it | |
| 14:34:22 | sean-k-mooney | gmann: ack that makes sense | |
| 14:34:32 | gibi | yeah nova-multicell did not run on the tempest patch | |
| 14:34:41 | sean-k-mooney | ok so we have two thing one we should skip this temporaly on the multi-cell-job | |
| 14:34:44 | sean-k-mooney | and then fix the bug | |
| 14:34:56 | gibi | yes | |
| 14:34:58 | dansmith | a normal run should have the api pointing at cell0: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8cf/831219/22/check/tempest-integrated-compute/8cfb267/controller/logs/etc/nova/nova_conf.txt | |
| 14:35:03 | dansmith | but that multicell run has nothing defined there | |
| 14:35:10 | sean-k-mooney | we at least shoudl check we are not unshelving across cells for now but that actully should work | |
| 14:35:18 | sean-k-mooney | since we use shelve for | |
| 14:35:21 | sean-k-mooney | cross cell resize | |
| 14:35:28 | dansmith | sean-k-mooney: it shouldn't work | |
| 14:35:38 | dansmith | because cross-cell migration does other stuff on top of the shelve | |
| 14:35:53 | sean-k-mooney | oh your right | |
| 14:36:22 | gmann | and cross-cell is not in that feature acope right? which is added in this cycle only. | |
| 14:36:24 | gmann | scope | |
| 14:36:33 | gmann | sean-k-mooney: agree to skip it in multi-cell job | |
| 14:36:38 | sean-k-mooney | it was not explcitly no | |
| 14:36:56 | sean-k-mooney | so we can just document the limitation i guess | |
| 14:37:05 | gmann | but will be good to add in documnt/releasenotes | |
| 14:37:05 | gmann | yes | |
| 14:38:07 | sean-k-mooney | dansmith: ohter then usign multiple port binding to test if the destination cell can bind the port. how much more does cross cell migration do over shelve. there is a bunch of stuff in the supper conductor to copy the instnace object between the cell db right | |
| 14:38:33 | dansmith | yeah it moves the thing between DBs, that's the big thing | |
| 14:39:13 | sean-k-mooney | well for now i guess its fine the error could be improved but the api wont let you currpt the db or anything like that | |
| 14:39:27 | dansmith | so this multicell job has two cells and two computes, one compute per cell? it must be disabling live migration and lots of other stuff that would normally require multinode right/ | |
| 14:40:04 | sean-k-mooney | ya its two nodes (contoller and compute) and both nodes are in a differnt cell | |
| 14:40:19 | dansmith | so it must disable a bunch of things | |
| 14:40:31 | sean-k-mooney | yep | |
| 14:41:20 | sean-k-mooney | https://github.com/openstack/nova/blob/master/.zuul.yaml#L514-L571 | |
| 14:41:21 | gmann | yeah, live migration is disabled https://github.com/openstack/nova/blob/master/.zuul.yaml#L554 | |
| 14:41:26 | sean-k-mooney | thats the deffintion | |
| 14:42:36 | dansmith | okay | |
| 14:42:51 | gibi | file the bug https://bugs.launchpad.net/nova/+bug/1988316 | |
| 14:43:05 | gibi | as it is blocks the gate I will set it to Critical | |
| 14:43:17 | dansmith | also, the api config I was looking at was on the compute, the controller is pointed at cell0 by default | |
| 14:43:25 | dansmith | gibi: ++ | |
| 14:43:28 | sean-k-mooney | gmann: does the tempest test have a compute_feature_flag or will i just add it to the exclude regex | |
| 14:43:34 | gibi | and I will push the skip | |
| 14:43:38 | gibi | to unblock the gate | |
| 14:43:45 | sean-k-mooney | oh ok | |
| 14:43:52 | sean-k-mooney | ill leave it to you so | |
| 14:44:11 | gmann | sean-k-mooney: exclude the test | |
| 14:44:38 | gmann | gibi: ok, I was doing but please you go ahead | |
| 14:45:15 | gibi | ack, I'm on it | |
| 14:45:17 | dansmith | sean-k-mooney and I can get our +2 hammers polished up | |
| 14:45:17 | gmann | gibi: we can add test class name in case more test are added for this https://github.com/openstack/nova/blob/master/devstack/nova-multi-cell-exclude-list.txt | |
| 14:45:55 | sean-k-mooney | ya UnshelveToHostMultiNodesTest | |
| 14:46:09 | sean-k-mooney | or tempest.api.compute.admin.test_servers_on_multinodes.UnshelveToHostMultiNodesTest | |
| 14:46:20 | sean-k-mooney | if we want to fully quallify it | |
| 14:48:20 | gibi | dansmith: would the current code fail to found the compute even if it is the same cell of the instance as it does not target the cell? | |
| 14:48:41 | dansmith | gibi: I think it has to be targeting the cell | |
| 14:48:47 | sean-k-mooney | it seamed to work for the host it started on | |
| 14:48:50 | gibi | I don't see where it targets it | |
| 14:48:54 | dansmith | I know it's not clear but it wouldn't be getting as far as it is if not | |
| 14:49:20 | sean-k-mooney | its presumably targeting the source cell and not finding the dest host | |
| 14:49:42 | dansmith | I confirmed it's using cell0 by default, which means it must be getting targeted | |
| 14:50:01 | sean-k-mooney | oh ok | |
| 14:52:19 | opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (db) https://review.opendev.org/c/openstack/nova/+/831193 | |
| 14:52:20 | opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (objects) https://review.opendev.org/c/openstack/nova/+/839401 | |
| 14:52:20 | opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (manila abstraction) https://review.opendev.org/c/openstack/nova/+/831194 | |
| 14:52:21 | opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (drivers and compute manager part) https://review.opendev.org/c/openstack/nova/+/833090 | |
| 14:52:21 | opendevreview | ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (api) https://review.opendev.org/c/openstack/nova/+/836830 | |
| 14:52:22 | opendevreview | ribaudr proposed openstack/nova master: Bump compute version and check shares support https://review.opendev.org/c/openstack/nova/+/850499 | |
| 14:52:22 | opendevreview | ribaudr proposed openstack/nova master: Add metadata for shares https://review.opendev.org/c/openstack/nova/+/850500 | |
| 14:52:23 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_attach notification https://review.opendev.org/c/openstack/nova/+/850501 | |
| 14:52:24 | opendevreview | ribaudr proposed openstack/nova master: Add instance.share_detach notification https://review.opendev.org/c/openstack/nova/+/851028 | |
| 14:52:24 | opendevreview | ribaudr proposed openstack/nova master: Add shares to InstancePayload https://review.opendev.org/c/openstack/nova/+/851029 | |
| 14:52:26 | opendevreview | ribaudr proposed openstack/nova master: Add instance.power_on_error notification https://review.opendev.org/c/openstack/nova/+/852084 | |
| 14:52:26 | opendevreview | ribaudr proposed openstack/nova master: Add instance.power_off_error notification https://review.opendev.org/c/openstack/nova/+/852278 | |
| 14:52:28 | opendevreview | ribaudr proposed openstack/nova master: Add helper methods to attach/detach shares https://review.opendev.org/c/openstack/nova/+/852085 | |
| 14:52:28 | opendevreview | ribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working. https://review.opendev.org/c/openstack/nova/+/852086 | |
| 14:52:30 | opendevreview | ribaudr proposed openstack/nova master: Add virt/libvirt error test cases https://review.opendev.org/c/openstack/nova/+/852087 | |
| 14:52:30 | opendevreview | ribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/854823 | |
| 14:52:32 | opendevreview | ribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/854824 | |
| 14:52:32 | opendevreview | ribaudr proposed openstack/nova master: Change microversion to 2.93 https://review.opendev.org/c/openstack/nova/+/852088 | |
| 14:52:39 | opendevreview | Balazs Gibizer proposed openstack/nova master: Skip UnshelveToHostMultiNodesTest in nova-multi-cell https://review.opendev.org/c/openstack/nova/+/855378 | |
| 14:52:56 | gibi | gmann, dansmith, sean-k-mooney: here is the skip ^^ | |
| 14:54:06 | sean-k-mooney | do we still use that file | |
| 14:54:39 | gibi | dansmith: in the rest api code we load the instance properly from its cell could that codepath alter the context too? | |
| 14:54:40 | sean-k-mooney | oh we do | |
| 14:54:58 | dansmith | no, because we're doing scatter | |
| 14:54:59 | gmann | yes | |
| 14:55:00 | dansmith | so actually | |
| 14:55:09 | dansmith | I think gibi might be right and that the feature itself is broken | |
| 14:55:27 | dansmith | but I can't explain why it's working on single-cell, since we're targeted to cell0 by default | |
| 14:55:35 | dansmith | it looks like we don't target until we hit superconductor | |
| 14:55:48 | dansmith | https://github.com/openstack/nova/blob/cbc9b516fb5e6b079b57cc40380e1d730751cd6c/nova/conductor/manager.py#L974-L975 | |
| 14:56:22 | dansmith | so I can't explain why this ever works, because the api should have only checked for the host being present in cell0, which should never have any hosts | |
| 14:56:33 | gibi | I think I see where we target the context | |
| 14:56:34 | gibi | _get_instance_from_cell | |
| 14:56:37 | gibi | https://github.com/openstack/nova/blob/3862cfc649d0099971c91d17e6f8800d10712a26/nova/compute/api.py#L2867-L2870 | |
| 14:56:44 | gibi | this actually has a side effect to alter the context | |
| 14:57:15 | dansmith | ah, yep, okay I missed that | |