| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2025-02-18 | |||
| 18:31:17 | sean-k-mooney | we can jsut use project manager in the roles becuase when you have the admin role we do not need to compre the project id | |
| 18:32:08 | sean-k-mooney | so we do actully need to have PROJECT_MANAGER_OR_ADMIN for api that are valid to call form both | |
| 18:32:38 | sean-k-mooney | although on the tempest side i dotn think that changes thigns | |
| 18:34:00 | gmann | yeah | |
| 18:35:22 | priteau | Hello. Could a Nova core approve this backport please? https://review.opendev.org/c/openstack/nova/+/940846 I have just tested it and it resolves our baremetal node deletion issues. | |
| 18:36:35 | sean-k-mooney | ah the 2024.1 backport | |
| 18:36:37 | sean-k-mooney | sure | |
| 18:36:39 | gmann | sean-k-mooney: anyways let me see if I can finish the testing otherwise I am ok not to be in hurry in this cycle. The main reason is that I am struggling with my bandwidth for upstream activities at least for next few months also. | |
| 18:37:24 | sean-k-mooney | gmann: i finsihed reviewing the manger changes and i dont think you updated it with the dicussion form the spec | |
| 18:37:46 | sean-k-mooney | i was also expecting you to modify instnace lock btu maybe that was nto covered | |
| 18:37:55 | sean-k-mooney | i didnt look at the spec in a while | |
| 18:38:56 | gmann | In spec, we left the lock and other server action. I explicitly mentioned those as not-changing-those | |
| 18:40:20 | sean-k-mooney | i was expectign you to allow the manager ot unlock an admin locked insntace | |
| 18:40:31 | sean-k-mooney | but not restrict lock to manager | |
| 18:40:35 | sean-k-mooney | but that is fine | |
| 18:41:12 | sean-k-mooney | priteau: approved :) | |
| 19:16:04 | dansmith | bauzas: still around? | |
| 19:31:16 | opendevreview | Michael Still proposed openstack/nova master: libvirt: allow direct SPICE connections to qemu https://review.opendev.org/c/openstack/nova/+/924844 | |
| 19:42:19 | priteau | sean-k-mooney: thanks! | |
| 20:29:36 | MengyangZhang[m] | <sean-k-mooney> "cinder shoudl not apply the..." <- unfortunately this is the case now. How should I proceed then? | |
| 20:31:30 | sean-k-mooney | it depend | |
| 20:31:34 | opendevreview | Merged openstack/nova stable/2024.1: ironic: Fix ConflictException when deleting server https://review.opendev.org/c/openstack/nova/+/940846 | |
| 20:32:27 | sean-k-mooney | MengyangZhang[m]: if it allow the volume type ot be updated after its in use does it also propagte those changes to instance when a new attachment is created | |
| 20:32:42 | sean-k-mooney | or does it update the conetnt for exsiting attachments | |
| 20:33:33 | MengyangZhang[m] | to new attachment, existing volumes must be reattached to have the update applied | |
| 20:34:20 | sean-k-mooney | ack so that not great as i treally should only update it if you do a volume retype | |
| 20:34:41 | sean-k-mooney | but its less bad then otherwise | |
| 20:37:36 | sean-k-mooney | what we really need is an per volume api call to trigger the atachment update. the worst case sensiario is you create a new qos policy, you apply it to a volume time and cinder tirggers all volumesof that type ot be udpated | |
| 20:38:17 | sean-k-mooney | by that i mean | |
| 20:38:50 | sean-k-mooney | it would be a problem if cidner tirgger nova to call libvirt to update all fontend qos on all affected volumes | |
| 20:41:29 | sean-k-mooney | its tricky but how nova would expect cidner to work is that when a voluem is created iwth a given voluem type. the defintion fo that voluem type should be snapshot and saved per voloume and used for the lifetime of the volume | |
| 20:42:00 | sean-k-mooney | meaning the only way to update a voluem to get the new qos would be via a voluem retype ot the same or a diffent volume type | |
| 20:42:29 | sean-k-mooney | that is how we handel flavors and iamges | |
| 20:43:33 | sean-k-mooney | it sounds like cinder is not doing this | |
| 20:48:44 | MengyangZhang[m] | No, not for live migration/resize coz qos limits are saved in cinder.quality_of_service_specs db table and it is only assoicated with volume type. I don't see qos values are saved on a per volume basis. Thus, when volume attachment event happens, they just fetch qos limits associated with the volume type and popolate to the connection_info field, which then passed back to nova to generate xml | |
| 20:50:52 | MengyangZhang[m] | The reboot however works a bit different. I believe it is reading the connection_info from nova.block_device_mapping db, which saves a snapshot of volume qos and use it for the lifetime of the instance. | |
| 20:51:09 | sean-k-mooney | right so that probelmatic becasue it shoudl not change on a live migration | |
| 20:51:57 | sean-k-mooney | the reboot behavior in nova is how it shoudl work | |
| 20:52:36 | sean-k-mooney | the qos policy applied to an instnace volume shoudl not change as a sideffect changing the host | |
| 20:53:09 | MengyangZhang[m] | understood, then what we should do? discuss this in next cinder-nova meeting? | |
| 20:53:25 | sean-k-mooney | i think so yes | |
| 20:53:45 | sean-k-mooney | i do not know if modifying the QOS could cause the live migration to fail for example | |
| 20:53:57 | sean-k-mooney | or cause issues with reverting a cold migrate or resize | |
| 20:55:50 | MengyangZhang[m] | i don't think it would cause live-migration fail. We have deployed qos to our private cloud for some time and we don't see increased live-migration failures when existing VMs are adapting the qos | |
| 20:56:20 | sean-k-mooney | it would fail if the destination host did not supprot the qos config | |
| 20:57:10 | sean-k-mooney | i.e. if the souce and destination host had differnt version of libvirt and the qos element was supported on the source but nto the desitnaiton | |
| 20:57:50 | sean-k-mooney | we shoudl be validating that in pre live migration | |
| 20:58:47 | opendevreview | ribaudr proposed openstack/nova master: Augment the LiveMigrateData object https://review.opendev.org/c/openstack/nova/+/942143 | |
| 20:58:48 | opendevreview | ribaudr proposed openstack/nova master: Add live_migratable flag to PCI device specification https://review.opendev.org/c/openstack/nova/+/942144 | |
| 20:58:48 | opendevreview | ribaudr proposed openstack/nova master: Update manager to allow vfio pci device live migration https://review.opendev.org/c/openstack/nova/+/942145 | |
| 20:58:49 | opendevreview | ribaudr proposed openstack/nova master: Update conductor and filters allowing migration with SR-IOV devices https://review.opendev.org/c/openstack/nova/+/942146 | |
| 20:58:49 | opendevreview | ribaudr proposed openstack/nova master: WIP: Update driver to map the targeted address for SR-IOV PCI devices https://review.opendev.org/c/openstack/nova/+/942147 | |
| 20:58:58 | sean-k-mooney | downstream we supprot deploying a single verison of nova across both rhel 8 and rhel 9 host for an extened peoid of time | |
| 21:00:05 | sean-k-mooney | we have at least one custoemr that told use that for regaltory reasoson can only update eithe revery 6-12 months | |
| 21:00:46 | sean-k-mooney | so they would realistically ahve some deployments with a mix of libvirt versions for up to a year | |
| 21:01:31 | MengyangZhang[m] | I think we discussed this at some point, the pre live migration step wouldn't fail plus openstack supported front-side disk qos 13 years ago | |
| 21:02:04 | sean-k-mooney | right for the current set we have it wont fail unless libvir remvoes supprot for a filed in a new version | |
| 21:02:29 | sean-k-mooney | but im making the point that nova does not really expect this to change on live migation | |
| 21:02:44 | MengyangZhang[m] | get it | |
| 21:04:04 | sean-k-mooney | a concreate example fo this is the weith element https://libvirt.org/formatdomain.html#block-i-o-tuning | |
| 21:04:25 | sean-k-mooney | the range [100, 1000]. After kernel 2.6.39, the value could be in the range [10, 1000] | |
| 21:04:37 | sean-k-mooney | so hte changed the range that was allowed | |
| 21:04:47 | sean-k-mooney | again it old enough that its ok | |
| 21:05:17 | MengyangZhang[m] | make sense | |
| 21:05:38 | sean-k-mooney | im going to call it a day btu we shoudl see what the cinder folks think about how to proceed | |
| 21:05:59 | MengyangZhang[m] | make sense thanks for the help! | |
| 23:23:44 | opendevreview | Merged openstack/os-vif master: Remove os-vif-linuxbridge https://review.opendev.org/c/openstack/os-vif/+/941567 | |
| 23:38:13 | opendevreview | sean mooney proposed openstack/os-vif master: Update gate jobs as per the 2025.1 cycle testing runtime https://review.opendev.org/c/openstack/os-vif/+/935580 | |
| #openstack-nova - 2025-02-19 | |||
| 00:38:25 | opendevreview | Merged openstack/nova stable/2024.1: repoduce post liberty pre vicoria instance numa db issue https://review.opendev.org/c/openstack/nova/+/932064 | |
| 02:26:29 | opendevreview | Amit Uniyal proposed openstack/nova master: Reproducer for cold migration on shared storage https://review.opendev.org/c/openstack/nova/+/940304 | |
| 03:03:33 | opendevreview | Merged openstack/os-vif master: Update gate jobs as per the 2025.1 cycle testing runtime https://review.opendev.org/c/openstack/os-vif/+/935580 | |
| 03:05:12 | gmann | mikal: approved the spice tests tempest change but melwitt reminded its depends-on nova change. if you can remove that depends-on I can quickly re-approve it https://review.opendev.org/c/openstack/tempest/+/942023 | |
| 04:24:36 | opendevreview | Amit Uniyal proposed openstack/nova master: Reproducer for cold migration on shared storage https://review.opendev.org/c/openstack/nova/+/940304 | |
| 05:19:33 | opendevreview | Tatsuya Hayashino proposed openstack/nova master: Allow update metadata and tag for error VM https://review.opendev.org/c/openstack/nova/+/942167 | |
| 09:22:57 | mikal | gmann: sure, I can do that now | |
| 09:25:50 | mikal | gmann: done | |
| 10:25:02 | opendevreview | Rajesh Tailor proposed openstack/nova master: Handle InstanceExists exception for duplicate instance https://review.opendev.org/c/openstack/nova/+/860938 | |
| 10:32:31 | sshmlbrg | Hi! I have a question about the 'attach volume to shelved instance' functionality. How does it work if an instance in the 'shelved_offloaded' status has no availability zone, but when attaching, the availability zones of the instance and the volume are checked? The documentation states that this functionality works starting from microversion 2.20. | |
| 10:37:56 | opendevreview | Rajesh Tailor proposed openstack/nova master: Fix KeyError on assisted snapshot call https://review.opendev.org/c/openstack/nova/+/900783 | |
| 11:21:43 | opendevreview | Rajesh Tailor proposed openstack/nova master: Fix case sensitive comparison https://review.opendev.org/c/openstack/nova/+/933107 | |
| 13:20:26 | opendevreview | ribaudr proposed openstack/nova master: Add managed flag to PCI device specification https://review.opendev.org/c/openstack/nova/+/937649 | |
| 13:20:27 | opendevreview | ribaudr proposed openstack/nova master: Update driver to deal with managed flag https://review.opendev.org/c/openstack/nova/+/938405 | |
| 13:39:19 | opendevreview | ribaudr proposed openstack/nova master: Update driver to deal with managed flag https://review.opendev.org/c/openstack/nova/+/938405 | |
| 14:36:22 | MengyangZhang[m] | <MengyangZhang[m]> "Do we know what would be the..." <- sean-k-mooney: Do you have an idea of how to implement this? | |
| 14:37:45 | sean-k-mooney | i dont have context for waht your asking | |
| 14:42:04 | opendevreview | Rajesh Tailor proposed openstack/nova master: Add support for showing finish_time https://review.opendev.org/c/openstack/nova/+/928933 | |
| 14:42:32 | dansmith | bauzas: I'm here | |
| 14:42:39 | bauzas | cool | |
| 14:43:11 | frickler | MengyangZhang[m]: doing quote replies in the matrix doesn't transfer well into IRC, best avoid those | |
| 14:43:12 | bauzas | so indeed the lazyloading is something like a good performance concern | |
| 14:43:48 | bauzas | when we discussed with sean-k-mooney, we said eventually "yeah, but just for the moment we wouldn't working on it" | |
| 14:44:04 | bauzas | that said, now, if you want, I can provide another dependent change for this | |
| 14:44:43 | dansmith | bauzas: we will lazy-load every instance on every host we're considering...sequentially | |
| 14:44:58 | dansmith | for ten hosts with 100 instances each, that's 1000 trips to the database for each schedule operation | |
| 14:45:08 | bauzas | indeed you see this in https://4d2664479333d0a7d727-a4adcb41e06ec456a00383225f090da6.ssl.cf5.rackcdn.com/940642/9/check/nova-multi-cell/bbf5fea/controller/logs/screen-n-sch.txt | |
| 14:45:21 | bauzas | Feb 17 19:20:02.725500 np0039881226 nova-scheduler[84264]: DEBUG nova.objects.instance [None req-6e249595-d7b0-4cbc-95d0-e2798d45bc99 tempest-ImagesOneServerNegativeTestJSON-728670221 tempest-ImagesOneServerNegativeTestJSON-728670221-project-member] Lazy-loading 'system_metadata' on Instance uuid e6f93654-3b12-429d-8f17-15ad1e605964 {{(pid=84264) obj_load_attr | |
| 14:45:23 | bauzas | /opt/stack/nova/nova/objects/instance.py:1141}} | |
| 14:45:28 | dansmith | yeah | |
| 14:45:52 | bauzas | so I don't disagree with you, if you think it's a priority issue, I can try to fix it then | |
| 14:46:07 | bauzas | or you if you want | |