Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2025-02-26
00:26:16 sean-k-mooney[m] basiclly it would be good for nova to hide this impelation detail form the user. its unlikely that archiveing 1000 instance would exceeed the transaction limit size
00:26:27 sean-k-mooney[m] posible but unlikely
00:26:40 sean-k-mooney[m] * purgeing not archiveing
00:27:25 sean-k-mooney[m] to me the fact a transaction size limit is a internal detail that a user of nova manage should ideallly not need to care about
00:27:26 melwitt fwiw the database transaction limit and packet size etc is all configurable on the database side so it's not something you can assume
00:27:46 sean-k-mooney[m] right but it is something we can look up
00:27:54 melwitt in my experience, operators do like to tune this to how they want, so I haven't thought about hiding bc of that
00:28:40 sean-k-mooney[m] im still of the opion that nova should not have shadow tables and archiving should be done externally to noav using other tooling :)
00:28:49 melwitt not really because we can't go looking into their innodb config or whatever
00:29:11 sean-k-mooney[m] you can make an sql query to look it up i touhgt
00:29:22 sean-k-mooney[m] the same way you can do that to get the connection limit
00:29:26 melwitt oh. I guess you could do that
00:30:59 sean-k-mooney[m] https://dev.mysql.com/doc/refman/8.4/en/group-replication-system-variables.html#sysvar_group_replication_transaction_size_limit
00:31:19 sean-k-mooney[m] melwitt that could be a future thing
00:31:28 melwitt I feel like that would be a separate thing where if we did it we would change archive to work the same way, at the same time. I think they should be consistent.
00:31:31 sean-k-mooney[m] the fact you providing a default kind of paper over the problem
00:32:06 sean-k-mooney[m] ya im not going to ask you to put it in the current patch fro that reason
00:32:14 melwitt and this patch is addressing the fact that today, if your shadow tables are too large, you are just screwed and you can't purge them with nova tooling and you have to truncate them manually
00:32:19 sean-k-mooney[m] i just wanted to raise that design point
00:32:36 melwitt fair enough :)
00:33:11 sean-k-mooney[m] yep so personally i think we should be deprecating the nova tooling and saying you should use some fo the external toolign for this
00:33:24 sean-k-mooney[m] but i know that wont happen any time soon
00:34:03 melwitt I think people might find that harsh. what kind of external tooling are you thinking of?
00:36:51 sean-k-mooney[m] there are two diffent set of tooling
00:37:22 sean-k-mooney[m] there is a tool that manages archiveing and purging records for all service in a common way inculdiign for serice that dont have shadow tables
00:38:04 sean-k-mooney[m] ill see if i can find the link but it handels nova neuron ectar all the same way with the ablity to archive them to a diffent database service instance or locally to files
00:38:13 melwitt oh, right, that does sound familiar
00:38:21 sean-k-mooney[m] there si allso osops tools which has a nova specifc srcipt https://opendev.org/openstack/osops/src/branch/master/tools/generic/nova/openstack_db_archive.sh
00:39:26 melwitt yeah, I see what you mean now. I agree I think that would be ideal for there to be a unified thing that will work for all of the projects
00:39:59 melwitt when you said "external" I thought you meant external to openstack and was confused
00:41:22 sean-k-mooney[m] https://github.com/ovh/osarchiver
00:41:34 sean-k-mooney[m] well i was thinking of ^
00:41:56 sean-k-mooney[m] thats the thing that basicaly does this for all proejcts in a consistent way
00:42:08 melwitt oh, interesting
00:42:59 sean-k-mooney[m] it basically does not use shadowtable at all
00:43:20 sean-k-mooney[m] it can arcive without deletign or it can archive and delete the rows in teh service dbs
00:43:27 sean-k-mooney[m] to get a similar effect
00:44:12 sean-k-mooney[m] given you can restore thing from the shadown tabels to the main tables i would actully say its more usefual because it mvoes the data to an entirly differnt db in teh same format
00:44:29 sean-k-mooney[m] which is you ever wanted to restore those rows back for any reason would be simpler
00:45:11 sean-k-mooney[m] its like doing incremental backups of only the rows marked as deleted
00:45:40 sean-k-mooney[m] the reason osarchiver is not hosted in opendev is because we didnt agree which repo to put it in
00:45:45 sean-k-mooney[m] which is a shame
00:45:50 melwitt yeah, I was just thinking that if people kept incremental backups there would be no use for shadow tables
00:46:14 melwitt hah, nice
00:46:53 sean-k-mooney[m] there was a maining list tread about it like 4 years ago i belvie and it just stalled out
00:47:29 melwitt reposhed
00:47:59 sean-k-mooney[m] melwitt: anyway i think that a better technial solution i just dont think we will ever get to that at least not unless something changes
00:48:29 sean-k-mooney[m] i wanted to use that in our new installer but…. it was too much effort to advocate for that and get by in
00:49:23 melwitt yeah. especially difficult if it's not in opendev
00:50:02 sean-k-mooney[m] it was more that pm didnt want use to focus on day two operations like automating backups of db in the operators
00:51:04 sean-k-mooney[m] so providing incremental backups and “archiving” of this in a centralised way was not a priority
00:51:16 sean-k-mooney[m] so we just use nova manage in a k8s cron job instead
00:51:39 sean-k-mooney[m] i.e. we keep is simpel using the upstream tooling
00:52:15 sean-k-mooney[m] im not saying it could never be in scope but it was not in the mvp list for v1
00:52:32 melwitt yeah
00:54:00 sean-k-mooney[m] ill try and add that to my review list for tomorrow
00:54:29 sean-k-mooney[m] its not nessisally impacted by ff but while i have context i might as well
00:55:26 melwitt haha thanks
00:55:48 sean-k-mooney[m] if you feel like reviewing something small this and the repoducer under neath is waiting for a second +2 https://review.opendev.org/c/openstack/nova/+/932931/2 although i pinged sylain to hopefully take a look above
00:57:01 sean-k-mooney[m] i think its time i went to sleep chat to you tomorrow o/
00:57:07 melwitt sylvain would be ideal for that one but if he doesn't get to it I can do it
00:57:13 melwitt seeya tomorrow o/
01:24:46 opendevreview Merged openstack/nova master: Fix device type when booting from ISO image https://review.opendev.org/c/openstack/nova/+/909611
01:39:28 opendevreview melanie witt proposed openstack/nova master: DNM test headers https://review.opendev.org/c/openstack/nova/+/942752
06:38:33 opendevreview Amit Uniyal proposed openstack/nova master: WIP: remove swap.disk from disk.info https://review.opendev.org/c/openstack/nova/+/942759
06:45:22 opendevreview melanie witt proposed openstack/nova master: DNM test headers https://review.opendev.org/c/openstack/nova/+/942752
08:50:33 opendevreview Merged openstack/nova master: Add repoducer test for bug 2074219 https://review.opendev.org/c/openstack/nova/+/932930
09:11:53 bauzas gibi: sean-k-mooney[m] : I'd appreciate if you could get a swing for https://review.opendev.org/c/openstack/nova/+/845757/
09:12:08 bauzas I'll also try to look at the functests failing
09:20:42 opendevreview Amit Uniyal proposed openstack/nova stable/2024.2: WIP: remove swap.disk from disk.info https://review.opendev.org/c/openstack/nova/+/942764
09:31:03 opendevreview Amit Uniyal proposed openstack/nova stable/2024.1: WIP: remove swap.disk from disk.info https://review.opendev.org/c/openstack/nova/+/942765
09:34:49 opendevreview Callum Dickinson proposed openstack/nova master: Add image meta to libvirt XML metadata https://review.opendev.org/c/openstack/nova/+/942766
09:54:29 opendevreview Amit Uniyal proposed openstack/nova stable/2023.2: WIP: remove swap.disk from disk.info https://review.opendev.org/c/openstack/nova/+/942769
10:17:21 gibi bauzas: is it something we need to land before FF or something we can land after FF?
10:17:53 bauzas gibi: well, this is not really a bugfix
10:18:17 bauzas but it fixes evecuates for VGPUs
10:38:37 opendevreview Merged openstack/nova master: Fix detaching devices by alias with mdevs https://review.opendev.org/c/openstack/nova/+/932931
11:18:49 opendevreview Callum Dickinson proposed openstack/nova master: Add image meta to libvirt XML metadata https://review.opendev.org/c/openstack/nova/+/942766
11:25:09 sean-k-mooney bauzas: ack ill add it to my list, i have to do a pass on the nova-operator stuff and then melwitt's nova-manage command today but i think i can get to those as well
11:25:59 sean-k-mooney bauzas: i think it might fix evacuate for anything with nested resouce providers if i rememebr correctly
11:26:41 sean-k-mooney maybe not
11:27:06 sean-k-mooney that not the patch i was thinkign of
11:43:02 opendevreview Stefan Hoffmann proposed openstack/os-vif master: [ovs] Add method to get external_ids from Interface https://review.opendev.org/c/openstack/os-vif/+/942715
12:04:40 opendevreview Stefan Hoffmann proposed openstack/nova master: optional wait for ovn plug and create tap https://review.opendev.org/c/openstack/nova/+/942786
16:30:52 frickler Uggla: please check https://review.opendev.org/c/openstack/governance/+/942507/comment/029df38c_6df9f638/ . it would likely also be helpful if you could join the #openstack-tc channel while you are PTL
16:32:36 Uggla frickler, I'm currently in a meeting, but I'will have a look after.
16:34:04 frickler Uggla: thx, not urgent, just so you don't miss it
16:53:44 sean-k-mooney dviroel: left some minor feedback on https://review.opendev.org/c/openstack/nova/+/938604 but i think this is mostly good to go
16:55:56 dviroel sean-k-mooney: oh, thanks, indeed, going to update it
16:56:25 sean-k-mooney im looking at the tempest patch now so far looks good to me
17:05:21 sean-k-mooney dviroel: minor comment on https://review.opendev.org/c/openstack/tempest/+/939375 as well
17:05:40 sean-k-mooney over all it looks fine to me but i htink you need to update teh import
17:06:19 sean-k-mooney gmann: ^ would you be able to comment on the questions i have inline and confirm the schema inheritance is correct or not?
17:38:49 opendevreview Douglas Viroel proposed openstack/nova master: Add support for showing scheduler_hints in server details https://review.opendev.org/c/openstack/nova/+/938604
17:40:42 dviroel sean-k-mooney: ^ updated. Lets see if functional is going to pass. I added a scheduler hint to the server creation req
17:41:00 dviroel in the api_sample_tests
17:45:34 sean-k-mooney assuming it green i think ill be +2
17:45:41 sean-k-mooney ill check back in a bit once we get some results
17:46:40 sean-k-mooney if it does break thign we can revert that chagne and try and update it in a follow uponce the feature is merged
17:48:15 sean-k-mooney dviroel: it looks a little funky https://5671b50e1757693146fd-d2676880cd4ce07846d9d5d9e5ab28d1.ssl.cf1.rackcdn.com/938604/11/check/build-openstack-api-ref/6d89054/docs/#id30

Earlier   Later