Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-01-13
15:44:32 sean-k-mooney we shoudl just compute the desired endstate then comiit the change in one trasaction
15:44:47 TheJulia a couple thousand specific column field updates in a heavily used table?
15:44:48 sean-k-mooney you just need to use a case statement to update the instance.host on all instance in one go
15:45:02 TheJulia eh... then again performance has changed drastically since the days I did that often manually
15:46:03 sean-k-mooney so i think its just one colume on one db table with many rows
15:46:11 sean-k-mooney that shoudl be quick
15:46:40 TheJulia eh, last time I did something like that manually like 4 thousand rows in a 100k row table it was a couple minutes, but again, that was like a decade ago
15:47:48 TheJulia Anyway, I'd need to double check a few things but a transaction, at least to me seems more harmful in that the compute node's rebalance is hidden state so we could get a bunch of conflicting transactions pile up, all fail past the first one, and trash to try and correct some of it as time goes on consistency wise
15:49:41 sean-k-mooney well if we dont have a transaction you will need to role back the partially reblanced state
15:49:52 sean-k-mooney manually right
15:50:07 sean-k-mooney or trigger a full rebalnace again
15:50:18 sean-k-mooney to even out the load on the compute services
15:50:45 opendevreview Merged openstack/nova-specs master: add per process healthcheck spec https://review.opendev.org/c/openstack/nova-specs/+/821279
15:52:32 TheJulia well... a transaction could work in try to generate the updates and then only check if things have changed before committing it if things have not changed, but I'd need to dig into the dbapi internals since to do so likely sounds like rpc functionality would be required to batch it all up properly outside of the normal transactions oslo_db drives
15:52:40 TheJulia since we're doing it one at a time instead of a bulk change
15:54:03 sean-k-mooney we would need to have a db method for this yes not just loop over the isntance set the filed and call save
15:54:09 TheJulia I guess I'm worried about trying to overthink it and then over engineer it, or if a model of eventual consistency is a happier place to be. I'm very much on the eventual consistency mindset since for ironic, it *really* doesn't matter which node proxies the request as long as it is online
15:54:51 TheJulia it all matters for rpc request routing ultimately
15:54:52 sean-k-mooney i wonder if for ironic it would be better to have a single shared topic queue
15:55:16 TheJulia what would that change/gain us in this situation?
15:55:17 sean-k-mooney so that all the compute shared one queue and any of them could deque it
15:55:34 sean-k-mooney it would mena the isntance.host woudl not matter anymore
15:55:35 TheJulia hmm, that could entirely do away with the hash ring
15:55:48 TheJulia well, kind of
15:56:20 TheJulia hmmmm it would have to know it is ironic
15:56:29 TheJulia but that could actually be navigated upgrade wise too
15:56:48 TheJulia that doesn't fix the UX issues which are ultimately bugs in past releases though
15:56:55 sean-k-mooney yes but we do know the hypervior type at least on the ocmpute node side not sure about the instnace object
15:57:21 TheJulia I don't think it is on the instance object
15:57:28 TheJulia but again, I've not looked at its structure in a while
15:58:00 sean-k-mooney well the way to hack it is to decied instance.host would always be set to "ironic" or simiarl for ironci contoled instnaces
15:58:23 sean-k-mooney anyway the impartnat thing is you still care about fixing this issue
15:58:30 sean-k-mooney but dont curretnly have time to work on it
15:59:40 sean-k-mooney so we shoudl consider if we "redhat comptue team" have capsity to help this cycle or next
16:00:26 sean-k-mooney we coudl land your patch as is but im not sure long term its the best approch but it would stop the bleeding in the short term
16:00:54 sean-k-mooney which is the whole perferect is the enemy of good enough argument
16:40:23 opendevreview Balazs Gibizer proposed openstack/nova master: DNM: trigger nova-next with new tempest test https://review.opendev.org/c/openstack/nova/+/824607
16:43:39 TheJulia sean-k-mooney: yeah. :(
16:43:58 TheJulia sorry, been distracted looking at a blocker issue
16:53:55 admin1 hi all .. how to figure out what puts messages in versioned_notifications.info ..but there are no consumers
16:54:06 admin1 so the queue just grows and grows .. and then i have to manually delete it
16:54:55 gibi admin1: you can disable notifications
16:55:09 admin1 gibi, from where/how ?
16:55:15 gibi sec...
16:56:30 gibi admin1: https://docs.openstack.org/oslo.messaging/latest/configuration/opts.html#oslo_messaging_notifications.driver
16:56:39 gibi admin1: so in the nova service configuration files
16:56:53 gibi admin1: set [oslo_messaging_notifications]driver=noop
16:59:07 sean-k-mooney admin1: its disabled by default
16:59:15 gibi sean-k-mooney: I don't think so
16:59:24 sean-k-mooney the noop driver is the defult
16:59:25 gibi sean-k-mooney: I think the default is messaging_v2
16:59:31 sean-k-mooney i dont think so
16:59:37 gibi hm
16:59:40 gibi interesting
16:59:46 gibi according to the doc you are right
16:59:55 gibi I alwas remembered it is enable by default
17:00:01 sean-k-mooney devstack enables it by default
17:00:27 sean-k-mooney and it also defaluts to unversioned if i remember
17:00:37 gibi yes, the unversioned is the default
17:00:44 gibi we never switched it to versioned
17:00:52 sean-k-mooney we really should
17:01:05 sean-k-mooney and eventurlly remove the deprecated unversioned notifications
17:01:13 sean-k-mooney they have been deprecated since mitaka
17:01:26 gibi sean-k-mooney: the problem that there are openstack services using the unversioned
17:01:33 gibi so we would migrate them first
17:01:57 sean-k-mooney ya but honestly we should propose that a a cross projectr goal
17:02:28 sean-k-mooney the problem really haing people to do the work
17:02:50 sean-k-mooney maybe a topic of next ptg
17:02:54 sean-k-mooney do we know which ones rely on it
17:03:16 gibi I have to dig
17:03:28 gibi ceilometer is one I'm sure
17:03:31 sean-k-mooney i support cilometer, cloud kitty, heat or masikari are the only ones that might be impacted
17:03:39 sean-k-mooney maybe watcher
17:04:25 sean-k-mooney as long as we are not adding any new unversion notification i guess it does not hurt use too much
17:04:51 gibi I agree, it does not hurt
17:05:02 gibi and we have a test in place to forbid new unversioned notifications
17:10:38 bauzas yuval: around ? I don't see a blueprint filed for https://review.opendev.org/c/openstack/nova-specs/+/824191
17:10:57 yuval I am here
17:11:03 bauzas yuval: could you please create one using the link you gave, ie. https://blueprints.launchpad.net/nova/spec/nova-support-lightos-driver
17:11:05 yuval yes I created blueprint
17:11:06 bauzas ?
17:11:10 yuval just a second
17:11:16 bauzas ok, then the URL is wrong
17:11:35 bauzas we can change it
17:12:12 bauzas yuval: found it https://blueprints.launchpad.net/nova/+spec/nova-support-lightos-driver
17:12:39 bauzas oh, yeah, I see
17:12:46 bauzas s/spec/+spec
17:12:50 yuval https://blueprints.launchpad.net/nova/+spec/nova-support-lightos-driver
17:13:23 bauzas yuval: could you create a change fixing the URL in https://review.opendev.org/c/openstack/nova-specs/+/824191/6/specs/yoga/approved/lightos_volume_driver.rst#11 ?
17:13:28 bauzas yuval: I'll fast approve it
17:13:38 yuval https://blueprints.launchpad.net/nova/spec/nova-support-lightos-driver
17:13:45 yuval this is currently in the spec
17:13:50 sean-k-mooney yep
17:13:51 yuval its missing the "+"
17:13:53 sean-k-mooney its missing the +
17:13:55 sean-k-mooney yep
17:13:55 yuval before the spec
17:13:57 bauzas yup, please update the link in the spec
17:14:02 yuval just a sec
17:14:10 bauzas (18:12:46) bauzas: s/spec/+spec

Earlier   Later