Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-12-07
17:41:51 sean-k-mooney no
17:42:05 sean-k-mooney the feature this would be interocudeing is the ablity to filter on the soft delete status
17:42:17 sean-k-mooney that does not require use to change the defualt behavior
17:42:28 sean-k-mooney so its not jsut signaling
17:44:19 gmann yeah, adding new filtering which is asked in current specless BP
17:44:52 gmann currently there is no way to filter soft deleted instance actions
17:45:39 gmann with new microversion, we solve both use case 1. existing keep working 2. way to filter the soft deleted one
17:45:45 sean-k-mooney so i think we agree that the current behaiovr is not broken or incorrect, that we could start marking the records as soft deleted when the instance is soft delete but only if we maintain the currnt behavior of returning both soft and not soft deleted records and we woudl also want to allow filtering if we made this change
17:47:56 sean-k-mooney the filtering is an api change as we are adding a new query arg even if the default behvior does not change.
17:48:21 pslestang sean-k-mooney: +1 for your proposition
17:49:28 sean-k-mooney pslestang: by the way what happend with your upstreaming efforts http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020383.html
17:49:49 sean-k-mooney that was the last mail i could find on that trhead was there a tc desicion made
17:50:59 sean-k-mooney i dont see it in https://opendev.org/openstack/osops/src/branch/master so i guess it was not added to osops
17:51:15 sean-k-mooney and i dont see a new repo for https://github.com/ovh/osarchiver/
17:51:27 sean-k-mooney did ye decide to just keep it in github in the end
17:53:33 sean-k-mooney pslestang: i think the prefernce was option 3 "Move it under its own repository under opendev and propose it as a new official OpenStack project" but i dont think that happened
17:54:46 sean-k-mooney if that was to happen i coudl see a day where nova could discontinue our current shadown tables eventually and rely on OSArchiver after a deprecation period
17:54:50 pslestang sean-k-mooney:this is what I was looking for, exact option 3 prefered but nothing done
17:57:18 pslestang We prefer to put it under opendev than keeping it in github, we really thing that other operators could take benefit of this tool
17:57:48 sean-k-mooney yep and honestly i think its a better approch then we have today in nova
17:58:17 sean-k-mooney so if it was mature enouch and operators started to use it i think we coudl eventually depercante and remove nova archiveal supprot
17:58:45 sean-k-mooney and replace it with your external soltuion once we had time to update the installer tools and work with the differnt sake holders to adopt it
17:59:31 sean-k-mooney the fact it can archive the data to an entirly differnt db or a csv file can signifcalty help with db load on the main db
17:59:46 sean-k-mooney and help keep the main db small
17:59:49 pslestang We use it on a weekly basis (crontab) to clean our DB, and we do not have any issue (except with instance_actions_* ;-))
18:00:54 sean-k-mooney oh i know it works well for you but for use to ever remove the native support in nova we woudl need all the installaiton tools or at least most of them to supprot your alternivie and we would need an upgrade stragy for exsing clouds
18:01:14 sean-k-mooney so its more maturing that process that would need work
18:01:40 sean-k-mooney cern were intersted in this too if i rememebr the mail thread
18:01:41 pslestang sure I understand
18:03:03 pslestang yes and I know also that ubisoft is using it
18:06:06 bauzas sean-k-mooney: distros would have to agree on it
18:06:16 bauzas this is shipping yet another dep
18:11:44 sean-k-mooney bauzas: it would be yes although we have talked about removing shadow tables in the past and not providing a replacement too
18:11:56 sean-k-mooney this would just be a way to provide a replacement
19:08:33 opendevreview mitya-eremeev-2 proposed openstack/nova master: Delete bogus attachments. https://review.opendev.org/c/openstack/nova/+/820935
22:31:59 opendevreview Merged openstack/nova stable/wallaby: Add functional regression test for bug 1853009 https://review.opendev.org/c/openstack/nova/+/811805
#openstack-nova - 2021-12-08
05:06:03 opendevreview Merged openstack/nova master: Extend the reproducer for 1953359 and 1952915 https://review.opendev.org/c/openstack/nova/+/820859
06:42:19 opendevreview mitya-eremeev-2 proposed openstack/nova master: Delete bogus attachments. https://review.opendev.org/c/openstack/nova/+/820935
09:56:52 opendevreview Pierre Libeau proposed openstack/nova master: Nova resize don't extend disk in one specific case https://review.opendev.org/c/openstack/nova/+/820531
10:45:03 gibi sean-k-mooney, slaweq: I looked at the gate bug https://bugs.launchpad.net/nova/+bug/1953478 and it seems to me that neutron sends the vif-plugged event at bind time instead of plug time in this case causing the timeout
10:45:47 gibi so far I thought that with ml2/ovs neutron always sends that event at plug time
10:47:24 slaweq gibi yes, afaik when You boot vm it will be sent when neutron will finish provisioning that port
10:47:41 gibi slaweq: it is an unshelve after shelve_offload
10:47:45 slaweq but during e.g. live migration it may be differently
10:47:55 slaweq I don't know about shelve and unshelve
10:47:56 gibi that is almost like a new boot
10:48:02 gibi but from a previous snapshot
10:48:25 gibi there is a new scheduling, a new port biding a new vif plug and spawn
10:48:52 slaweq so it should be sent when plug is completed
10:49:04 slaweq in such case
10:49:06 gibi yes, that is how I would expect it (and how nova expects it)
10:49:27 slaweq please move that bug to neutron then, I will check in our logs what happened there
10:49:31 opendevreview mitya-eremeev-2 proposed openstack/nova master: Delete bogus attachments. https://review.opendev.org/c/openstack/nova/+/820935
10:49:43 slaweq and thx for checking that
10:49:45 gibi slaweq: ok, I summarized this in the bug so you have the logs with timestamps from nova
10:49:51 slaweq ++
10:51:36 gibi I haven't looked at the other case in the same bug the tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert
10:51:45 gibi but I can do that too
12:12:59 sean-k-mooney slaweq: unshelve should be the same as first boot
12:13:00 lyarwood does anyone know when/where during a spawn that we would expect the metadata service to have details on a given instance ready to serve up?
12:14:34 lyarwood I'm looking at a CI failure downstream where cloud-init gets Connection refused everytime it tries to pull from the api but I can't see errors in the actual service logs
12:19:12 gibi lyarwood: I have limited knowledge too, but the request is goes from the guest, goes to the neutron-metadata service via the guest's network and the neutron forwards the request to nova-metadata
12:23:13 lyarwood thanks I think I was missing the neutron part
12:24:12 sean-k-mooney ill get the code one sec
12:24:32 sean-k-mooney lyarwood: it will always be ready before we call libvirt
12:25:20 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L168
12:25:28 sean-k-mooney that is where we generate the network metadata
12:26:15 sean-k-mooney which is called form the init of InstanceMetadata https://github.com/openstack/nova/blob/052cf963583ab7c6bbe4fcbf7bfe69f8f6733bdb/nova/api/metadata/base.py#L176
12:27:49 lyarwood ACK thanks
12:28:14 lyarwood looks like an issue before that as the neutron-metadata-agent on the compute isn't even seeing the requests
12:28:29 lyarwood I guess with plugging but AFAICT from the compute logs that worked
12:29:53 sean-k-mooney the neutorn-metadata-agent is typeiclay not on the compute but on the contler
12:30:09 sean-k-mooney it runs in either the router or dhcp agent network namespaces
12:30:14 sean-k-mooney depening on your config
12:30:27 sean-k-mooney ovn works slightly differntly and i think it does it on each compute
12:30:43 sean-k-mooney but the agent there is jsut calling the nova metadta api
12:31:06 sean-k-mooney the api i belive generates the instnace mentadta directly itself and caches it
12:32:48 sean-k-mooney for libvirt we generate it here by the way as part of creating the config drive https://github.com/openstack/nova/blob/e537d90d6fc0977742f7126c3f8cfef6bf8b2a15/nova/virt/libvirt/driver.py#L4782
12:36:10 sean-k-mooney lyarwood: are we using memcache downstream
12:36:25 sean-k-mooney we had to enable it upstream to make the ci stable
12:36:40 sean-k-mooney without it differrnt request could go to differnt api workers
12:36:57 sean-k-mooney cloud-init will only retry the first request
12:37:09 sean-k-mooney and after that it assuem the data is avaiable and does not retry them
12:37:28 sean-k-mooney if you hit a different worker and the node is under powered that can lead to timeouts
12:37:47 sean-k-mooney or failure to retive the data as genertatd the metadata is actully quite expensive
12:56:39 lyarwood sorry had to drop quickly for lunch
12:57:51 lyarwood this is part of a queens to train upgrade job downstream and it looks like the neutron metadata services are running on the computes
12:58:04 lyarwood I think this is failing post upgrade to train but it's super confusing
12:58:08 lyarwood jenkins--
13:00:45 lyarwood ah no it's pre upgrade on queens
13:09:23 gryf asfdsdf
13:11:57 kashyap gryf: Thanks; hope you don't change your password :D
13:13:57 gryf kashyap, not really, just the ssh connection hiccup ;)
13:14:27 kashyap Heh, was just trolling ya
13:16:05 gryf I figured :)
13:22:18 pslestang ccccccdlucfhelkhccngbvunedtddhhlecgrlcdverhc
13:22:40 pslestang ouup's sorry
13:22:49 pslestang hello by the way!
13:22:52 kashyap That's a YubiKey :)
13:22:58 pslestang ;-)

Earlier   Later