Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-10-13
02:01:22 opendevreview Steve Baker proposed openstack/nova master: Remain in DELETING for ironic CLEANING, CLEANWAIT https://review.opendev.org/c/openstack/nova/+/813729
02:03:05 opendevreview Merged openstack/placement master: Use 'functional-without-sample-db-tests' tox env for placement nova job https://review.opendev.org/c/openstack/placement/+/813680
03:29:45 opendevreview melanie witt proposed openstack/nova master: Handle urlencoded lists when filtering servers by tags https://review.opendev.org/c/openstack/nova/+/813736
06:54:26 bauzas good morning Nova
06:55:03 gibi bauzas: o/
06:56:38 gibi gmann: thanks for unblocking the placement gate with https://review.opendev.org/c/openstack/placement/+/813680 I have concerns that we only solved the local problem and not the root of the problem in zuul. I do think that tox_extra_args is defined to pass extra args to tox, and the test case filter regexp is such extra args
06:57:31 gibi it is not even a problem that zuul passes that args to tox when printing the config as tox accepts the extra args there too, but the problem is that it is passed without proper quoting
06:57:56 gibi so whoever will use tox_extra_args for anything in zuul that needs quoting will hit the same issue as we hit
07:02:40 bauzas can't disagree with gibi here
07:02:52 bauzas but at least we unblocked the periodic run
07:03:24 gibi bauzas: yeah, I'm OK to unblock the gate then fix the real problem
07:03:28 gibi that is a good order
07:07:43 gibi bauzas: if you have time for a smallish workaround fix to review that would be appreciated https://review.opendev.org/c/openstack/nova/+/813419
07:08:14 bauzas gibi: sure, today I'm trying to dedicate time for upstream reviews and bug triage
07:08:53 gibi thanks
07:17:45 alecorps7 hello
07:38:23 bauzas gibi: -1 for a relnote missing but see my comments https://review.opendev.org/c/openstack/nova/+/813419
07:39:25 gibi bauzas: thanks, I will respin
07:39:36 gibi bauzas: regarding a better solution I discussed that with sean-k-mooney
07:39:56 gibi bauzas: we need more info from neutron to decide when to wait for a plug event from neutron
07:40:12 bauzas gibi: we can discuss this next week with the neutron folks
07:40:21 gibi bauzas: we can try yes
07:40:37 bauzas gibi: what I'd like is that *all* neutron backends should provide an event
07:40:56 bauzas nova shouldn't know which backend neutron uses
07:40:59 gibi bauzas: yes that would be also an option but I think that is a harder one
07:41:00 bauzas at least for an event
07:41:41 bauzas I don't understand why it'd be hard for neutron to provide an event for plugs
07:42:02 bauzas again, having events be different between neutron backends looks bad to me
07:44:40 bauzas gibi: just added a point about this in https://etherpad.opendev.org/p/nova-yoga-ptg L105
07:49:10 gibi bauzas: thanks I will try to fill in the details htere
07:49:12 gibi sean-k-mooney: ^^
07:49:13 gibi fy
07:49:15 gibi fyi
08:42:58 frickler bauzas: lots of jobs with POST_FAILURE for failing log upload, but they seem to be back now, so safe to recheck hopefully
08:44:16 bauzas frickler: yeah the outage is now done
08:44:40 bauzas they isolated the faulty DC
08:45:19 bauzas thread in French https://twitter.com/olesovhcom/status/1448196879020433409
08:47:55 opendevreview Elod Illes proposed openstack/nova stable/ussuri: Revert "[stable-only] Set lower-constraints job as non-voting" https://review.opendev.org/c/openstack/nova/+/813784
09:45:52 sean-k-mooney bauzas: nova really really really need to know what what backend neutron uses
09:46:17 sean-k-mooney bauzas: or they need to tell use exactly when they will send the envents for that backend opacly
09:47:01 sean-k-mooney bauzas: not all backend can send plug time events because they dont all have a way to send info too neutron about when the port is created and its finsihed wiring it up
09:47:24 sean-k-mooney bauzas: at present ovn cannot do that although the ml2 dirver can be modifed to eventually do that
09:48:04 sean-k-mooney bauzas: for odl they had to create a websocket on the netorn server and modify odl to seend events to that websocket to implemente plug time events whihc invovled a lot fo code development in odl
09:48:41 sean-k-mooney bauzas: contrail still has not implemnted multiple port binding gettingthem to send plug time events likely will never happen
09:49:27 sean-k-mooney linux bridge can send pulg time event but because it currently pool for interface chcnages insted of opening a netlink socket and reciving event it can miss the removal and addtion of interface on hard reboot
09:50:40 bauzas sean-k-mooney: sorry but for me, events are like notifications
09:50:49 bauzas sean-k-mooney: it's kinda a public API
09:51:23 sean-k-mooney bauzas: its more a contract
09:52:16 sean-k-mooney bauzas: i proposed that neutron tell use when they send events or normalise to a common contract back in train and neutron did not agree to either
09:52:49 opendevreview Alexey Stupnikov proposed openstack/nova master: Rollback problematic port bindings on source host only https://review.opendev.org/c/openstack/nova/+/813659
09:55:34 sean-k-mooney bauzas: you can see in the first version fo thsi we had a network_events section https://review.opendev.org/c/openstack/neutron-specs/+/645173/1/specs/train/port-binding-extended-information.rst#141
09:58:40 sean-k-mooney bauzas: this is the orginal etherpad i wrote with rodolfo https://etherpad.opendev.org/p/portbinding-records
09:59:38 sean-k-mooney it was coverd durign the train corss project session https://etherpad.opendev.org/p/ptg-train-xproj-nova-neutron
10:00:47 sean-k-mooney bauzas: i am fine with finding a better solution to what i proposed in thte past however nova must know the behaivor of the neutron backend to operat correctly
10:01:39 sean-k-mooney what we are doing today is complex and very error prone and i spend far too much of my time currently fixing bugs that are cause by not knowing when events will be sent
10:06:49 bauzas sean-k-mooney: that's my concern
10:06:59 bauzas the more nova needs to know about neutron, the more issues we could get
10:07:08 bauzas as operators need to set different options per service
10:07:21 bauzas and they can miss some related options
10:07:53 sean-k-mooney bauzas: i have been trying to get neuton to expose the info we need for litrally year at this point with the goal of ensureing that no config option are need in nova
10:08:17 sean-k-mooney this has been an ongoing battle since before placment or os-vif was a thing
10:08:27 sean-k-mooney so i agree
10:08:37 sean-k-mooney we shoudl not need operators to care or set anything
10:09:09 sean-k-mooney but to do that we must know either what contract the neutron backend has abstractly or we need to embed that knolage in nova and just know what backend it is
10:09:35 sean-k-mooney currently we try tro guess based on the vif_type and some other partmeter in the port binding_details field
10:09:49 sean-k-mooney but we dont really have the info we need
10:11:06 bauzas sean-k-mooney: yeah hence the need of a nova-neutron discussion at the PTG
10:11:35 sean-k-mooney https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/network/model.py#L560-L579 this is what i have tried to do lately since the last time they rejected adding the events
10:12:11 sean-k-mooney with has_bind_time basically being https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/network/model.py#L481-L490
10:13:24 sean-k-mooney bauzas: im happy for there to be a PTG discussion on this since i planned to bring this up anyway as part fo the ovn migration disucssion
10:13:37 bauzas cool
10:13:39 sean-k-mooney bauzas: but i just dont have high confidence that anything will be done about it
10:14:07 sean-k-mooney its at least the 4th time we will have talked about it
10:14:53 bauzas it's a yet again "Dare to Care" discussion, heh
10:15:11 sean-k-mooney if we really want to fix this perhaps we (nova pepole) might need to go implement it in neuton
10:15:18 bauzas well
10:15:37 bauzas first, let's see what's coming
10:16:06 sean-k-mooney for reasons i know that our neturon folk at redhat are proposing adding plug time event support to ovn
10:16:13 sean-k-mooney reason being live migration
10:16:28 sean-k-mooney but to actully fix ovn live migration we need to actully change ovn too
10:17:01 sean-k-mooney the way ovn is currently desigined its not possibel to have 0 down time live migration with libvirt
10:26:10 lpetrut hi, could you please take another look over the gmr patch? https://review.opendev.org/c/openstack/nova/+/810922
10:27:27 sean-k-mooney sure
10:27:56 lpetrut thanks
10:31:22 bauzas lpetrut: excellent catch
10:31:49 bauzas lpetrut: could you please fill a bug against gmr not working properly due to uswsgi
10:32:05 bauzas I'd like this to be documented in the yoga relnotes if we merge it
10:32:27 sean-k-mooney bauzas: its not really a GMR bug
10:33:05 sean-k-mooney you already found that we can pass signals to the python app if we configure uwsgi correctly
10:33:30 sean-k-mooney but by default it will trap the sig_usr2
10:43:12 lpetrut I've added a Nova release note. indeed, it doesn't seem like an oslo.reports bug, if needed I can file a bug against nova. last time, the consensus was that it's a minor feature that doesn't require a blueprint
10:53:12 opendevreview Lucian Petrut proposed openstack/nova master: api: enable oslo.reports when using uwsgi https://review.opendev.org/c/openstack/nova/+/810922
11:26:42 gibi sean-k-mooney: I made a bit of progress witht the unshelve functional test. There is a reschedule happening, but it feels like it is on an instance from another test case?! see my last comment with logs in https://review.opendev.org/c/openstack/nova/+/813674
11:33:52 gibi sean-k-mooney: the subunit file has a lot more information that what is visible from the job-output.txt
11:34:31 gibi it can be extracted to individual tests and it shows that the instance uuid logged in our failed test is actually mentioned in another test case log as well
11:34:34 sean-k-mooney gibi: dod upi see https://review.opendev.org/c/openstack/nova/+/813695
11:35:10 sean-k-mooney ah you did
11:35:45 sean-k-mooney gibi: i feel like when we are waiting for the virsion notificiaotn in that case we need to wait for one related to the vm we are unshlving
11:36:04 sean-k-mooney but i tought the notifier was per test instnace
11:36:18 gibi sean-k-mooney: the notifier should be unique per test case too

Earlier   Later