Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-09-16
18:25:33 sean-k-mooney cool that ^ will make mdbooth happy
18:27:04 sean-k-mooney by the way can i get a second +2+w on https://review.opendev.org/#/c/670585/
18:27:39 sean-k-mooney i really want to ensure that is deprecated in train so we can remove it in Ussuri
18:27:52 sean-k-mooney as in like 2 weeks if not sooner
18:29:15 sean-k-mooney dansmith: mriedem: melwitt: could one of ye take a look when ye have time ^
20:03:02 mriedem sean-k-mooney: commented,
20:03:08 mriedem and added moshe and adrianc to it
20:04:22 sean-k-mooney thanks. if stephen does not adress the docs bugs ill do them tomorrow
20:05:26 sean-k-mooney mriedem: assuming no objection from moshe or adrianc are you generally ok with this?
20:06:41 mriedem i can't say i'm very familiar with the pci passthrough whitelist dev name stuff
20:06:51 mriedem or how much it's used
20:07:15 sean-k-mooney its an alternitive to useing the pci addres of vendor id and product id
20:07:40 sean-k-mooney the issue with is if you restart the compute agnet while its passed to a vm the device wont be found
20:08:03 sean-k-mooney we recently prevented the compute agent from removeing in use device from the db
20:08:11 efried mriedem: guessing that az fail is yet another global test var race. We should go on a crusade to murder all of those.
20:08:45 sean-k-mooney but we didnt always, also the name can change after you release teh VF/PF form teh vm
20:09:30 sean-k-mooney so the main issue is its not realible and we keep getting bug downstream that we cant fix
20:09:39 sean-k-mooney so we just want to stop supproting it
20:10:19 mriedem efried: is this a recent failure? b/c i'm not sure what is global about this one and it's been around awhile (the test that is)
20:10:34 efried mriedem: from what I could tell, it started hitting on 9/10
20:10:53 efried mriedem: I'm looking at the delta in test_aggregates.py for I9ab9d7d65378be564b3731b5227ede8cece71bef
20:11:06 efried https://review.opendev.org/#/c/671075/21/nova/tests/functional/test_aggregates.py
20:13:32 mriedem hmm, yeah probably related to that series
20:13:58 artom sean-k-mooney, https://review.opendev.org/#/c/682435/3
20:14:04 efried never mind, that one only merged yesterday.
20:14:10 artom Which mean that upstream whitebox has now "caught up" and can be used in tests
20:14:25 efried and its predecessor the day before
20:14:28 efried so that's not the culprit.
20:14:31 mriedem efried: so from the error it looks like the actual failure is not really the issue,
20:14:36 mriedem it's that we failed to put the instance on the host
20:14:38 mriedem b/c of the az filter
20:14:39 efried yup
20:14:47 artom Next step: update https://review.opendev.org/#/c/656890/ to move whitebox under openstack-qa/
20:14:49 sean-k-mooney artom: ok you didnt need to merge them for me to test with them
20:14:53 sean-k-mooney but ok
20:14:59 mriedem the boot server function is just waiting for the server to exit BUILD status, which it does when it changes to ERROR status
20:23:44 mriedem efried: so if i had to guess,
20:24:24 mriedem changing the base class on the test to ProviderUsageBaseTestCase but not cleaning up a bunch of the duplicate setup means we have multiple fixtures and multiple copies of the same conductor/api/scheduler services, and the api syncs informatoin to the scheduler about aggregates, which is probably not multi-scheduler aware in the functional tests,
20:24:40 mriedem so it's probably intermittent b/c sometimes we hit hte scheduler process that knows about the az metadata and sometimes we don't
20:25:40 efried mm. so why hitting since the 10th
20:25:41 mriedem i can put up a patch to cleanup that duplicate setup
20:25:50 mriedem efried: the first change to hit it in logstash was the one you pointed at
20:25:57 mriedem 671075
20:26:08 mriedem and then it was just a recheck grind
20:26:17 efried yeah...
20:26:27 mriedem see how many times someone named Eric Fried blindly rechecked it :)
20:26:44 mriedem i'll put up a patch
20:26:52 efried If only I had full debuggability from my phone.
20:27:39 mriedem or make shilpa responsible for investigating the failures and rechecking them
20:27:45 mriedem but whatever, done is done
20:30:28 efried mriedem: looks to me like everything (except the super()) through the self.computes = {} can be removed, yah?
20:30:51 mriedem yup, just did it, running the tests
20:31:11 efried k, just prepping for review :P
20:31:25 efried +2 "that's how I would have done it"
20:37:16 efried tests pass locally for me
20:37:34 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove redundancies from AggregateRequestFiltersTest.setUp https://review.opendev.org/682475
20:38:11 efried mriedem: two more lines
20:38:52 mriedem ack
20:39:42 efried mriedem: why only partial-bug?
20:40:02 mriedem b/c i don't know if it's the root fix
20:40:12 mriedem hence the "at least rule it out" comment
20:40:28 mriedem if we see the hits drop off in e-r by next week we can close the bug
20:40:45 efried this way requires us to remember to go back at the bug :)
20:40:48 efried but okay
20:40:58 mriedem i will when i cleanup old e-r queries
20:41:27 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove redundancies from AggregateRequestFiltersTest.setUp https://review.opendev.org/682475
20:42:57 efried thanks for the quick fix mriedem <high five>
20:43:08 mriedem <down low>
20:45:02 mriedem i wonder if the fake messaging driver doesn't do rpc fanout cast properly, if we should have some sort of fixture that blows up if you try to start more than one non-nova-compute service in a functional test
20:45:25 mriedem since that's twice in 2 weeks that we've had some issue from functional test inheritance doubling up on fixtures and stuff
20:46:02 sean-k-mooney are you suggestion on;ly allowing 1 comptue service in the fucntional tests
20:46:21 efried no
20:46:23 sean-k-mooney or havign a fixture to detect when you have more then one but have not set things up correctly
20:46:35 efried he's suggesting allowing only one *non* nova-compute service.
20:46:52 efried we clearly need multiple compute services for many tests.
20:47:01 sean-k-mooney oh so only one conductor or scheduler
20:47:01 efried but we should never need more than one superconductor
20:47:04 efried right
20:47:20 openstackgerrit Matt Riedemann proposed openstack/nova master: Remove SchedulerReportClient from AggregateRequestFiltersTest https://review.opendev.org/682480
20:47:38 mriedem correct, one controller service
20:47:43 mriedem one of each
20:50:53 mriedem i think multiple apis and conductors are probably ok since those are stateless
20:51:02 mriedem but the scheduler has some stateful crap in the HostManager
20:51:06 mriedem like aggregate info
20:52:12 mriedem b'2019-09-16 15:10:19,491 INFO [nova.service] Starting scheduler node (version 19.1.0)'
20:52:12 mriedem but yeah the test was definitely starting 2 schedulers
20:52:17 mriedem b'2019-09-16 15:10:20,156 INFO [nova.service] Starting scheduler node (version 19.1.0)'
21:01:13 efried mriedem: what was the other bug with the duplicated services?
21:01:26 mriedem duplicated placement fixture
21:01:27 mriedem sec
21:01:41 mriedem https://github.com/openstack/nova/commit/5e1b096894f6de4cfbca254cf74dcfcf56358ea5#diff-5befe429f14247314e6ca487aa4e13bd
21:02:07 efried thank you sir
21:02:13 efried I'm posting a poison patch now
21:02:23 mriedem heh i was just writing my commit message ofr one
21:02:25 mriedem *for
21:02:30 efried oh, no bug for I057a07c8d0b880c8d09fc2e618ce1f7fc885beda ?
21:02:39 mriedem guess not
21:03:20 efried hm, if that one was placement fixture, my fix isn't going to hit it.
21:03:59 openstackgerrit Matt Riedemann proposed openstack/nova master: Do not allow mutiple nova-scheduler workers https://review.opendev.org/682485
21:04:06 mriedem efried: ^ is what i was thinking of

Earlier   Later