Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-03-06
14:59:48 mtreinish mriedem: iirc those tests make multiple servers and networks, login to them with an ipv4 network and test connectivity between servers over different ipv6 configurations. It takes forvever just waiting for things to come up
14:59:48 HD|Laptop and horizon seems to sum up all the disks... to "Lokale Festplattenbelegung
14:59:51 HD|Laptop 120GB von 582,8TB benutzt
15:00:01 mriedem HD|Laptop: that comes from the libvirt driver in here https://github.com/openstack/nova/blob/dfaa513fe1329aabe6e5d83b6ac42e0e25ad9eb6/nova/virt/libvirt/driver.py#L5747
15:00:11 mriedem https://github.com/openstack/nova/blob/dfaa513fe1329aabe6e5d83b6ac42e0e25ad9eb6/nova/virt/libvirt/driver.py#L5762
15:00:26 mriedem since instances_path is on shared storage, it's going to show the entire storage pool for that path
15:00:31 mriedem i'm not aware of a way around that
15:00:40 HD|Laptop ah okay. damn :'D
15:01:07 mriedem mtreinish: time for an "extra-slow" tag soon
15:01:41 mriedem mtreinish: will openstack health show me how long on average the entire TestGettingAddress class takes?
15:01:57 openstackgerrit sean mooney proposed openstack/nova master: libvirt: auto detach/attach sriov ports on migration https://review.openstack.org/629589
15:02:29 mtreinish it doesn't let you filter the averages by class iirc
15:02:46 mtreinish but if you give me a sec I can hack together a graph for you manually
15:05:57 mriedem took ~35 minutes in logs.openstack.org/72/638072/14/check/nova-next/c8ecf61/job-output.txt.gz
15:06:00 mriedem 8 tests
15:14:07 efried mriedem: where/how can I run a CI that exercises ironic-in-nova?
15:14:18 mriedem efried: we have a non-voting job on all nova changes
15:14:23 openstackgerrit Matt Riedemann proposed openstack/nova master: Do not run tempest.scenario.test_network* tests in nova-next https://review.openstack.org/641416
15:14:26 mriedem ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa
15:14:47 mriedem and i think there are a couple in the experimental queue as well for multinode and neutron stuff
15:14:59 efried aha, okay.
15:15:26 efried mriedem: background: I finally bit the bullet and started hacking up a way for ironicclient to use the ksa adapter we go through all the trouble of building up on the nova side, only to throw it away.
15:15:30 sean-k-mooney efried: the other way to do it is a DNM change to ironic that depends on the nova change
15:15:51 sean-k-mooney that will runn the ironcis jobset and pull in the relevent nova patch
15:16:03 efried sean-k-mooney: and that exercises the nova side?
15:16:26 sean-k-mooney assuming they have jobs that boot ironic instances via nova
15:16:43 sean-k-mooney it just depends on what you want to test
15:16:47 efried in case you care.
15:16:47 efried nova: https://review.openstack.org/#/c/641170/
15:16:47 efried ironicclient: https://review.openstack.org/641169
15:16:49 mriedem melwitt: looks like functional test failures at the end of your counting quotas series are real https://review.openstack.org/#/c/638324/
15:17:41 mriedem efried: the one thing i'm not sure about is if that nova change will pull in the ironicclient code dependency or if it will pull in ironicclient from upper-constraints
15:17:53 mriedem might depend on if the job is zuul v3-ified
15:18:03 efried o
15:18:40 mriedem http://logs.openstack.org/70/641170/1/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/0e5cd32/controller/logs/pip2-freeze.txt.gz indicates it doesn't
15:18:44 mriedem python-ironicclient==2.6.0
15:18:47 sean-k-mooney efried: mriedem looks like the ironic base job is a zullv3 native job but it does not list openstack/nova as a required project
15:18:48 mriedem doesn't pull in the source dep
15:18:55 openstackgerrit Surya Seetharaman proposed openstack/nova master: Block automatic transport_url update for cell0 https://review.openstack.org/605414
15:19:10 efried well crap
15:19:20 mriedem so,
15:19:41 mriedem the "legacy" way to do this is a devstack change which depends on the nova/ironicclient changes and you configure devstack to pull in the client from source
15:19:47 mriedem assuming devstack has an ironic job...
15:20:09 efried and if it doesn't, make the devstack change and then make a nova change that depends on it.
15:20:13 mriedem or, your nova change depends on a devstack change that pulls in ironicclient from source which depends on the ironicclient patch
15:20:18 mriedem yeah
15:20:19 efried yeah
15:20:20 efried whee
15:20:50 mriedem https://github.com/openstack-dev/devstack/blob/master/stackrc#L345
15:21:16 sean-k-mooney efried: i can mock up a quick test patch that ill pull in one of the ironic jobs and force it to use nova and ironcic client form source if iyou like
15:21:58 efried sean-k-mooney: If you have that at your fingertips, that would be great. It would take me a while to figure out.
15:22:48 sean-k-mooney efried: if you list a project in required-project and it auto adds it to LIBS_FROM_GIT and zuul follow the depends-on stuff to pull in the requried patches
15:23:06 sean-k-mooney efried: it should only take me a minute to create one
15:38:35 openstackgerrit Jose Castro Leon proposed openstack/nova master: Extend volume for libvirt network volumes (RBD) https://review.openstack.org/613039
15:39:24 openstackgerrit sean mooney proposed openstack/nova master: [DNM] tempest client testing jobs https://review.openstack.org/641421
15:39:37 sean-k-mooney efried: i think ^ will do what you want
15:40:42 efried ...
15:41:36 sean-k-mooney i proably should have also dropt the tox jobs but they are quick
15:41:41 efried sean-k-mooney: Cool. How is that pulling in the ironicclient patch?
15:42:17 sean-k-mooney it is based on your nova patch an your nova patch has a depends on
15:42:37 efried right, but I thought we established that the depends-on wasn't working?
15:42:57 efried or is that magic instated by virtue of this having been done in .zuul.yaml?
15:42:57 sean-k-mooney so because i added the ironic client to the require projects it will pull it in when zuul prepares the git repos and add it ot devstack LIBS_FROM_GIT list
15:43:05 efried okay, nice.
15:43:36 sean-k-mooney we should still check when it actully runs just to be sure but it should work
15:44:53 sean-k-mooney i might quickly resping to drop the tempest full and greade job too since i has not started in zuul yet
15:45:00 sean-k-mooney well its ther ebut no vms yet
15:45:09 efried sean-k-mooney: Thanks for doing that. I have plenty of work left to do on the code side, but if I can at least get it running, then I have a way to debug.
15:46:56 openstackgerrit sean mooney proposed openstack/nova master: [DNM] tempest client testing jobs https://review.openstack.org/641421
15:48:24 sean-k-mooney there we go i missed doce but that will be much more efficent
15:48:33 efried thanks sean-k-mooney
15:48:50 sean-k-mooney and no worries hopefully that works the way we want it too
15:49:44 sean-k-mooney if it does i can move those jobs into the experimental pipline or replace the current non voting jobs we have
15:49:59 mtreinish mriedem: hmm, well getting the time series graph took longer than I thought (I must be rusty): https://blog.kortar.org/wp-content/uploads/2019/03/v6_network_total-1.png
15:50:24 mtreinish we also must have dropped a test (or skip it in some configurations) which made the aggregation difficult after Sept 2018
15:50:50 mtreinish I also box and whiskered each individual test: https://blog.kortar.org/wp-content/uploads/2019/03/v6_network_scenario_run_times.png
15:51:07 mtreinish it's definitely way too slow
15:53:15 mriedem geez yeah
15:53:34 mriedem that aligns with the 2100 seconds i saw in that timed out job
15:53:51 mriedem so i posted https://review.openstack.org/#/c/641416/ as a short-term workaround for nova-next
15:54:51 mriedem i can't really decide if we should whitelist the scenario modules we care about in the nova-next job, because then new ones could be added and we'd miss them, but at the same time there is probably more we should blacklist
15:55:29 mriedem i remember the good old days when tempest took 45 minutes
15:55:35 mriedem the whole job
15:55:44 mtreinish man, that was a long time ago
15:55:45 dansmith stephenfin: can you answer my questions about the test assertions in this? https://review.openstack.org/#/c/624592
15:55:53 stephenfin dansmith: sure
15:55:56 mriedem granted, the gate was fubar'd on a weekly basis, but still
15:56:01 mriedem it was fubar'ed FAST
15:56:04 dansmith stephenfin: I was thinking it would be nice to merge another good chunk of those this cycle, but...
15:56:43 sean-k-mooney mriedem for what its work im hoping to get the os-vif gates down to that by droping the keystone and cinder apis test.
15:57:11 mtreinish keystone tests take like zero time
15:57:18 mtreinish dropping cinder will save a lot of time
15:57:45 mtreinish sean-k-mooney: if there is no integration between os-vif and cinder I was just disable cinder all together in those jobs
15:57:51 sean-k-mooney mtreinish: ya but os-vif is a lib that never talk to any other openstack service
15:57:53 mriedem that's why i dropped the non-compute api tests from the nova-next job
15:57:53 mtreinish s/was/would/g
15:58:08 sean-k-mooney mtreinish: yes im going ot
15:59:28 sean-k-mooney i think in the os-vif jobs i might keep the compute and networking api tests + the senario test for now and see what the job times look like
16:00:05 sean-k-mooney if they are under an hour ill be happy if i can get down to 45 mins or so even more so
16:02:11 mtreinish well if you disable cinder (not just blacklist the cinder api tests) it won't run a good chunk of the scenarios

Earlier   Later