Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-04-22
18:53:57 jaypipes oh look, a case for forbidden aggregates :)
18:54:05 mriedem and yeah that's what we'd use ^
18:54:07 sean-k-mooney jaypipes: orgiginly in the code is said an aggrate for enabled because i fogot negitive member of was merged
18:54:23 jaypipes sean-k-mooney: it's the forbidden fruit.
18:55:00 sean-k-mooney from a db perspecitve this really would not be that expensice either way
18:55:27 sean-k-mooney the negitive case would be better but it will be similar
18:55:49 jaypipes so, I suppose, in order of preference, I would personally go with: a) finish up https://review.opendev.org/#/c/623558/ and the other cell-specific-service-listing thing mentioned in there, b) use an aggregate for disabled computes and member_of=! placement request filter
18:55:57 sean-k-mooney the aggrates table is just too uuid fields both of which are indexed
18:56:21 jaypipes actually, it's not really an either/or. can/should do https://review.opendev.org/#/c/623558/ as well as a placement request filter for !member_of a disabled aggregate.
18:57:26 sean-k-mooney yes they are not mutally exclucive and i think both are useful
18:58:40 jaypipes mriedem, sean-k-mooney: maybe adding a CONF.disabled_aggregate_uuid option that would be used to construct a placement request filter that used a member_of=!<uuid> query parameter to allocation candidates?
18:58:54 jaypipes or something like that..
18:59:06 sean-k-mooney jaypipes: well it would be a prefilter so it would be enabled via config
18:59:26 sean-k-mooney or are you also suggesting a config option for the compute node too
18:59:41 mriedem if we use an aggregate we have to use the same uuid everywhere,
18:59:51 mriedem either via config or hard-coded in api/compute/scheduler
18:59:53 sean-k-mooney ya that is simple however
19:00:14 sean-k-mooney just use a uuid5 that we caluate for a constat or hardcode it
19:14:59 efried oo, a special aggregate UUID. It's almost like... metadata.
19:15:22 sean-k-mooney that is sotred in nova
19:15:31 sean-k-mooney not placement
19:15:57 sean-k-mooney clinets of placement are free to use aggreate to group rps whatever way the like
19:27:25 sean-k-mooney by the way before i go i noticed some strange errors in the nfv-ci job
19:27:29 sean-k-mooney libvirtError: Cannot access storage file '/opt/stack/data/nova/instances/b4afcea9-12d0-4f79-953c-d09dad3e5515/disk' (as uid:107, gid:107): Permission denied
19:27:50 sean-k-mooney i also got DiskNotFound: No disk at /opt/stack/data/nova/instances/f5f5ce5f-a616-4e5e-aa35-d3cc469f869a/disk
19:28:46 sean-k-mooney i have set the concurance to 1 and its reruning currently as i think that is part of the issue but i have no iday way those disk errors are showing up
19:30:37 sean-k-mooney if people want to take a look the logs are here http://logs.openstack.org/97/652197/11/check/tempest-nfv-multinode/e1fa793/compute/logs/screen-n-cpu.txt.gz?level=TRACE#_Apr_20_22_14_03_503486
19:32:13 mriedem efried: except metadata that is unparseable by the human eye :)
19:32:46 sean-k-mooney other vms worked fine so im wondering if this is related to load/memory pressure.
19:41:39 mriedem d15ab1ed-0000-0000-0000-00000000000 :)
19:43:32 efried sean-k-mooney: speaking of CI...
19:44:08 sean-k-mooney what did i break now :)
19:44:15 efried I got an extremely vague question from within the blue walls today about what kinds of tests the PCI job ought to be running.
19:44:25 efried I hadn't a clue, but thought you might.
19:46:07 sean-k-mooney ideally pci passthough via alaise in the flavor + neutron sriov interface, then it would be nice to alos test cold migration/resize and possibly live migration in the neutron case
19:46:18 efried someone said something like the PCI job was just setting up devstack and not running any actual tests. I'm not sure how accurate that could be, or if they were looking at the wrong thing, or if at some point we disabled all the actual tests while we were working on the infra, or what.
19:46:32 sean-k-mooney i would just run the standared tempest test with tweeked flavors and different vnic_types
19:46:58 sean-k-mooney efried: it runs test but it boots like 1 vm or very few tests
19:48:02 efried was the PCI job just hitting SR-IOV?
19:48:17 efried Cause there's an SRIOV job that runs against neutron
19:48:22 efried not sure if those are related
19:49:17 sean-k-mooney the pci job was ment for testing QAT integration with flavor bassed passthough
19:49:29 sean-k-mooney there was a seperate one for neutron sriov testing
19:49:53 sean-k-mooney looking at http://52.27.155.124/pci/482200/7/
19:50:07 sean-k-mooney there are not test in the test repofrt
19:50:35 sean-k-mooney but im not sure that means threare are not test
19:51:18 sean-k-mooney it looks like some test are run http://52.27.155.124/pci/482200/7/console_status.log.gz
19:52:48 efried I can't tell if they pass
19:54:21 sean-k-mooney nor can i
19:54:46 sean-k-mooney looking at the devstack output and the tempest config i cant see tehm creating any custom flavor either
19:55:55 efried how did you know where to find these logs?
19:56:11 efried the CI is running, just not reporting?
19:56:41 sean-k-mooney i clicked on an old log got a 404 and went up one directory
19:57:13 efried how did you know this patch had any results? Did you search for reviewedby:Intel PCI?
19:57:52 sean-k-mooney i looked at one of my patches that i knew it had commented on in the past
19:58:08 sean-k-mooney http://52.27.155.124/ you can see the rest of the logs are here for the other intel cis
19:58:28 efried IC
19:59:00 sean-k-mooney this is a ovh log server that i used ot own/payfor via intel for the ci logs
19:59:49 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: debug config opts infinite recursion https://review.opendev.org/654468
20:00:11 sean-k-mooney or maybe it moved
20:00:16 sean-k-mooney it used to be http://intel-openstack-ci-logs.ovh
20:00:37 efried I think the 52. is aws
20:01:18 sean-k-mooney ya the pci ci used an aws log server. they moved to using our ovh one for a while i guess it moved back to aws at some point
20:01:20 efried yeah, that whole class A is amazon's. Not sure if that means aws, but probably
20:01:54 sean-k-mooney basicaly it just had to be a server that was not on the intel network
20:02:06 efried right, so the outside world could see it.
20:02:45 sean-k-mooney actully no os that you did not have to go through a 3 level security review to host an external facing service on the coperate network
20:03:10 efried yeah, that too
20:03:33 sean-k-mooney if its not on the copreate network it was way eaiser aslo the pci ci was originally in china which casued issues
20:04:07 efried okay, I'm going to dive over the firewall and throw some of this intel back at intel. Thanks for the help.
20:04:52 sean-k-mooney cool anyway my adivice would be to do something like im doing in the nfv ci test job
20:05:19 sean-k-mooney e.g. run standared tempest test but use a custom flavor or tweek teh vnic_type used in the tempest config
20:05:31 sean-k-mooney that will test 90% of the edgecases
20:05:50 sean-k-mooney at that point if the ci is stable it can be extended to test more neich things
20:07:13 mriedem dansmith: despite my typo in https://review.opendev.org/#/c/640197/ i'd like to get that in so we can move the grenade live migration job in-tree (this is under the queens backport) - because of devstack changes in queens + grenade i have to disable the ceph stuff in the grenade case, but it's either that or just drop the job altogether from queens which i'd like to avoid
20:07:48 mriedem the commit message attempts to explain the mess and reasoning for just punting and disabling the ceph case
20:09:26 dansmith okay
20:14:54 sean-k-mooney mriedem: speaking of ceph we have a downstream request to backport two of your patches to newton wich is eol upstrema. https://review.opendev.org/#/q/topic:bug/1635008+(status:open+OR+status:merged) i know its been quite a while but are you aware of any reaon of the top of your head why that woudl be a bad idea.
20:15:28 openstackgerrit Matt Riedemann proposed openstack/nova master: Add functional recreate test for regression bug 1825537 https://review.opendev.org/654066
20:15:30 openstack bug 1825537 in OpenStack Compute (nova) "finish_resize failures incorrectly revert allocations" [Medium,In progress] https://launchpad.net/bugs/1825537 - Assigned to Matt Riedemann (mriedem)
20:15:30 openstackgerrit Matt Riedemann proposed openstack/nova master: Drop source node allocations if finish_resize fails https://review.opendev.org/654067
20:15:34 sean-k-mooney it looked resonable to me at first glance but just said i woudl ask
20:15:58 mriedem sean-k-mooney: yes
20:16:20 mriedem there was a regression introduced
20:16:39 sean-k-mooney oh im glad i asked :)
20:16:50 mriedem for $5 i'll tell you the secret
20:17:54 mriedem https://review.opendev.org/#/q/I6fc7108817fcd9df4a342c9dabbf14ab7911d06a
20:18:00 sean-k-mooney https://bugs.launchpad.net/charm-nova-compute/+bug/1671422?
20:18:01 openstack Launchpad bug 1671422 in OpenStack nova-compute charm "charms: nova/cinder/ceph rbd integration broken on Ocata" [Critical,Fix released] - Assigned to James Page (james-page)
20:18:08 mriedem that might be the other one
20:18:37 mriedem https://review.opendev.org/#/q/Ieba216275c07ab16414065ee47e66915e9e9477d
20:19:01 sean-k-mooney ok so it did not handel olde connection info. so if we are to backprot it we need to backport the fixes for the regression too
20:19:35 mriedem sure
20:19:48 mriedem or just tell the bank to upgrade
20:20:16 sean-k-mooney i mean if i could i would but...
20:20:44 sean-k-mooney that said depending on how invasive the other fixes are that might be answer
20:23:28 sean-k-mooney huh the last one was only fixed recently in the grand scheme fo things
20:41:44 efried sean-k-mooney: I can see whitelist/alias stuff in the nova.confZ
20:42:04 efried pci_passthrough_whitelist = [{"vendor_id":"8086","product_id":"1520"}]
20:42:05 efried pci_alias = {"name":"pci_network_card","vendor_id":"8086","product_id":"1520"}

Earlier   Later