Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-16
09:05:00 sean-k-mooney sapd1: https://bugs.launchpad.net/nova/+bug/1708433 was opened by my old team as we had a customer request to enable sriov attach generically
09:05:38 sean-k-mooney sapd1: that was before my team was pulled off upstream work for internal projects. i have now move to redhat i might be able to work on this but not sure
09:05:52 sean-k-mooney i would have to ask
09:10:05 sean-k-mooney mriedem: there are a few more edge cases that we would have to handel. e.g. vm runs out of pci ports, updating instace request spec, numa affintiy and updating placement allocation would be the main ones. a spec is the best way to capture all the edgecases
09:11:14 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Remove the deprecated API extensions policies https://review.openstack.org/586872
09:12:37 sapd1 sean-k-mooney: good point. Because I am not a developer, So I can't help you. Because nova souce code is too complicated
09:13:03 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Remove the deprecated API extensions policies https://review.openstack.org/586872
09:13:09 mriedem sean-k-mooney: that's why there was a spec :)
09:13:21 mriedem but it's outdated by now probably
09:13:56 mriedem https://review.openstack.org/#/c/139910/
09:14:54 sean-k-mooney yes i remember. i think we referenced that spec in the bug or it was pointed out at least
09:16:05 sean-k-mooney sapd1: well if this would have been your first topic in nova, it is proably in one of the most complicated parts.
09:16:13 cdent mriedem: I left a comment on https://review.openstack.org/#/c/592285/ trying to get more info about what is actually being fixed, because it's not clear what the actual problem is. Do you have more info?
09:17:04 sean-k-mooney sapd1: mriedem am i can write a spec for this and submit it if you like. but i will have to check internally if i can do more then that and impelmnte it assumeing it was approved.
09:17:44 mriedem cdent: yeah yikun was going to update the ML thread on cells perf testing,
09:17:55 mriedem but he needed to monkey_patch the time module
09:18:02 sean-k-mooney there are some other topics related to this i was planning on writeing proposals for so its somwhat alinged with what i was going to suggest working on anyway.
09:18:26 sapd1 sean-k-mooney: I will vote for you. :D
09:19:26 openstackgerrit jichenjc proposed openstack/nova master: Add zvm admin intro and hypervisor information https://review.openstack.org/533125
09:19:27 openstackgerrit jichenjc proposed openstack/nova master: Add zvm CI information https://review.openstack.org/533512
09:20:01 mriedem sean-k-mooney: i'm indifferent to sriov port attach capabilities in nova
09:20:34 cdent mriedem: would be interesting to see/know what ways nova api (as a wsgi app) needs eventlet at all
09:20:55 cdent or it just happens to be picking it up accidentally and getting into trouble
09:22:04 sean-k-mooney mriedem: for me originally it was parity thing. e.g. it was yet one more thing you had to be aware of did not work with sriov.
09:22:30 sean-k-mooney mriedem: it became more important with hardware offloaded ovs
09:22:33 mriedem cdent: i left a comment
09:22:38 cdent thanks
09:22:50 mriedem sean-k-mooney: sure i get that
09:22:59 mriedem much like volume-backed instances with 33% of the compute API
09:23:08 mriedem or 80% of the compute API with non-libvirt drivers :)
09:27:20 sean-k-mooney mriedem: :) the main thing i was thinking of working on in stien was a generic device management framework for nova. to unify handeling of all passthough devices via a form of flavor decompostion. that is a lot of work though and im not sure its a high enough proirty to try and do in stien.
09:28:02 sean-k-mooney basicaly the openstack server create --device stuff i mentioned in vancouver.
09:30:30 mdbooth Ah, ha! My test is non-deterministic because the fake image service returns images as a list from a dict, hence in non-deterministic order. If the first one returned defined an external kernel then the libvirt driver uses the Flat imagebackend explicitly, which I'm not mocking in the Rbd tests.
09:30:57 mdbooth That was satisfying :)
09:31:11 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge extended availability zone response into server controller https://review.openstack.org/502859
09:31:23 sean-k-mooney mdbooth: if you ran it with python3 the dict has determistic ordering
09:31:34 mdbooth sean-k-mooney: Interestingly I was running py3
09:31:43 mdbooth Although py35, which doesn't have deterministic dict order
09:31:52 mdbooth iirc that 36 or 37?
09:32:06 sean-k-mooney py36 i think
09:33:02 sean-k-mooney mdbooth: that said the iteration order is fixed not sure about calls to values or keys but would assume they return ordered lists also
09:33:16 mdbooth Yeah, I'd assume so
09:33:33 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge config drive extension response into server controller https://review.openstack.org/584223
09:33:58 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge extended server attributes extension response https://review.openstack.org/584590
09:35:20 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge extended server attributes extension response https://review.openstack.org/584590
09:35:44 mriedem it's not sorted https://github.com/openstack/nova/blob/master/nova/tests/unit/image/fake.py#L164
09:35:45 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge keypair extension response into server view builder https://review.openstack.org/584748
09:35:55 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge server usage extension response into server view builder https://review.openstack.org/585262
09:36:18 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Merge security groups extension response into server view builder https://review.openstack.org/585475
09:36:20 tssurya mriedem: correct basic flavors (larger flavors users open special request) are available everywhere, special flavors for special projects in special cells where those tenants have access, by default projects are not mapped (when they are normal tenants/personal projects) and they land in a set of default cells. we have around ~4500 projects roughly which depending on where the resources are allocated,
09:36:22 tssurya get mapped to that cell
09:36:55 tssurya sean-k-mooney: we use the pre-filter from dansmith for scheduling based on the same criteria: projects mapped to cells/aggregates
09:37:14 tssurya so that placement returns only those nodes
09:38:02 sean-k-mooney tssurya: cool. i was wondering if ye were still using the oldway with the tenant affinity/isolation filter but that does not work right post palcement
09:38:46 tssurya right
09:39:56 sean-k-mooney tssurya: how have ye found the perfromce with the prefilter + placement vs cellsv1 and no placement. is it working well for ye?
09:41:50 tssurya sean-k-mooney: not really that good, we use the "limits" thing for limiting placement results for getting a decent performance
09:42:09 tssurya we are trying to improve the scheduling part performance wise, mriedem had some patches
09:42:43 tssurya but we have not seen much improvement in production after applying those
09:46:02 sean-k-mooney thats unfortunet. there is still alot of work going into makeing schdulign faster in general. hopefully the performacne is adaquit for now.
09:46:24 mriedem i imagine the big chunk of the per-request scheduling time is taken in the DB queries which iterate all selected compute nodes and then pull all instance UUIDs on each node
09:46:29 mriedem across all cells
09:46:42 mriedem that could be optimized into a single big join per cell probably
09:47:25 mriedem because CERN isn't using the track_instance_changes config option right?
09:47:34 mriedem where the computes report instance per host changes to the scheduler
09:47:35 mriedem ?
09:48:34 mriedem this method https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L710
09:48:46 mriedem specifically https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L735
09:49:00 mriedem that queries the cell db per compute host to find the list of instance uuids on that host
09:50:29 sean-k-mooney mriedem: for the cern case you really only need to fine out what cell the project is mapped too right? so if we had placement aggreates per cell we would just need to add a member of to the placement query for the cells assocated with the tenant and let placement do all the work
09:51:13 mriedem sean-k-mooney: CERN already has that
09:51:19 tssurya mriedem: yea the time consumption is in the pulling of all instances info
09:51:22 mriedem they have tenants mapped to host aggregates which are mapped to resource provider aggregates
09:51:26 mriedem and that does the pre-filtering
09:51:46 sean-k-mooney mriedem: oh ok cool
09:52:07 mriedem they use this stuff https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#aggregates-in-placement
09:52:09 sean-k-mooney mriedem: so why do we ever pull the per host info back?
09:52:34 openstackgerrit jichenjc proposed openstack/nova master: Add zvm admin intro and hypervisor information https://review.openstack.org/533125
09:52:35 openstackgerrit jichenjc proposed openstack/nova master: Add zvm CI information https://review.openstack.org/533512
09:52:51 mriedem sean-k-mooney: for the affinity filters
09:53:18 sean-k-mooney mriedem: ah but that is after the placmenet query right?
09:53:25 mriedem yes
09:53:46 mriedem pre-filter -> GET /allocation_candidates -> get_host_states -> per-host-instance query -> filters/weighers
09:54:11 sean-k-mooney ok i miss understood. i taught you were implying we were doing the db query in the prefilter
09:54:33 mriedem no the only thing we query there is which host aggregates are restricted for the tenant
09:54:52 sean-k-mooney ya that makes sense.
09:55:03 mriedem https://github.com/openstack/nova/blob/master/nova/scheduler/request_filter.py#L26
09:56:58 openstackgerrit Matthew Booth proposed openstack/nova master: Py3 fix in fake image service https://review.openstack.org/592056
09:57:00 openstackgerrit Matthew Booth proposed openstack/nova master: Add regression test for bug 1550919 https://review.openstack.org/591733
09:57:01 openstack bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9)
09:57:01 openstackgerrit Matthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate https://review.openstack.org/578846
09:58:15 sean-k-mooney mriedem: right we just get the aggreate uuids and stuff them in the destination object
09:59:41 mdbooth mriedem: ^^^ is my proposed for failed evacuate on shared storage. The functional tests exercise all 4 combinations of shared/nonshared instance directory and share/nonshared block.
10:00:03 mdbooth mriedem: Good call ;)
10:01:39 sean-k-mooney there is no stien spec template ye correct? im going to make one reusing the rocky template.
10:02:05 sean-k-mooney is it stien or stein?
10:02:29 mdbooth sean-k-mooney: The latter
10:03:11 sean-k-mooney damb german words not doing the i before e thing
10:03:35 sean-k-mooney that said more english word dont follow that then do also
10:03:42 mdbooth Our crazy european neighbours

Earlier   Later