Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-01-05
16:54:45 mriedem melwitt: which service should i see these log statements? cauth? https://review.openstack.org/#/c/320063/25/nova/objects/console_auth_token.py@116
16:54:56 mriedem or will this not actually run yet in ci in this change?
16:56:27 melwitt mriedem: yeah I think you won't see it until the last change in the series, in the novnc service log
16:57:07 mriedem ah yes
16:57:08 mriedem http://logs.openstack.org/12/527812/1/check/tempest-full/26f5e5f/controller/logs/screen-n-novnc-cell1.txt
16:57:15 mriedem ess_path=None,port=5900,token='***',updated_at=None) {{(pid=6917) validate /opt/stack/nova/nova/objects/console_auth_token.py:159}}
16:57:15 mriedem Jan 05 01:44:25.338490 ubuntu-xenial-citycloud-sto2-0001695415 nova-novncproxy[10280]: DEBUG nova.objects.console_auth_token [None req-166854c4-3f01-4db1-890b-212b7ba78e95 None None] Validated token - console connection is ConsoleAuthToken(access_url_base='http://10.0.1.151:6080/vnc_auto.html',console_type='novnc',created_at=2018-01-05T01:44:25Z,host='127.0.0.1',id=1,instance_uuid=4453afe4-e6eb-4479-a073-0f99a76c71f1,internal
16:57:26 mriedem NOICE
16:57:33 melwitt :D
16:58:14 mriedem finucannot: you were +2 on https://review.openstack.org/#/c/320063/ before a rebase and some changes
16:58:22 leakypipes fried_rice: question...
16:58:30 melwitt I did a bunch of token masking for the logs and then realized the websocketproxy logs the whole thing on its own, not our code
16:59:25 leakypipes fried_rice: did we say that for allocation candidates responses, we would include "intermediary nodes" in the provider_summaries section even if those nodes weren't involved in any of the allocation_requests?
16:59:51 melwitt but it's at level INFO so if someone cares they can turn the level up
17:00:31 leakypipes fried_rice: for example, say we have a compute node provider, two numa cell providers and two pfs that are children, one of each numa cell. and the allocation_requests contains the compute node (for VCPU and MEMORY_MB) and both PFs (for SRIOV_NET_VF resources), should the provider_summaries include the numa cell providers as well even though no resources are directly consumed from them?
17:01:27 cdent bbs
17:02:12 mriedem melwitt: on that index, i'd be fine with it in the same change as long as you call it out in the commit message and check the columns instead of the index name - i doubt anyone would want/need to backport that index add
17:02:18 mriedem if that makes life easier
17:03:25 figleaf leakypipes: it would make sense to include the whole "tree" of RPs, so that filters etc. could operate on that info
17:04:34 leakypipes figleaf: ack, that's what I thought I remembered. thanks for reminding me. :)
17:04:43 sean-k-mooney leakypipes: that is assuimg the resouce provider tree has 3 branches basically host->numa_node->pf, host->vcpu, host->memory_mb and vcpu and memory are not inventories of the numa nodes correct
17:04:58 leakypipes sean-k-mooney: right.
17:04:58 melwitt mriedem: okay. it probably makes more sense to be separate since the query isn't till a later change anyway. but curious, how could it end up part of a backport?
17:05:52 leakypipes sean-k-mooney: plus, keep in mind the allocation_requests and the provider_summaries sections of GET /allocation_candidates response are different things... the former section is the providers that would be actually allocated against. the latter section is just informational.
17:06:24 openstackgerrit Merged openstack/nova master: Use UEFI as the default boot for AArch64 https://review.openstack.org/489951
17:09:22 sean-k-mooney leakypipes: if the summery is just informational and wont be just to make desision then i dont see any harm in simplfying the representation and just weak referncing the numa afinity of the resouces.
17:09:53 leakypipes sean-k-mooney: could you elaborate on what you mean by that?
17:10:11 leakypipes sean-k-mooney: the numa affinity of the PFs is not a tag/trait. rather, it's the structure of the tree itself.
17:12:21 sean-k-mooney leakypipes: if the provider_summaries was used by a weigher for example then it would be good to keep the numa afinity info in the summary however if the chose beteewn which allocation candiate is made based on the tree structure then the summary could be less detailed
17:12:35 melwitt mriedem_luncharo: nevermind, my question is answered in your review comment
17:14:01 leakypipes sean-k-mooney: understood.
17:14:03 leakypipes thx
17:23:55 openstackgerrit Merged openstack/nova stable/pike: Use instance.project_id when creating request specs for old instances https://review.openstack.org/529384
17:24:00 openstackgerrit Merged openstack/nova stable/pike: Workaround missing RequestSpec.project_id when moving an instance https://review.openstack.org/529385
17:25:03 fried_rice leakypipes Sorry, was otp. My understanding was that we did intend to include resource *classes* that weren't part of the request. I don't know if we talked about including other unused providers in the same tree. I don't see the harm in doing it, and rather have it and not need it than need it and not have it. But no strong feelings either way.
17:25:49 fried_rice And to be clear, that's in the provider summaries, *not* in the allocations portion.
17:30:07 leakypipes fried_rice: ack
17:30:26 openstackgerrit Ed Leafe proposed openstack/nova master: Fix race condition in retrying migrations https://review.openstack.org/531022
17:30:27 openstackgerrit Ed Leafe proposed openstack/nova master: Change compute RPC to use alternates for resize https://review.openstack.org/526436
17:33:40 sean-k-mooney fried_rice: leakypipes haveing the extra info in the summery could help debuging i guess but do ye enviosion end users reading them directly? e.g. would it be stored in the instance info when you do an openstack server show like the cached flavor or something similar?
17:34:44 sean-k-mooney leakypipes: fried_rice basically is the summary for people or python?
17:35:18 leakypipes sean-k-mooney: no, this is an admin-only API endpoint that is only for the scheduler to make better decisions. none of this information is available to end users.
17:35:35 openstackgerrit Jay Pipes proposed openstack/nova master: func tests for nested providers in alloc candidate https://review.openstack.org/531443
17:35:55 leakypipes fried_rice, figleaf, sean-k-mooney: thou comments welcome... ^^
17:36:53 sean-k-mooney leakypipes: i like that the fist think i see is an ascii diagram of what is about to be tested :)
17:37:09 leakypipes sean-k-mooney: you can thank gibi_away for that practice :)
17:41:31 sean-k-mooney leakypipes: out of scope of queens but rocky+ could the VCPU and memory_mb(or perhaps just hugepages) inventories be moved to the numa node resouce providers?
17:42:22 leakypipes sean-k-mooney: possible. definitely something we need to be (very carefully) handle and discuss in Dublion.
17:42:25 leakypipes or even Dublin
17:44:45 sean-k-mooney memory_mb is basically the non hugepage memory(e.g. 4k pages) which is basically managed by the kernel auptomatically so may that shoudl stay at the compute node level but vcpus i think would be a gould fit for the numa resouce provider. im just not sure how straight forword it would be do do the db migration required.
17:45:53 leakypipes sean-k-mooney: right
17:46:15 cdent db migration? presumably the resource tracker would intead choose to change the inventory for its tree
17:46:40 cdent we _really_ need to not think of placement as authoritative on that stuff. reality should be.
17:46:54 cdent with placement second
17:47:08 cdent ideally reality should be able to recover all of placement
17:47:24 leakypipes fried_rice, figleaf: do you think it would be better to have an entirely different code path in the allocation candidates code when we notice that there are provider trees? you know how we currently have a fast-path for when there are no sharing providers? should we do the same kind of thing for provider trees?
17:47:51 leakypipes cdent: ^^
17:48:04 cdent my initial reaction is "yes"
17:48:13 leakypipes didn't know you were back from your walkabout... ;)
17:48:17 sean-k-mooney cdent: well what i ment by db migration was going from 1 inventory of all the cpus to n, one per numa node and updating the inventries to match what were actully used
17:48:34 cdent my walkabout was "drive home from the place where that dog was"
17:48:35 leakypipes cdent: yes, as in "separate code path for the provider tree deployments"?
17:49:04 cdent yeah, if only to make it more scrutable
17:49:08 leakypipes ack
17:49:15 leakypipes that was my initial thought as well.
17:50:13 cdent I still kinda wish, however, that there was one single magical sql expression (that we never have to see but we know works) that did the whole thing
17:50:20 sean-k-mooney leakypipes: as long as the second code path is not like cells v1 and solves a proablem maybe.
17:51:06 leakypipes sean-k-mooney: no, it would be just fancier SQL/Python to determine the results.
17:51:22 leakypipes cdent: there is... but it would be inscrutable...
17:51:34 cdent thus "that we never have to see"
17:51:37 leakypipes heh
17:51:54 leakypipes always remember, all of OpenStack can be rewritten entirely in SQL.
17:52:07 cdent as well as rewritten without any
17:52:08 leakypipes or Golang.
17:52:14 sean-k-mooney leakypipes: you could tripple your consluting fee to debug it if you wrote it lol
17:52:29 leakypipes sean-k-mooney: zero times three is zero. :)
17:54:16 SamYaple leakypipes: you mispelled rust
17:54:21 sean-k-mooney you know if you joke about rewriting openstack in golang 3 times it might summon the Ciao gods...
17:54:26 leakypipes SamYaple: :)
17:54:42 leakypipes SamYaple: you misspelled misspelled.
17:54:51 leakypipes :P
17:54:51 SamYaple i speel with the best of em
17:55:07 leakypipes sean-k-mooney: Beetlejuice!
17:55:38 SamYaple i did start on a libvirt replacement in rust (not 100% replacement, just spawn/monitor qemu process replacement)
17:55:50 SamYaple rust is suuuuuuuper ugly syntax wise
17:56:01 sean-k-mooney SamYaple: danial berage is considing that too. or golang
17:56:19 SamYaple im pretty sure i pushed *something* to git about it
17:56:29 SamYaple i had it spawning and interfacing with the monitor socket
17:56:52 SamYaple https://github.com/SamYaple/qemuwatch
17:57:03 SamYaple sean-k-mooney: i would love to not have to use libvirt anymore
17:57:13 sean-k-mooney SamYaple: there was a faily active thread on the libvirt mailing list back in december not sure if they reached a decision or not
17:57:37 leakypipes SamYaple: libvirt is a fine library. except for the XML. :)
17:57:50 SamYaple leakypipes: and the 500MB of deps needed to compile it
17:58:02 leakypipes SamYaple: the XML deps you mean, yeah?
17:58:21 SamYaple probably. i never dug into why libvirt was such a pita to build from source
17:58:24 SamYaple probably xml :)
17:59:33 SamYaple in truth, i was writing the above to spawn vms in kubernetes, libvirt is a bit overkill for that scenario (container per vm)
17:59:53 SamYaple but i stopped working on it because there are 1001 projects doing that right now and i prefer to get a more openstack native solution going
18:00:20 sean-k-mooney i havent done anything to maintain this since i gave it rodolfo to upstream but this made it less of a pain https://github.com/openstack/devstack-plugin-libvirt-qemu
18:01:06 sean-k-mooney i mean you are replaceing the pain of building libvirt from source with using devstack but at least you get to chose your poision

Earlier   Later