Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-10-02
19:47:36 sean-k-mooney cdent: i have started to use https://www.grammarly.com more when writing docs/commit message/release notes but i have not figured out a good way to integreate it or a spellchecker into my normal patch authoring workflow.
19:48:17 cdent ah, interesting, thanks
19:49:09 efried Technically "dysgraphia" is the writing part and "alexia" is the reading part. In a move which surely caused the classical-language-root scholars to tear out their hair, "dyslexia" became the portmanteau covering both.
19:50:19 efried My favorite is when sean-k-mooney uses "taught" for "thought", because it combines phonetic spelling with his Irish accent.
19:52:02 sean-k-mooney i was about to drink some coffee when i read that lol its ture i do that alot
19:55:47 edleafe Question on the retry flow for failed VM builds. Does the cast from superconductor to cell conductor use the same method in the cell conductor as the cast from the ComputeTaskAPI
19:56:04 edleafe 's cast to the cell conductor's build_instances?
19:56:24 edleafe I'm trying to determine if the cell conductor can tell if it's retrying or not
19:57:48 sean-k-mooney edleafe: it was my impression that reties would only propergate within the same cell as we did not want an upcall to the superconductor
19:58:40 edleafe sean-k-mooney: true. what I'm wondering is if the cell conductor can tell that the build is a retry or not
19:59:55 sean-k-mooney edleafe: good question dansmith would propably be the best person to ask. i would assume so but im not sure if that would chage anything other then the fact we are eliminating the fail hosts form the allocation candidates
20:00:32 dansmith edleafe: yeah because it's two operations
20:00:45 dansmith edleafe: the top level one is schedule_and_build_instances, the other is build_and_run or something like that
20:00:56 edleafe dansmith: thanks
20:01:07 edleafe rewriting the alternate hosts spec
20:01:21 edleafe totally spaced that the call to compute is a cast
20:01:59 mriedem hmm, if we deprecate personality files from the API, we would presumably also deprecate showing limits on personality files in the API too - and if the alternative for personality files is user_data and config drive, we don't have any quota limits on those - so would we need to add some?
20:02:36 mriedem edleafe: superconductor doesn't cast to cell conductor
20:03:38 edleafe mriedem: I was talking about the cell conductor -> compute. That's the cast
20:03:43 mriedem superconductor (schedule_and_build_instances) -> scheduler (select_destinations) -> superconductor (schedule_and_build_instances) -> compute (build_and_run_instance) -> cell conductor (retry = build_instances)
20:03:56 mriedem yeah then it's build_and_run_instance both ways
20:04:02 sean-k-mooney mriedem: well the user-data is stored perinsance correct so im not sure we need limits on it if its a fixed lenght string in the db
20:04:30 cdent ‘night all
20:04:32 mriedem sean-k-mooney: yeah, wasn't sure if that's why we had quota on injected files or what
20:04:36 mriedem i guess it's rate limiting
20:05:27 mriedem ooo we have 3 options for file injection quota
20:05:47 sean-k-mooney if its sored as a text field we may want a quota
20:06:13 sean-k-mooney injected files, injected file content bytes and ?
20:06:19 mriedem quota_injected_file_path_length
20:07:05 mriedem melwitt: AbsoluteResource just means it's global per deployment? or per project?
20:07:07 sean-k-mooney is that litrally just a path lenght ? e.g. a file system constrait
20:07:23 mriedem yes, default is 255 for the file path length
20:07:41 sean-k-mooney that the fat32 limit right
20:07:56 mriedem yeah i think so
20:11:16 mriedem god i've already forgotten how all of this quota checking code works already
20:11:43 sean-k-mooney looking at the instances table it looks like user_data mediumtext field which means its got a 16 Mib limit on mysql
20:12:11 sean-k-mooney *user_data is a mediumtext
20:14:39 mriedem ok it looks like this quota limit is per project/user
20:15:32 mriedem so by default you can have 5 files, at 10mb each
20:15:45 mriedem wait, no
20:15:46 mriedem kb
20:17:10 sean-k-mooney mriedem: yes it is. userdata however is handeled seperatly from that limit today
20:18:17 mriedem right, there is no quota limit on user_data
20:18:20 mriedem it's just the limit in the db per instance
20:18:25 mriedem which is as you said 16MB
20:18:30 sean-k-mooney yes
20:19:04 sean-k-mooney we could change that in the future or mysql could be we dont have a contract in place with the end user on this today
20:20:32 sean-k-mooney having a qouta on this in the future would not nessicalily be a bad thing it is sotre on you db node after all so it could cause issues if allowed to grow too large
20:35:05 dansmith mriedem: are you aware of any cells-based short circuiting around InstanceMapping?
20:35:17 dansmith in the api_samples_tests
20:35:46 dansmith in looking at some of them, they create a server, which exists in the cell db, but we never set the InstanceMapping.cell_mapping for those
20:35:59 dansmith ISTR these were problematic with cells for some reason, but I can't really remember why
20:37:34 mriedem not really
20:38:13 dansmith ah, maybe it's SingleCellSimple that's getting me
20:38:25 mriedem i was going to say, i thought there was a fixture we used, but don't see where it's set
20:38:28 dansmith yasss
20:39:38 dansmith yeah, the wrinkle is that the api sample tests use singlecellsimple unlike the other functional ones
20:39:53 mriedem oh i'm looking at newton still, derp
20:40:27 mriedem If1138331f3a46f5aed87e898ce19879a787d435f
20:40:51 dansmith I added a new CellMappingList method and so I have to put that in the fixture or we always get back an empty list
20:47:48 mriedem we said at the ptg that if we deprecated personality files from the api, and you can specify personality files during rebuild, that we'd allow passing new user_data during rebuild, but i'm not sure if we should
20:47:56 mriedem i think we just allow passing personality files during rebuild b/c we don't persist them
20:48:05 mriedem so that was a hack workaround for the lack of persistence,
20:48:15 mriedem so i'm not sure we should allow passing user_data to rebuild just b/c personality is gone
20:48:24 mriedem sdague: remember that discussion? ^
20:48:54 mriedem cfriesen: ^ you might care since you seem to love rebuild
20:49:39 sdague mriedem: you are right
20:50:52 openstackgerrit Dan Smith proposed openstack/nova master: Make get_instance_objects_sorted() be smart about cells https://review.openstack.org/509003
20:50:52 openstackgerrit Dan Smith proposed openstack/nova master: Add CellMapping.get_by_project_id() query method https://review.openstack.org/509002
20:51:29 sdague mriedem: though, I think people were using that as an end run
20:51:37 sdague we can't update user_data on instances, right?
20:51:48 dansmith mriedem: ^ that's the last cellsv2 perf optimization I had in my head to consider it legit usable for queens
20:52:04 mriedem sdague: no you can't specify user_data during update
20:52:20 mriedem i don't really know if people are using personality files on rebuild as a workaround for missing user_data
20:52:26 mriedem i don't know how people use our apis really
20:52:36 sean-k-mooney jaypipes: after nested resouce providers lands would you have any objection to seperating https://github.com/openstack/nova/blob/master/nova/objects/resource_provider.py into a placement-lib repo so it could be imported by neutron/os-vif for the ovo neutron port binding work and bandwith based scheduling?
20:53:02 efried os-placement ftw
20:53:18 mriedem dansmith: are you going to run https://review.openstack.org/#/c/509003/ through your performance tests, compared to before/after that change?
20:53:28 mriedem but on top of https://review.openstack.org/#/c/505418/
20:53:38 dansmith mriedem: I'd have to have lots of cells to make any difference
20:53:52 dansmith so I wasn't really planning to, unless you just want to make sure it's not super painful or something
20:53:58 jaypipes sean-k-mooney: yes, I would. those objects are not intended for use by anything outside placement. That said, I think the /nova/scheduler/client/report.py code can be moved into a placement lib
20:53:59 mriedem dansmith: couldn't you measure the relative difference by just having 1000 instances in cell1, all ACTIVE?
20:54:06 mriedem that would exclude cell0 at least
20:54:46 dansmith mriedem: you're trading one one-row api db result for one empty cell0 result
20:54:47 dansmith so I doubt you'd be able to measure it
20:55:17 dansmith if you really want, I could just synthesize a bunch of cells with two tenants or something
20:55:21 sean-k-mooney jaypipes: ok the usecase was to be able to define a placement request and a placement allocation object that could be used between nova and neutron
20:55:58 jaypipes sean-k-mooney: yeah, that's report.py, not resource_provider.py :)
20:56:06 sean-k-mooney jaypipes: i originally taught of having them defined in os-vif but taught that felt a litle weired as they could be used in cinder
20:56:32 jaypipes sean-k-mooney: note that objects in nova/objects/resource_provider.py do NOT get sent over the wire, ever.
20:56:42 jaypipes sean-k-mooney: and we intend to keep it that way :)
20:57:42 sean-k-mooney but they do map to the structure recived from the rest api?
20:58:11 sean-k-mooney currently neutron is manipulating raw json responces which is a little annoying
20:59:58 sean-k-mooney actully that code look kind of like what neutron is doing but i was hoping we could have a singel place for it that both nova and neutron used so they dont get out of sync
21:01:54 sean-k-mooney jaypipes: https://github.com/openstack/neutron/blob/b1dd13abfb121ca2a3e1cebc6bc2cef4a056f401/neutron/services/segments/placement_client.py is basically a stripped down version
21:05:23 jaypipes sean-k-mooney: right, and I'm agreeing with you that it would be good to have a placement-lib repo that would store common code for callers of the placement API :)
21:05:44 jaypipes sean-k-mooney: I'm just saying that that common code is the scheduler/client/report.py code and not the nova/objects/resource_provider.py code :)
21:07:21 sean-k-mooney i am glad we are in violent agreement :) ya that make senese this is the same comment code that would be imported in osc
21:08:49 sean-k-mooney does it make sense to look at createing placement-lib after nested resouce providers or are there other api level changes to plaement in queens that we should also wait for

Earlier   Later