| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-15 | |||
| 11:36:25 | peter-hamilton | johnthetubaguy: https://review.openstack.org/#/c/357202/ and https://review.openstack.org/#/c/457678/ if you have time | |
| 11:44:47 | murugesh | Hi Guys | |
| 11:45:03 | murugesh | Is soembody there | |
| 12:13:53 | ArchiFleKs | hi, when using configdrive with nova, are the userdata still written inside the database ? it seems like it, I have a configdrive but still can retrieve userdata with tha metadata api. I thought i could use configdrive to bypass the 64k limit for user-data | |
| 12:30:02 | sdague | ArchiFleKs: the 64k limit is API side enforced, back end implementation doesn't matter | |
| 12:35:59 | ArchiFleKs | sdague: thanks, so there is not way to extend userdata size ? (https://bugs.launchpad.net/nova/+bug/1669979) , even if the database field is bigger. What is the right approach from your opinion ? heat-sofwaredeployment ? | |
| 12:35:59 | openstack | Launchpad bug 1669979 in OpenStack Compute (nova) "failed to boot instance when user data is larger than 65535 bytes" [Undecided,Incomplete] - Assigned to wind (better) | |
| 12:38:42 | johnthetubaguy | peter-hamilton: sorry between jobs at the moment, so not going to be super active review wise right now, migrating all my stuff to personal machines at the moment | |
| 12:40:21 | ArchiFleKs | sdague: because config drive limit is 64mb | |
| 12:40:57 | sdague | ArchiFleKs: expanding userdata size would definitely be a spec, there are lots of fall out implications of that, including memory bloat in the API server | |
| 12:42:59 | sdague | ArchiFleKs: it also looks like zaneb recommended for folks to go in a different direction here - https://ask.openstack.org/en/question/105427/how-to-increase-user-data-limit-beyond-64k/ | |
| 12:54:57 | abhi89 | johnthetubaguy: Hi John.. can you please review https://review.openstack.org/#/c/462561/ | |
| 12:55:06 | peter-hamilton | johnthetubaguy: not a problem, just trying to generate interest | |
| 13:03:09 | openstackgerrit | Béla Vancsics proposed openstack/nova master: Transform instance.trigger_crash_dump notification https://review.openstack.org/396225 | |
| 13:03:20 | openstackgerrit | Dave Johnston proposed openstack/nova master: DocImpact Adds man page for nova-manage db online_data_migrations explaining the return codes https://review.openstack.org/464634 | |
| 13:07:19 | bauzas | ArchiFleKs: so, while you could use configdrivers for a quick resolution, I think you should possibly use other ways (like using a config management tool) for passing more to the instance in case 2^24 chars is not enough for you | |
| 13:07:42 | lyarwood | any fellow f25 devstack users seeing this trace when sync'ing the api_db? http://paste.openstack.org/show/609562/ | |
| 13:15:12 | ArchiFleKs | bauzas: yes but there no resolution at all because nova is still using the DB to store the data anyway and the 64k limit is still there. This is for magnum which uses heat, i think we should test heat-software-deployment so we do not depend on other config tools | |
| 13:17:14 | ArchiFleKs | sdague: there is a way for atomic to use swift as a storage for configdrive data, I know AWS also has a limit that can be bypass by storing stack template inside an S3 bucket. This might be a solution to not add issues to nova database | |
| 13:17:26 | ArchiFleKs | s/atomic/ironc | |
| 13:17:31 | ArchiFleKs | *ironic | |
| 13:17:51 | bauzas | ArchiFleKs: the MediumText column accepts 2^24 chars, ie. 16M IIUC | |
| 13:19:30 | bauzas | ArchiFleKs: like sdague said, the 64K limit is an API one | |
| 13:19:49 | bauzas | so, if you're using configdrives, you'll get a max size of 16M | |
| 13:19:57 | bauzas | AFAIUI | |
| 13:20:12 | stvnoyes1 | mriedem: ildikov: please review cinder v3/live migration WIP when you have some time - https://review.openstack.org/#/c/463987/ thanks... | |
| 13:22:22 | ArchiFleKs | bauzas: yes but my point is that there is no way to bypass it unless this is change (https://github.com/openstack/nova/blob/master/nova/compute/api.py#L100) even when using confidrive with nova, right ? | |
| 13:26:20 | bauzas | ArchiFleKs: if you're using a configdrive, then it bypasses the metadata service https://docs.openstack.org/user-guide/cli-config-drive.html | |
| 13:26:53 | bauzas | and http://cloudinit.readthedocs.io/en/latest/topics/datasources/configdrive.html | |
| 13:27:28 | bauzas | anyway, need to go find my daughter \o | |
| 13:29:12 | mriedem | stvnoyes1: ack | |
| 13:29:37 | bauzas | ArchiFleKs: ah, nevermind, you mean when passing the userdata for POST /servers then | |
| 13:31:35 | mriedem | edleafe: i won't be able to make the scheduler meeting in 30 minutes, not sure how much was going to be talked about besides recaps from summit discussions | |
| 13:33:06 | mriedem | edleafe: for the most part claims in the scheduler/conductor had agreement from ops, although they still wanted us to pass the chosen hosts down to the cell conductor for retries | |
| 13:33:31 | ArchiFleKs | bauzas: yes I mean even if i enable configdrive inside a Nova::Server heat resource, it will still use the API and apply the limits | |
| 13:33:46 | mriedem | edleafe: and https://review.openstack.org/#/c/463597/ came up as a side item during that discussion, as just a good to have | |
| 13:37:45 | mriedem | johnthetubaguy: doing summit recaps is going to be weird this time because there weren't a set of vertical 'nova' tracks; were you going to provide a recap of the 3 vm/bm sessions to the ML/ | |
| 13:37:46 | mriedem | ? | |
| 13:37:49 | mriedem | b/c i didn't make it to all of them | |
| 13:39:16 | edleafe | mriedem: thanks for the info. I'm sure we will discuss all of that at the meeting. | |
| 13:39:24 | edleafe | mriedem: I know I'll be asking about it :) | |
| 13:40:31 | mriedem | edleafe: cool; notes are in https://etherpad.openstack.org/p/BOS-forum-cellsv2-developer-community-coordination and https://etherpad.openstack.org/p/BOS-forum-move-claims-from-compute-to-scheduler | |
| 13:40:46 | mriedem | i'll try to do some high level recaps in the ML this week | |
| 13:41:26 | mriedem | Kevin_Zheng: were you going to update https://review.openstack.org/#/c/459593/ soon? | |
| 13:42:45 | mriedem | avolkov: btw thanks for the sqla make_url tip | |
| 13:43:32 | Kevin_Zheng | mriedem: yeah, I will update tomorrow, btw, I left one remaining comment about deleting tags, could you have a look? | |
| 13:43:34 | ildikov | stvnoyes1: on my list, thanks for the heads up | |
| 13:43:58 | mriedem | Kevin_Zheng: this? https://review.openstack.org/#/c/459593/10/nova/conductor/manager.py@1086 | |
| 13:44:28 | Kevin_Zheng | Ah yes | |
| 13:44:34 | mriedem | :) | |
| 13:45:52 | Kevin_Zheng | mriedem: thanks, we are flying to Suzhou tomorrow and I will update it tomorrow evening or the day after that | |
| 13:45:52 | edleafe | Scheduler subteam meeting in 15 minutes in #openstack-meeting-alt | |
| 13:47:19 | bauzas | edleafe: just in case you don't know, I'm working on updating my series | |
| 13:47:40 | bauzas | edleafe: please stop updating your ones, given I'd prefer you reviewing mine | |
| 13:50:20 | avolkov | mriedem: you're welcome | |
| 13:51:21 | mriedem | Kevin_Zheng: thanks | |
| 13:57:30 | edleafe | bauzas: I saw your notes | |
| 13:58:04 | bauzas | ack | |
| 14:02:49 | edleafe | Scheduler subteam meeting going on now in #openstack-meeting-alt | |
| 14:13:55 | lyarwood | slaweq_: https://bugs.launchpad.net/nova/+bug/1690819 FYI, I'm also seeing that in F25 | |
| 14:13:55 | openstack | Launchpad bug 1690819 in OpenStack Compute (nova) "nova-manage [..] api_db sync on F25 results in RuntimeError: no suitable implementation for this system" [Undecided,New] | |
| 14:23:11 | sdague | lyarwood: that's a blow up deep in monotonic | |
| 14:23:41 | sdague | lyarwood: I wonder why it's blowing up there | |
| 14:25:28 | lyarwood | sdague: yeah this is odd, stepping through this in a venv now and getting the following trace http://paste.openstack.org/show/609576/ | |
| 14:27:01 | openstackgerrit | Mathieu Gagné proposed openstack/nova master: Add ability to signal and perform online volume size change https://review.openstack.org/454322 | |
| 14:27:20 | artom | So, I tried find_library('foo') and it just returns False, without blowing up... | |
| 14:30:37 | bauzas | oh man | |
| 14:31:23 | bauzas | I was about to write a reservoir sampling method for getting a random subset of a long iterator | |
| 14:31:52 | bauzas | until I just discovered that we put all the hosts in memory after weighing those ! | |
| 14:32:04 | bauzas | so, weighed_hosts is a full list | |
| 14:32:19 | bauzas | I thought we were smarter | |
| 14:33:49 | bauzas | like getting weights for each of the items of an iterator and just sorting them by indexes instead of putting all in memory | |
| 14:33:55 | bauzas | fairly inefficient | |
| 14:36:01 | stvnoyes1 | nova core: review 'add Cinder v3 detach to shutdown_instance' - https://review.openstack.org/#/c/463987/ - it has a +2, but it needs a workflow +1. thanks. | |
| 14:36:10 | bauzas | oh man, even the filtering creates a list... | |
| 14:36:48 | bauzas | so, we're gentle with operators by using iterators when getting a full list from DB, but then we put that all in memory once we loop over filters | |
| 14:36:58 | bauzas | and then we do that again with weighers | |
| 14:37:12 | bauzas | so we basically duplicate the size in memory | |
| 14:38:14 | bauzas | ^ that could be a problem with large deployments | |
| 14:39:22 | lyarwood | sdague / artom ; I think it's https://github.com/eventlet/eventlet/issues/401 , I can't reproduce with eventlet 0.20.1 | |
| 14:41:19 | sean-k-mooney | sfinucan: hi o/, back from boston? | |
| 14:41:58 | sdague | lyarwood: yeh, that seems likely, I wonder why you are hitting it consistently but it's not showing up other places | |
| 14:46:34 | lyarwood | sdague: no idea, we have eventlet==0.21.0 in the gate at the moment | |
| 14:47:12 | sdague | lyarwood: I guess it's a race, I wonder if your env is on the slower or faster side of things | |
| 14:47:20 | sdague | just to figure out when things expose | |
| 14:49:07 | lyarwood | sdague: hopefully on the faster side vs the gate given it's the only heavy workload on my current pretty beefy host | |
| 14:50:36 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Make discover_hosts only query for unmapped ComputeNode records https://review.openstack.org/427903 | |
| 14:58:37 | mriedem | jaypipes: https://review.openstack.org/#/c/427903/ is the last piece in the 'make auto-discover_hosts faster' series | |
| 14:59:01 | jaypipes | mriedem: k, will review. currently on a plane out to China... | |
| 14:59:08 | jaypipes | mriedem: spotty internet.. | |
| 14:59:10 | mriedem | oh fun | |
| 15:01:14 | alex_xu | jaypipes: $20 for more than 10 hours flight, it is totally worth :) | |
| 15:03:13 | openstackgerrit | Monty Taylor proposed openstack/nova master: WIP Add description of server addresses objects https://review.openstack.org/464331 | |
| 15:05:07 | jaypipes | mriedem: done. | |
| 15:05:23 | mriedem | thanks | |
| 15:05:34 | jaypipes | alex_xu: still on way to LAX first, then the biggie. :) | |
| 15:06:59 | jaypipes | mriedem, sdague, anyone: seen this? from monotonic import monotonic as now # noqa | |
| 15:07:00 | jaypipes | RuntimeError: no suitable implementation for this system | |
| 15:07:00 | jaypipes | ||
| 15:07:00 | jaypipes | File "/home/jaypipes/src/git.openstack.org/openstack/nova/.tox/functional/local/lib/python2.7/site-packages/monotonic.py", line 169, in <module> | |