| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-22 | |||
| 16:49:38 | sean-k-mooney | i think that is ok you could squees it down to 6 or 4GB for the host but without data to support that it does not need more i would start with leaveing 8GB free | |
| 16:50:22 | spatel | okay! let me go with 8GB free for host | |
| 16:51:21 | sean-k-mooney | spatel: "wasting" 2-4GB by not allocating extra hugepages is still proably better then having the OOM killer trigger because you were too agressive | |
| 16:51:48 | spatel | totally valid point ++ | |
| 16:52:07 | sean-k-mooney | as you start to operate the cluster over time and you mesure/log the ram usage you will have data that can let you make a more informed desision | |
| 16:52:52 | spatel | also if i run some agent for data collection or collected / snmp etc.. they will suck some memory too | |
| 16:53:02 | sean-k-mooney | yep | |
| 16:53:35 | sean-k-mooney | 4Gs might be fine without loging but adding collectd or someing else might bump it to 5Gs | |
| 16:53:59 | spatel | see its alway helpful to have second set of eye :) | |
| 16:54:47 | spatel | Thanks! | |
| 16:54:59 | spatel | i am going with 8G everywhere now! | |
| 16:55:52 | sean-k-mooney | leaving 8G free on a host with 32GB is a little painful but 8G on a host with 256GB of ram is perfectly resonable. finding a blance will depend on your workload so you will figure out what makes sense once you deploy and mesure your workload | |
| 16:56:54 | spatel | yup! i am slowly learning all equation and tuning compute nodes based on hardware specs | |
| 16:57:07 | spatel | Do you disable "specter" in kernel ? | |
| 16:57:19 | spatel | I heard it hit you CPU performance | |
| 16:57:32 | sean-k-mooney | disable the mitigations or disable the speculation | |
| 16:57:58 | sean-k-mooney | spatel: that really depends on your threat model. | |
| 16:58:06 | spatel | This is what i am doing on my compute nodes "spectre_v2=off nopti pti=off nospec_store_bypass_disable spec_store_bypass_disable=off l1tf=off" | |
| 16:59:12 | sean-k-mooney | spatel: that will give you more performance but also mean your system is vulnerbale to specter attacks | |
| 16:59:28 | sean-k-mooney | spatel: that said you have a private cloud correct | |
| 16:59:47 | spatel | yes totally private and we are not holding any PI data of any personal | |
| 16:59:53 | sean-k-mooney | spatel: so you controll all application that are running on the cloud e.g. there are no untrsed applications | |
| 16:59:59 | spatel | we are just streaming voip data | |
| 17:00:01 | mriedem | mnaser: replied | |
| 17:00:47 | sean-k-mooney | spatel: well if the lack of mitigations is fine based on your treat model and you know the implications of truning them off then that is your choice to make. | |
| 17:01:07 | sean-k-mooney | spatel: it will certenly impove the performnce of some workloads | |
| 17:01:32 | spatel | yup! i have noticed performance in benchmark based on that i have disabled | |
| 17:04:04 | artom | melwitt, did you get a chance to look over https://review.openstack.org/#/c/610739/1? The patch on top (which is the "real" bugfix is already +W) | |
| 17:05:19 | melwitt | artom: oh, yes. I started reviewing it and then didn't finish. sorry about that. I'll review it in a few | |
| 17:05:32 | artom | melwitt, no worries. And thanks! | |
| 17:09:30 | openstack | Launchpad bug 1799186 in OpenStack Compute (nova) "Queens compute node is not compatible with Pike Controller node" [Undecided,New] | |
| 17:09:30 | mriedem | pretty sure this is invalid https://bugs.launchpad.net/nova/+bug/1799186 | |
| 17:12:57 | dansmith | mriedem: definitely | |
| 17:13:01 | dansmith | mriedem: newer compute than control | |
| 17:13:29 | mriedem | the whole thing is confused, | |
| 17:13:30 | mriedem | "The RPC version of Pike compute is not supported with RPC version of Queens controller." | |
| 17:13:38 | mriedem | they just aren't doing the rpc version pin setting | |
| 17:13:40 | mriedem | i linked in docs | |
| 17:17:34 | openstackgerrit | Dan Smith proposed openstack/nova master: Modify get_by_cell_and_project() to get_not_deleted_by_cell_and_projects() https://review.openstack.org/607663 | |
| 17:17:34 | openstackgerrit | Dan Smith proposed openstack/nova master: Make CellDatabases fixture reentrant https://review.openstack.org/611665 | |
| 17:17:35 | openstackgerrit | Dan Smith proposed openstack/nova master: Refactor scatter-gather utility to return exception objects https://review.openstack.org/607934 | |
| 17:17:35 | openstackgerrit | Dan Smith proposed openstack/nova master: Minimal construct plumbing for nova list when a cell is down https://review.openstack.org/567785 | |
| 17:17:36 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova service-list when a cell is down https://review.openstack.org/584829 | |
| 17:17:36 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova show when a cell is down https://review.openstack.org/591658 | |
| 17:56:12 | dansmith | mriedem: on that metadata thread from the ML, we shouldn't be doing the join thing that multiplies instances * metadata * sysmeta row counts | |
| 17:56:26 | dansmith | because we fetch the metadata bits separate and explicitly | |
| 17:56:34 | AJaeger | nova cores, I have two changes for your zuul config up - please review https://review.openstack.org/610271 and https://review.openstack.org/612230 . The first converts a job to new Zuul v3 tempest job, the second renames an ill-named job. If you have questions, I'm around to answer... | |
| 17:57:44 | dansmith | mriedem: because of this: https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L2244 | |
| 17:58:20 | AJaeger | nova cores, one more job change: https://review.openstack.org/610141 | |
| 18:12:24 | mriedem | dansmith: that's when pulling an instance list though, yeah? the metadata api isn't listing instances. | |
| 18:12:51 | dansmith | hmm, is it not on get too? | |
| 18:13:48 | dansmith | okay I guess not | |
| 18:13:54 | dansmith | we use it from a bunch of places but not there | |
| 18:14:06 | dansmith | I guess because it was always that instances * meta * sysmeta made it big(ger) | |
| 18:14:47 | dansmith | so, makes sense I guess | |
| 18:15:21 | mriedem | as far as i can see from a quick search i just don't think we need to pre-load system_metadata in the meta api | |
| 18:15:33 | dansmith | we did at the time, because of flavors | |
| 18:16:19 | mriedem | hmm, ok, and that's in case of 2007-08-29 being requested i guess | |
| 18:16:22 | mriedem | to return the flavor name | |
| 18:16:26 | mriedem | well, anything after 2007-08-29 | |
| 18:17:41 | AJaeger | mriedem: thanks for reviews! | |
| 18:18:58 | dansmith | mriedem: so I guess I'd just say we could/should just convert single-instance to do the same fill approach, as the real exposure is the user putting a ton of metadata items in (which they control) and making it get big | |
| 18:24:38 | openstackgerrit | sean mooney proposed openstack/os-vif master: add support for generic tap device plug https://review.openstack.org/602384 | |
| 18:32:32 | dansmith | mriedem: commented on this: https://review.openstack.org/#/c/602174/13 | |
| 18:32:58 | dansmith | mriedem: ready to +2 either way, but seems like a quick removal of the related bug and a host assertion can be done easily if you're up for it | |
| 18:34:23 | mriedem | i can | |
| 18:35:15 | mriedem | note the coincidental fun with forcing the evacuate to bypass the ComputeFilter https://review.openstack.org/#/c/602174/13/gate/test_evacuate.sh@56 | |
| 18:36:57 | dansmith | what about it? | |
| 18:37:14 | mriedem | related to what was being discussed in gibi's spec about deprecating the force flag, | |
| 18:37:32 | mriedem | and when people might actually use it (to bypass the ComputeFilter) | |
| 18:37:44 | mriedem | but in this case, it's just negatively trying to force us to the down host | |
| 18:37:44 | dansmith | oh | |
| 18:42:11 | dansmith | okay I'm not sure if I'm missing something, | |
| 18:42:20 | markguz_ | Hi nova peeps. Question about nova-compute on pike , on ppc64le on ubuntu 16.04LTS. | |
| 18:42:22 | dansmith | you're downing libvirt so the current host is marked as down, | |
| 18:42:34 | dansmith | and then making sure that we fail even when forcing it right? | |
| 18:42:41 | markguz_ | after running updates recently i noticed that i couldn't spin up vms any longer | |
| 18:42:46 | markguz_ | getting this error qemu-system-ppc64: failed to initialize KVM: Invalid argument | |
| 18:43:08 | markguz_ | any other ppc64le users out there having issues on ubuntu? | |
| 18:44:27 | AJaeger | thanks, melwitt ! | |
| 18:44:52 | dansmith | markguz_: see topic | |
| 18:45:00 | melwitt | AJaeger: thanks for the patches | |
| 18:45:13 | markguz_ | dansmith: i see. sorry! | |
| 18:47:17 | sean-k-mooney | markguz_: #openstack may be more helpful but it sounds like you may have some packaging/depency issues. not many people use powerkvm as far as i know | |
| 18:49:26 | markguz_ | sean-k-mooney: could be, but i'm using the official ubuntu stuff, not making my own packages. | |
| 18:50:56 | sean-k-mooney | markguz_: well i was more thinking you kvm module paramters file my have somthing that was invalid. anyway this seams more like a distro issue then nova. sorry i can be of more help. | |
| 18:57:14 | mmedvede | markguz_: Hi, I can help. Not sure what is the best place to discuss, probably #openstack is as good place as any | |
| 18:58:05 | markguz_ | mmedvede: i'm logged into #openstack | |
| 18:58:30 | mriedem | dansmith: "and then making sure that we fail even when forcing it right?" yes | |
| 18:58:35 | mriedem | it's a negative scenario, | |
| 18:58:39 | dansmith | right | |
| 18:58:43 | mriedem | then bring the target back up and evacuate to it | |
| 19:03:34 | mriedem | is lee still out for awhile? | |
| 19:03:44 | dansmith | I think another week at least, IIRC | |
| 19:05:17 | mriedem | ok | |
| 19:07:03 | dansmith | back wednesday of next week it looks like | |
| 19:33:11 | openstackgerrit | Merged openstack/nova master: Move live_migration.pre.start to the start of the method https://review.openstack.org/610739 | |
| 19:33:19 | openstackgerrit | Merged openstack/nova master: Ensure attachment cleanup on failure in driver.pre_live_migration https://review.openstack.org/587439 | |
| 19:35:45 | openstackgerrit | Matt Riedemann proposed openstack/nova master: conductor: Recreate volume attachments during a reschedule https://review.openstack.org/587071 | |
| 19:35:57 | mriedem | stephenfin: addressed my nits in ^ and i'll +2 | |