| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-22 | |||
| 14:32:04 | efried | but yeah, I didn't vote because I wasn't sure if we were now intending to actually merge it or if it was still just for people to debug with. | |
| 14:33:25 | mriedem | i just replied | |
| 14:33:40 | mriedem | if you want to wait for awaugama to recreate and say, yes this helped, then let's wait | |
| 14:34:20 | awaugama | owalsh: can you help with pulling in a patch from upstream into a build for extra logging? | |
| 14:39:31 | openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: OVA and StrOpt images as VM templates https://review.openstack.org/609736 | |
| 15:04:40 | mriedem | stephenfin: lyarwood: https://review.openstack.org/#/c/587013/15/nova/tests/fixtures.py@1728 | |
| 15:04:43 | mriedem | am i missing something? | |
| 15:23:43 | melwitt | ||
| 16:02:22 | stephenfin | mriedem: Yeah, but I've confused you in addressing your nit here https://review.openstack.org/#/c/587013/12/nova/tests/fixtures.py@1721 | |
| 16:02:51 | stephenfin | mriedem: _attachment_id on that line is actually the two item tuple | |
| 16:03:13 | stephenfin | mriedem: See https://review.openstack.org/#/c/587013/15/nova/tests/fixtures.py@1711 | |
| 16:04:17 | openstackgerrit | sean mooney proposed openstack/nova master: harden placement init under wsgi https://review.openstack.org/610034 | |
| 16:04:31 | stephenfin | and two lines above that, where we explode it out into the two item tuple. I guess that should go back to be called 'attachment' | |
| 16:07:15 | mriedem | ok commented | |
| 16:07:26 | mriedem | i'd like that re-worked because the variable naming is really confusing | |
| 16:08:24 | stephenfin | Agreed. It was called 'attachment' before but I guess that isn't much better | |
| 16:20:54 | openstackgerrit | Merged openstack/nova master: Remove more code related to extensions and testing https://review.openstack.org/607088 | |
| 16:21:01 | openstackgerrit | Merged openstack/nova master: Log the operation when updating generation in ProviderTree https://review.openstack.org/597553 | |
| 16:21:19 | openstackgerrit | Stephen Finucane proposed openstack/nova master: fixtures: Track volume attachments within CinderFixtureNewAttachFlow https://review.openstack.org/587013 | |
| 16:21:20 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add regression test for bug#1784353 https://review.openstack.org/587014 | |
| 16:21:20 | openstackgerrit | Stephen Finucane proposed openstack/nova master: conductor: Recreate volume attachments during a reschedule https://review.openstack.org/587071 | |
| 16:39:12 | openstackgerrit | sean mooney proposed openstack/nova master: harden placement init under wsgi https://review.openstack.org/610034 | |
| 16:41:07 | openstackgerrit | Matthew Booth proposed openstack/nova master: Add regression test for bug 1550919 https://review.openstack.org/591733 | |
| 16:41:07 | openstack | bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9) | |
| 16:43:38 | mnaser | fyi -- if anyone from the nova team hasn't seen this (yet): http://lists.openstack.org/pipermail/openstack-dev/2018-October/135941.html | |
| 16:43:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix jsonutils.to_primitive UserWarning https://review.openstack.org/612445 | |
| 16:44:06 | spatel | sean-k-mooney: you there? | |
| 16:45:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix jsonutils.to_primitive UserWarning https://review.openstack.org/612445 | |
| 16:47:18 | sean-k-mooney | spatel: i was away for a minute but im back | |
| 16:47:49 | spatel | On 64GB compute node i have configure huge page "hugepagesz=2M hugepages=27000 transparent_hugepage=never" | |
| 16:48:05 | spatel | based on calculation around 55GB huge pages | |
| 16:48:34 | spatel | is that ok or i should squeeze more pages and keep 4GB for host only | |
| 16:49:38 | sean-k-mooney | i think that is ok you could squees it down to 6 or 4GB for the host but without data to support that it does not need more i would start with leaveing 8GB free | |
| 16:50:22 | spatel | okay! let me go with 8GB free for host | |
| 16:51:21 | sean-k-mooney | spatel: "wasting" 2-4GB by not allocating extra hugepages is still proably better then having the OOM killer trigger because you were too agressive | |
| 16:51:48 | spatel | totally valid point ++ | |
| 16:52:07 | sean-k-mooney | as you start to operate the cluster over time and you mesure/log the ram usage you will have data that can let you make a more informed desision | |
| 16:52:52 | spatel | also if i run some agent for data collection or collected / snmp etc.. they will suck some memory too | |
| 16:53:02 | sean-k-mooney | yep | |
| 16:53:35 | sean-k-mooney | 4Gs might be fine without loging but adding collectd or someing else might bump it to 5Gs | |
| 16:53:59 | spatel | see its alway helpful to have second set of eye :) | |
| 16:54:47 | spatel | Thanks! | |
| 16:54:59 | spatel | i am going with 8G everywhere now! | |
| 16:55:52 | sean-k-mooney | leaving 8G free on a host with 32GB is a little painful but 8G on a host with 256GB of ram is perfectly resonable. finding a blance will depend on your workload so you will figure out what makes sense once you deploy and mesure your workload | |
| 16:56:54 | spatel | yup! i am slowly learning all equation and tuning compute nodes based on hardware specs | |
| 16:57:07 | spatel | Do you disable "specter" in kernel ? | |
| 16:57:19 | spatel | I heard it hit you CPU performance | |
| 16:57:32 | sean-k-mooney | disable the mitigations or disable the speculation | |
| 16:57:58 | sean-k-mooney | spatel: that really depends on your threat model. | |
| 16:58:06 | spatel | This is what i am doing on my compute nodes "spectre_v2=off nopti pti=off nospec_store_bypass_disable spec_store_bypass_disable=off l1tf=off" | |
| 16:59:12 | sean-k-mooney | spatel: that will give you more performance but also mean your system is vulnerbale to specter attacks | |
| 16:59:28 | sean-k-mooney | spatel: that said you have a private cloud correct | |
| 16:59:47 | spatel | yes totally private and we are not holding any PI data of any personal | |
| 16:59:53 | sean-k-mooney | spatel: so you controll all application that are running on the cloud e.g. there are no untrsed applications | |
| 16:59:59 | spatel | we are just streaming voip data | |
| 17:00:01 | mriedem | mnaser: replied | |
| 17:00:47 | sean-k-mooney | spatel: well if the lack of mitigations is fine based on your treat model and you know the implications of truning them off then that is your choice to make. | |
| 17:01:07 | sean-k-mooney | spatel: it will certenly impove the performnce of some workloads | |
| 17:01:32 | spatel | yup! i have noticed performance in benchmark based on that i have disabled | |
| 17:04:04 | artom | melwitt, did you get a chance to look over https://review.openstack.org/#/c/610739/1? The patch on top (which is the "real" bugfix is already +W) | |
| 17:05:19 | melwitt | artom: oh, yes. I started reviewing it and then didn't finish. sorry about that. I'll review it in a few | |
| 17:05:32 | artom | melwitt, no worries. And thanks! | |
| 17:09:30 | mriedem | pretty sure this is invalid https://bugs.launchpad.net/nova/+bug/1799186 | |
| 17:09:30 | openstack | Launchpad bug 1799186 in OpenStack Compute (nova) "Queens compute node is not compatible with Pike Controller node" [Undecided,New] | |
| 17:12:57 | dansmith | mriedem: definitely | |
| 17:13:01 | dansmith | mriedem: newer compute than control | |
| 17:13:29 | mriedem | the whole thing is confused, | |
| 17:13:30 | mriedem | "The RPC version of Pike compute is not supported with RPC version of Queens controller." | |
| 17:13:38 | mriedem | they just aren't doing the rpc version pin setting | |
| 17:13:40 | mriedem | i linked in docs | |
| 17:17:34 | openstackgerrit | Dan Smith proposed openstack/nova master: Make CellDatabases fixture reentrant https://review.openstack.org/611665 | |
| 17:17:34 | openstackgerrit | Dan Smith proposed openstack/nova master: Modify get_by_cell_and_project() to get_not_deleted_by_cell_and_projects() https://review.openstack.org/607663 | |
| 17:17:35 | openstackgerrit | Dan Smith proposed openstack/nova master: Minimal construct plumbing for nova list when a cell is down https://review.openstack.org/567785 | |
| 17:17:35 | openstackgerrit | Dan Smith proposed openstack/nova master: Refactor scatter-gather utility to return exception objects https://review.openstack.org/607934 | |
| 17:17:36 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova show when a cell is down https://review.openstack.org/591658 | |
| 17:17:36 | openstackgerrit | Dan Smith proposed openstack/nova master: Return a minimal construct for nova service-list when a cell is down https://review.openstack.org/584829 | |
| 17:56:12 | dansmith | mriedem: on that metadata thread from the ML, we shouldn't be doing the join thing that multiplies instances * metadata * sysmeta row counts | |
| 17:56:26 | dansmith | because we fetch the metadata bits separate and explicitly | |
| 17:56:34 | AJaeger | nova cores, I have two changes for your zuul config up - please review https://review.openstack.org/610271 and https://review.openstack.org/612230 . The first converts a job to new Zuul v3 tempest job, the second renames an ill-named job. If you have questions, I'm around to answer... | |
| 17:57:44 | dansmith | mriedem: because of this: https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L2244 | |
| 17:58:20 | AJaeger | nova cores, one more job change: https://review.openstack.org/610141 | |
| 18:12:24 | mriedem | dansmith: that's when pulling an instance list though, yeah? the metadata api isn't listing instances. | |
| 18:12:51 | dansmith | hmm, is it not on get too? | |
| 18:13:48 | dansmith | okay I guess not | |
| 18:13:54 | dansmith | we use it from a bunch of places but not there | |
| 18:14:06 | dansmith | I guess because it was always that instances * meta * sysmeta made it big(ger) | |
| 18:14:47 | dansmith | so, makes sense I guess | |
| 18:15:21 | mriedem | as far as i can see from a quick search i just don't think we need to pre-load system_metadata in the meta api | |
| 18:15:33 | dansmith | we did at the time, because of flavors | |
| 18:16:19 | mriedem | hmm, ok, and that's in case of 2007-08-29 being requested i guess | |
| 18:16:22 | mriedem | to return the flavor name | |
| 18:16:26 | mriedem | well, anything after 2007-08-29 | |
| 18:17:41 | AJaeger | mriedem: thanks for reviews! | |
| 18:18:58 | dansmith | mriedem: so I guess I'd just say we could/should just convert single-instance to do the same fill approach, as the real exposure is the user putting a ton of metadata items in (which they control) and making it get big | |
| 18:24:38 | openstackgerrit | sean mooney proposed openstack/os-vif master: add support for generic tap device plug https://review.openstack.org/602384 | |
| 18:32:32 | dansmith | mriedem: commented on this: https://review.openstack.org/#/c/602174/13 | |
| 18:32:58 | dansmith | mriedem: ready to +2 either way, but seems like a quick removal of the related bug and a host assertion can be done easily if you're up for it | |
| 18:34:23 | mriedem | i can | |
| 18:35:15 | mriedem | note the coincidental fun with forcing the evacuate to bypass the ComputeFilter https://review.openstack.org/#/c/602174/13/gate/test_evacuate.sh@56 | |
| 18:36:57 | dansmith | what about it? | |