| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-15 | |||
| 15:46:24 | sean-k-mooney | well docker yes | |
| 15:46:37 | MarkMielke | we didn't move up to RHEL 8 yes - will start to soon - but Docker being gone was a surprise to me | |
| 15:46:43 | sean-k-mooney | but redhat is not really encouraging use to work upstream in kolla | |
| 15:46:51 | sean-k-mooney | to move kolla ansible to podman | |
| 15:46:52 | MarkMielke | right | |
| 15:47:40 | sean-k-mooney | i was thinking about getting involved again to help with that but people internally didnt care that much so i focused on nova | |
| 15:47:56 | MarkMielke | only so much time in day :-) | |
| 15:48:16 | sean-k-mooney | they didnt disucrage me but suggested i look at getting involved in triplo instead | |
| 15:48:36 | sean-k-mooney | and i have no interst in that | |
| 15:48:44 | MarkMielke | at OpenStack conference, I thought it was amusing when they were talking about future of TripleO | |
| 15:48:57 | MarkMielke | and one of the Kayobe people spoke up "we've already done this" | |
| 15:49:35 | MarkMielke | it's fine though - community needs to develop ideas and then discover intersection of common objectives | |
| 15:49:54 | sean-k-mooney | well i mention trying to adopt kayobe when the topic of simplify triople o came up and it was more or less ignored | |
| 15:50:20 | sean-k-mooney | there is too much of a sunken cost falacy around triplo | |
| 15:50:32 | MarkMielke | yeah i see that | |
| 15:51:09 | MarkMielke | but despite working a bit in parallel and duplication of work | |
| 15:51:12 | sean-k-mooney | they have a vaild point that we would have to support migarate all existing customer to it | |
| 15:51:20 | MarkMielke | they're converging | |
| 15:51:28 | sean-k-mooney | but that seams simpler to solve then simplfying triple0 | |
| 15:51:51 | MarkMielke | if TripleO is using containers and Ansible for deployment by the end of it | |
| 15:51:55 | sean-k-mooney | they will diverge again when triplo moves to k8s | |
| 15:51:56 | MarkMielke | it's not really that different from Kayobe | |
| 15:52:17 | MarkMielke | more superficial than not | |
| 15:52:37 | MarkMielke | :-) | |
| 15:52:37 | MarkMielke | just taking the long way around to get to that point | |
| 15:53:03 | sean-k-mooney | well kayobe and kolla-ansible will not move to running the contaienr in k8s | |
| 15:53:11 | sean-k-mooney | so while the continer may be the same | |
| 15:53:24 | sean-k-mooney | all the tooling around upgrades and operation will diverge | |
| 15:53:34 | MarkMielke | yeah, the orchestration around how runc launches will be different | |
| 15:53:42 | MarkMielke | but fundamentally, it'll be the same technology doing the same thing underneath | |
| 15:54:24 | MarkMielke | two groups vehemently disagreeing | |
| 15:54:28 | MarkMielke | but fundamentally agreeing :-) | |
| 15:55:21 | MarkMielke | you can see how I am jaded :-) | |
| 15:56:15 | MarkMielke | i haven't used podman yet much - but the idea that it supports Kubernetes pods, | |
| 15:56:22 | MarkMielke | is interesting | |
| 15:56:34 | sean-k-mooney | well it does and it does not | |
| 15:56:54 | sean-k-mooney | it suport the cncf cri | |
| 15:57:23 | sean-k-mooney | it also provides cli compatiablity with docker | |
| 15:58:02 | sean-k-mooney | so you can use it to interact/manage pods but on its own it would do any of the orchstartion fo the pods that k8s does | |
| 15:58:12 | MarkMielke | probably with some limitations, it still means that something designer for Kubernetes | |
| 15:58:18 | MarkMielke | can almost work in stand-alone capacity | |
| 15:58:26 | sean-k-mooney | for example without the newst version it wont start the contaner after a host reboot | |
| 15:58:44 | MarkMielke | systemd :-) | |
| 15:58:50 | sean-k-mooney | the newest version will creat systemd files for you to implement docker restart policies | |
| 15:58:58 | sean-k-mooney | before that you had to do it by hand | |
| 15:59:07 | MarkMielke | as you mentioned earlier - | |
| 15:59:17 | MarkMielke | i like the systemd model of managing containers | |
| 15:59:22 | MarkMielke | vs the brainless auto-restart | |
| 15:59:35 | sean-k-mooney | where as the k8s kubelet would have done that via the podsepc | |
| 15:59:40 | MarkMielke | dependency management, as well as notification of when an application is up and healthy | |
| 16:00:03 | sean-k-mooney | so its more a tool for defineing pods on a local system without any of the orchetration | |
| 16:00:28 | MarkMielke | primarily for development stand-alone vs running in a production cluster with overhead | |
| 16:00:34 | MarkMielke | right? | |
| 16:00:52 | sean-k-mooney | kind of but it can be used as a building block for you orchelstartion | |
| 16:01:02 | sean-k-mooney | it does support heatbeat continer for examle | |
| 16:01:12 | sean-k-mooney | but no direct graphs fo dependce | |
| 16:01:25 | sean-k-mooney | that said systemd supports that so who know what will happnen | |
| 16:01:35 | MarkMielke | that's what i would mostly use systemd for | |
| 16:01:57 | sean-k-mooney | ya you are just missing the syntax in podman to express the depences | |
| 16:02:12 | sean-k-mooney | then rely on the before/after freatuers in systemd or targets | |
| 16:02:18 | sean-k-mooney | to tie it all toghter | |
| 16:02:29 | MarkMielke | today we cheat a bit... /etc/systemd/system/docker.conf.d/ lists dependencies, but they all dependencies for all contianers | |
| 16:02:47 | MarkMielke | so delay docker start until pre-requisites are met | |
| 16:02:59 | MarkMielke | it's not great for inter-container dependencies | |
| 16:04:05 | MarkMielke | i.e. PostgreSQL should be up before Jira | |
| 16:04:12 | MarkMielke | we mostly rely on PostgreSQL coming up really really fast :-) | |
| 16:04:31 | MarkMielke | and Jira, in Java, coming up slow | |
| 16:04:47 | sean-k-mooney | what i wanted to fix in kolla is that the ovsdb shoudl be up before the ovs-vswtichd starts | |
| 16:05:08 | MarkMielke | do you containerize those too? or you just mean in general | |
| 16:05:33 | sean-k-mooney | no ovs is containerised | |
| 16:05:53 | sean-k-mooney | it runs with net=host | |
| 16:05:57 | MarkMielke | interesting - I guess it's fine since the data plane is in kernel | |
| 16:06:06 | sean-k-mooney | not always | |
| 16:06:14 | sean-k-mooney | that is what i was trying to fix | |
| 16:06:29 | sean-k-mooney | if you are using ovs-dpdk where the dataplane is in userspace | |
| 16:06:34 | sean-k-mooney | the ovsdb need to be started first | |
| 16:06:36 | MarkMielke | ah | |
| 16:06:50 | sean-k-mooney | so that the vswtihcd can detect that dpdk is enabled when it starts up | |
| 16:07:31 | MarkMielke | ovs-dpdk... that's an interesting one for a related subject... | |
| 16:07:56 | MarkMielke | it's presumedly faster than kernel, but I also discovered that the Red Hat builds of Docker use seccomp | |
| 16:08:05 | MarkMielke | and seccomp normally triggers some of the more expensive Spectre mitigations | |
| 16:08:21 | MarkMielke | i think i have that right | |
| 16:08:27 | MarkMielke | do you disable that? | |
| 16:09:58 | sean-k-mooney | by default redhat does not tend to disabel security feature | |
| 16:10:10 | sean-k-mooney | im not sure it would impact dpdk performace it might | |
| 16:10:17 | sean-k-mooney | when we mesure something like that as an imparct | |
| 16:10:34 | sean-k-mooney | we usually docmet how to disable it and let it up to the custoemr | |
| 16:10:39 | MarkMielke | it disables indirect branch spectulation for the threads :-( | |
| 16:11:07 | MarkMielke | i started changing them to prctl instead of seccomp | |
| 16:11:15 | sean-k-mooney | if there threat profile allwos it then there is no reas for them to not do it but they need to make that threat assement themselves | |
| 16:11:20 | MarkMielke | but then I found that Red Hat build of OpenJDK also uses prctl :-) | |
| 16:11:59 | MarkMielke | so now we often disable it entirely (shocker!) | |
| 16:12:00 | MarkMielke | hehe | |
| 16:13:07 | sean-k-mooney | my dev machines uses to have almost all the mitgation disabled at one point | |
| 16:13:45 | MarkMielke | i settled on a config which ends up being similar to RHEL 7.7 ended up setting by default | |
| 16:13:52 | MarkMielke | spectre_v2=retpoline | |
| 16:13:54 | sean-k-mooney | it got kind fo annoying to have to keep updating my grub config | |
| 16:15:24 | MarkMielke | ok - well i should get a few other things done. Thanks for the chat. I think I'll open that bug with regard to oslo_utils unit tests being left in nova. | |
| 16:15:42 | sean-k-mooney | o/ | |
| 16:15:58 | MarkMielke | and if it's easy to add test i'll add to oslo_utils, but if too hard, i may add another test or two for nova as separate submission | |