Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-11-11
10:55:26 gibi wait a cycle, then move to hard fail?
10:55:27 owalsh gibi: I think we need /etc/nova/nova-compute.conf
10:55:37 owalsh we can not longer share /etc/nova/nova.conf
10:56:07 gibi I don't think nova suggested to share the nova.conf between services
10:56:27 gibi by default devstack generates nova-cpu.conf and that is the only conf passed to n-cpu
10:56:47 owalsh then nova-compute shouldn't default to /etc/nova/nova.conf
10:57:42 openstackgerrit chengsheng proposed openstack/nova master: Add hypervisor CPU feature check during live migration https://review.opendev.org/762330
10:58:12 owalsh I can fix this for tripleo and it's use of puppet-nova, but I would be concerned about other use cases of puppet-nova
10:58:38 gibi owalsh: do you refer to the doc in https://docs.openstack.org/nova/latest/cli/nova-compute.html#files that states nova.conf?
10:59:54 openstackgerrit YumengBao proposed openstack/nova-specs master: libvirt supports composing cyborg owned vGPU accelerator into domain XML https://review.opendev.org/750116
11:01:17 gibi owalsh: I did not find any other places that makes the suggestion that nova-compute needs nova.conf
11:01:51 owalsh gibi: how about the .debs/.rpms?
11:02:25 gibi I have no power over packagers, but sure we can ask them on the ML
11:03:02 gibi zigo: does the deb packages share nova.conf between the controller and the compute service?
11:03:15 owalsh gibi: for the rpms at least, the compute systemd runs /usr/bin/nova-compute
11:03:25 owalsh which will default to /etc/nova/nova.conf
11:05:16 gibi I don't find where nova implements that defaulting to /etc/nova/nova.conf
11:05:28 gibi stephenfin, dansmith: ^^ do you know where it is implemented?
11:11:46 gibi owalsh: I can reproduce that default behavior in devstack. So somewhere we says that nova-compute should read the nova.conf by default
11:13:49 owalsh gibi: ack, and puppet-nova (at least) is designed to generate one and only one nova.conf for all services by default
11:15:10 gibi that behavior is problematic regardless of what we are doing here. How puppet-nova is able to support multi cell deployments then? The different cell conductors needs to be pointed to different dbs
11:16:24 owalsh typically not on the same host
11:17:39 gibi but then if the nova-compute is on its own host then that nova-compute can have a nova.conf generated without db credentials
11:18:06 owalsh gibi: yes, but an all-in-one api+compute deployment will not work
11:18:11 gibi so I guess the only problematic scenario is when all in one
11:18:18 gibi yeah
11:19:31 owalsh snap :-) that's where the issue came up in CI but it shows how the conf logic in nova is far from ideal
11:19:33 gibi so with the current change we break all-in-one puppet-nova. But if we change what is the default config read by nova-compute we would still break puppet-nova as it only generates a singel config
11:20:04 gibi but also we break not only the all-in-one case but every csae
11:20:06 gibi case
11:20:14 owalsh yea, would need a deprecation cycle to alter this
11:20:33 gibi I need second opinion from dansmith about all these
11:21:57 gibi I do feel that nova should not bend over to support all-in-one cases, and the normal deployment cases migth depend on the fact that nova-compute reads the nova.conf by default
11:22:24 owalsh ack. FWIW I can fix the tripleo use case but I'm nervous about more general usage of nova and/or puppet-nova (e.g manual install with rpms/debs or traditional puppet usage)
11:23:30 owalsh and seem to have a contradiction in nova now where it explicitly forbids sharing nova.conf but defaults to sharing nova.conf
11:33:00 gibi I found where we default nova.conf. It is in oslo.config https://github.com/openstack/oslo.config/blob/68cefad313bd03522e99b3de95f1786ebea45d4b/oslo_config/cfg.py#L281
11:33:44 gibi nova defines no default config, oslo assigns default files based on the project name and binary name
11:36:59 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Add support for virtio-based input devices https://review.opendev.org/756552
11:36:59 gibi owalsh: so I would not be suprised that other openstack modules have the same behavior
11:39:14 gibi owalsh: just out of curiosity how many config files puppet generate for neutron? In devstack I see that neutron promotes sharing the neutron.conf but customize it with extra service specific conf files as well
11:40:13 owalsh gibi: no idea, only really know puppet-nova...
11:40:41 gibi OK, no worries, I was just curious what is the pattern in puppet
11:41:06 owalsh gibi: *_config from https://github.com/openstack/puppet-neutron/tree/master/lib/puppet/provider
11:42:00 gibi owalsh: based on that puppet-neutron follows what devstack does
11:42:12 gibi and generates one neutron.conf and a list of service specific .ini files as config
11:42:38 owalsh gibi: ack but the services read those specific config files I assume
11:43:42 gibi owalsh: I think only if they are passed with --config-file to the binary
11:44:12 gibi I just quickly checked what files neutron-metadata-agent opens without any flag, and it seems it does the same default behavior as nova
11:44:38 gibi so it basically reads only neutron.conf by default
11:44:42 gibi http://paste.openstack.org/show/799911/
11:45:19 gibi so puppet-neutron not just generates those ini files but also adds them to the binary commandline
11:46:02 gibi so puppet-neutron does not realy on what file neutron binaries reads by default
11:47:56 owalsh gibi: I don't think puppet is adding the conf to the commandline, probably the .deb/.rpm service config
11:48:10 gibi could be
11:48:19 owalsh https://github.com/rdo-packages/neutron-distgit/blob/rpm-master/neutron-metadata-agent.service#L8
11:49:03 gibi yep, so this rpm config seems to follow what devstack does
11:49:49 gibi compared that to https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova-compute.service
11:50:04 gibi I would say the neutron packaging config is better
11:50:56 gibi at least it is more structured and follow what devstack does
11:51:08 gibi so it is more closer what upstream CI tests
11:51:28 owalsh gibi: yea, but nova.conf has been around a lot longer :-)
11:52:17 gibi sure, I can imagine nova is historically used like that
12:26:18 zigo gibi: I don't unserstand your question. Do you mean, is it the same nova.conf on both roles? Yeah, it is.
12:27:17 zigo The nova.conf file is stored in the nova-common package, which is a dependency of all nova services.
12:27:48 zigo It must be the same file anyways, because there's the case were we got all-in-one.
12:27:54 zigo Does this answer your question?
12:29:01 gibi zigo: thanks
12:29:04 gibi zigo: yes it does
12:29:15 zigo Oh, I see, it's about https://review.opendev.org/#/c/762176/
12:29:30 zigo Well, indeed, -1 for me as it would prevent an all-in-one setup.
12:29:31 gibi zigo: yes
12:29:40 gibi zigo: I guess that patch breaks debian then too
12:30:26 gibi in the all-in-one-case
12:30:45 gibi as nova.conf will have a [database]connection config but
12:30:50 zigo It'd be ok if nova-compute was using a different filename, in which case we should just not add the [database] section for nova-compute.
12:31:17 gibi zigo: owalsh figured out the nova-compute binary reads nova.conf by default
12:31:49 gibi so if there is no --config-file added to the binary then it reads the same file as other nova binaries in an all-in-one deployment
12:31:50 zigo Please don't attempt to use /etc/nova/nova-compute.conf, as this is already in use in Debian.
12:32:34 zigo to startup nova-compute.
12:32:34 zigo nova-compute --config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf --log-file=/var/log/nova/nova-compute.log
12:32:34 zigo The Debian package already does:
12:33:12 zigo Though, feel free to make it use something else, like /etc/nova/nova-compute-service.conf, then nova-compute can get rid of reading /etc/nova/nova.conf reading...
12:33:26 zigo (in fact, whatever, I'll adapt the packaging anyways...)
12:33:53 gibi zigo: thanks for the input
12:34:11 gibi I will try to get oppinions from the rest of the core team before moving forward
12:43:42 sean-k-mooney zigo: nova-compute should not need to read nova.conf today
12:44:03 sean-k-mooney zigo: what you could do is remove all the db creds form nova.conf and put them in nova-db.conf
12:44:15 sean-k-mooney then put common config in nova.conf
12:44:28 sean-k-mooney and start nova compute with -config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-compute.conf
12:44:41 sean-k-mooney and nova-api with -config-file=/etc/nova/nova.conf --config-file=/etc/nova/nova-db.conf
12:44:48 sean-k-mooney or similar
12:45:34 sean-k-mooney i really dont think we need to add a new file
12:46:09 sean-k-mooney we could but the expectation was for a very long time that nova.conf is used by the contol plane service only
12:46:31 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.binary_type/integer_types/string_types https://review.opendev.org/728094
12:46:33 sean-k-mooney and nova-compute.conf or similar was used for the compute service which would not read nova.conf
12:46:55 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.text_type (1/2) https://review.opendev.org/728109
12:50:09 sean-k-mooney this was hit in kolla in queens https://github.com/openstack/kolla-ansible/commit/11b9eba2a48b793d00ba04d7ace8f644c2a14f97 and after devstack hit this in rocky
12:52:12 sean-k-mooney zigo: i think the fix for debian is to jsut continue to use the nova-compute.conf and not use nova.conf at all
12:52:36 openstackgerrit Lee Yarwood proposed openstack/nova-specs master: WIP - Image and flavor defined ephemeral storage encryption https://review.opendev.org/752284
12:54:35 openstackgerrit Takashi Natsume proposed openstack/nova master: Update contributor guide for Wallaby https://review.opendev.org/754427

Earlier   Later