| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-11-19 | |||
| 16:05:48 | artom | Well... | |
| 16:05:56 | sean-k-mooney | anytime we define cpu_dedicated_set you should also disable the fallback | |
| 16:06:06 | sean-k-mooney | we likely are not doing the latter | |
| 16:06:09 | artom | We don't define it | |
| 16:06:18 | sean-k-mooney | its enabled by default | |
| 16:06:32 | artom | I mean cpu_dedicated_set | |
| 16:06:42 | artom | What I'm seeing is the test_resize_unpinned_server_to_pinned tearDown is failing | |
| 16:06:51 | artom | Because it can't delete the pinned instance | |
| 16:07:08 | artom | Because nova.exception.CPUUnpinningInvalid: CPU set to unpin [2, 6] must be a subset of pinned CPU set [] | |
| 16:07:46 | artom | AFAICT there's no race or anything, it's just booting the resized pinned instance on the host without cpu_dedicated_set or vcpu_pin_set configured | |
| 16:07:54 | artom | Pinning it to 2 random CPUs | |
| 16:07:59 | artom | Then complaining when it can't delete it | |
| 16:08:11 | artom | Which... does kinda smell like a minor bug... | |
| 16:08:11 | sean-k-mooney | well we need delete teh vms before we reconfigure | |
| 16:08:18 | artom | We *don't* reconfigure | |
| 16:08:31 | artom | This is all without any config changes or service restarts | |
| 16:08:46 | sean-k-mooney | no i mean im not sure we are waiting for all vms from the previous test to be deleted | |
| 16:09:08 | artom | AFAICT, yes | |
| 16:09:54 | sean-k-mooney | artom: we should never see nova.exception.CPUUnpinningInvalid: CPU set to unpin [2, 6] must be a subset of pinned CPU set [] | |
| 16:10:12 | sean-k-mooney | if we do it means the config have chagned | |
| 16:10:31 | sean-k-mooney | or we are hitting the race which stephen fixed | |
| 16:10:40 | artom | But... what happens in the case of having booted an instance with pinned CPUs on a host without cpu_dedicated_set configured? | |
| 16:10:46 | artom | And then deleting that | |
| 16:10:52 | artom | Wouldn't we expect that error? | |
| 16:10:57 | sean-k-mooney | if its still not set its fine | |
| 16:11:07 | artom | Or does it use the database and not the config? | |
| 16:11:17 | sean-k-mooney | its not vallide to add cpu_dedicate_set to a host with vms | |
| 16:11:22 | artom | I know | |
| 16:11:26 | artom | That's not what we're doing here | |
| 16:11:38 | sean-k-mooney | can you point me to the test | |
| 16:11:43 | artom | sean-k-mooney, gmeet? :) | |
| 16:11:49 | sean-k-mooney | sure | |
| 16:13:03 | artom | sean-k-mooney, https://meet.google.com/eqm-jjcx-fjw | |
| 16:13:04 | sean-k-mooney | im going to make a cup of coffee but say 15 past | |
| 16:13:15 | sean-k-mooney | ill join when i get back | |
| 16:13:20 | artom | Take your time :) | |
| 16:16:24 | sean-k-mooney | just back :) | |
| 16:24:21 | dansmith | gibi: the distro-ness also affects our ability to describe what should go in each config file, | |
| 16:24:42 | gibi | dansmith: do you have an example? | |
| 16:25:04 | dansmith | gibi: because the problem on the ML is making all the distro packages work both across multiple nodes, as well as AIO, and currently there's no nova-conductor-cellN vs nova-superconductor service(s) | |
| 16:25:23 | dansmith | gibi: so I'm not quite sure how to write this to make sense for both | |
| 16:26:04 | dansmith | gibi: I'm going to try, but we'll probably need some input from people like owalsh .. maybe that means we need to use something other than the install doc to describe things, I dunno | |
| 16:26:14 | dansmith | writing docs is hard :( | |
| 16:27:13 | gibi | so the problem is that distros map packages to services but the nova-conductor package needs to be mapped to two type of services with different config? | |
| 16:27:52 | dansmith | yeah | |
| 16:27:58 | dansmith | and also, | |
| 16:28:02 | owalsh | hey | |
| 16:28:05 | dansmith | and this is what he was complaining about, | |
| 16:28:15 | dansmith | normally they'd all point to nova.conf, assuming they'd be on different hosts, | |
| 16:28:39 | dansmith | but if they're all on the same host, I'd have to describe changing/overriding unit files to get the right config to conductor vs. api, etc | |
| 16:29:22 | owalsh | tkajinam made an important point on the ML. It's not just AIO, it's the typical production deployment of ironic nova-compute | |
| 16:29:32 | owalsh | on the same host as nova-api etc... | |
| 16:30:02 | dansmith | owalsh: ack, yeah, that's the same problem, just another reason you might hit it | |
| 16:30:30 | dansmith | owalsh: anyway, what I'm describing above is just that I was going to start working on the config samples in this doc: https://docs.openstack.org/nova/latest/install/controller-install-ubuntu.html | |
| 16:30:43 | dansmith | but realized that I'll not be able to describe it fully without talking about changing the distro unit files, etc | |
| 16:31:07 | gibi | dansmith: so either we start describing deployment scenarios (which feels bad) or start using separate config file name for each service | |
| 16:31:10 | owalsh | dansmith: ack, and that's something puppet-nova could do | |
| 16:32:10 | dansmith | gibi: well, I really think we should avoid getting into prescribing config file names for things, because again, that's very very distro and deployment specific, | |
| 16:32:22 | dansmith | gibi: and I think that the suggestion on the ML to standardize those layers a bit actually got us into a more sticky situation | |
| 16:34:38 | gibi | but then we need to iterate on at least AIO deployment scenario, Ironic, and a real cellv2 scenario | |
| 16:35:53 | gibi | anyhow I agree that docing this properly is hard | |
| 16:36:24 | owalsh | implementing it in code is no picnic either :-) | |
| 16:36:33 | gibi | owalsh: good point :) | |
| 16:36:36 | dansmith | at the very least, I think we should get rid of the 3x distro-based flavored docs for each | |
| 16:36:45 | gibi | +1 ^^ | |
| 16:37:28 | dansmith | maybe I can just sprinkle some notes in around the configuration bits about service differences, and see if that helps clarity at all | |
| 16:37:46 | gibi | yeah, that could be a good starting point | |
| 16:41:07 | openstack | Launchpad bug 1903908 in puppet-nova "nova conf [api]/dhcp_domain is required on nova-compute" [Undecided,New] - Assigned to Oliver Walsh (owalsh) | |
| 16:41:07 | gibi | dansmith: I don't know if you see this but there are also ~bugs uncovered by these config discussion https://bugs.launchpad.net/nova/+bug/1903908 (thank to owalsh for the report) | |
| 16:42:03 | dansmith | I saw that, I'm not sure how I feel about it | |
| 16:42:15 | dansmith | anything that the configdrive code uses will be shared with the metadata api | |
| 16:43:28 | gibi | yeah, I don't like that metadata calls deep into the metadata code | |
| 16:44:11 | dansmith | configdrive? | |
| 16:44:18 | dansmith | well, that's pretty much the whole point of it :) | |
| 16:44:23 | gibi | the gneeration of the config drive | |
| 16:44:45 | gibi | but this result in the code use by two different services | |
| 16:45:56 | gibi | I guess it is just the fact that I associate the the metadata code only to the metadata service and always forget that ther is a dependency to it from the nova-compute | |
| 16:46:22 | gibi | I guess that lead to the fact that we moved the domain config to the [api] section | |
| 16:46:47 | gibi | anyhow this is for another day as I have to go offline | |
| 16:46:53 | gibi | o/ | |
| 16:49:27 | dansmith | ack, I'll put this up for discussion in a bit | |
| 16:50:56 | owalsh | dansmith: I was going to respond to tkajinam on the ML but I don't really have a response, just a +1 really | |
| 16:52:12 | dansmith | owalsh: okay I guess it's really the same thing anyway | |
| 16:52:49 | owalsh | yes, the first point IIUC is if we have nova.conf and nova-cpu.conf, both containing the generated sample config, how do I know where to set option foo? | |
| 16:52:50 | dansmith | if the packages want to use nova-compute.conf for the compute package, that'd solve it, AFAIK | |
| 16:55:01 | owalsh | ack, only realistic solution I can see right now, but it's also a bit unpleasant | |
| 16:56:22 | dansmith | well, nova/ironic has pretty much never fit into nova properly so .. unsurprising that we have this problem | |
| 16:56:35 | dansmith | we could also just special case the check and not complain if we're using ironic, | |
| 16:56:48 | dansmith | since we're not on a compromise-able situation | |
| 16:57:27 | owalsh | I don't think so, the deb/rpm doesn't know what nova-compute will be used for | |
| 16:57:54 | owalsh | ... when it creates the service | |
| 16:59:11 | dansmith | what I meant was, the compute package could keep using nova.conf, and if your tool then configures nova.conf for ironic-and-db-creds-because-single-node, nova-compute wouldn't explode when it starts | |
| 16:59:31 | dansmith | it wouldn't solve the AIO case, but it would solve the ironic one | |
| 17:00:51 | owalsh | rpm/deb would need a new service unit file e.g "nova-ironic" that uses nova.conf vs nova-compute.conf | |
| 17:01:04 | dansmith | going back to the nova-db.conf suggestion.. if we allow the wsgi app to read that, then you can again just configure all the services to read from that, except for nova-compute, and then the only thing that doesn't work is AIO-multicell right/ | |
| 17:02:20 | owalsh | did seem like the most elegant solution, but also not had a lot of time to think of any gotchas | |
| 17:02:52 | owalsh | and I think AIO-multicell could be considered not supported outside of devstack | |
| 17:03:19 | owalsh | or if you really really want to do this for some of CI job then use containers | |
| 17:03:23 | dansmith | well, I think we should fix wsgi to let you specify config files like everything else, and then let you guys decide how you want to template and split the config files to make the packages work (or not) | |
| 17:06:42 | owalsh | ack, might also be worth looking at how we could parameterize the conf file path in the dev/rpm services | |