| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-02 | |||
| 14:45:26 | mriedem | lyarwood: looks like we've probably always created another volume on a reschedule if the source_type!='volume', i don't see anything that deletes a volume that nova created prior to rescheduling - unrelated to your issue really, but a super latent bug | |
| 14:46:16 | mriedem | most likely another reason we should move volume creation to api or conductor so that compute doesn't have to manage that | |
| 14:46:37 | mriedem | compute would just get a bdm and assume the volume already was created | |
| 14:49:54 | lyarwood | mriedem: wonderful, happy to look at that as well if you wouldn't mind writing that up in a bug? | |
| 14:50:37 | mriedem | i'd need to recreate it first, and don't have a 2 node system handy | |
| 14:51:13 | mriedem | i just know we do a better job of tracking ports to know which we've created ourselves and which we haven't, and cleaning those up properly before reschedule (delete the ports we created, unbind the ports we didn't) | |
| 14:51:29 | mriedem | we don't really do anything like that for volumes | |
| 14:59:55 | openstackgerrit | Merged openstack/nova master: tox: Ensure reused envdirs share the same deps https://review.openstack.org/588207 | |
| 15:16:40 | cfriesen | this is kind of a basic question but I'm having a hard time finding an answer. the nova docs suggest we default to using durable AMQP queues with persistent messages. The oslo.messaging code suggests the default is non-durable queues. which is it? | |
| 15:18:21 | openstackgerrit | Merged openstack/nova master: Fix a typo in comment in resource_provider.py https://review.openstack.org/588145 | |
| 15:20:42 | mriedem | cfriesen: where do the nova docs say that? | |
| 15:21:08 | mriedem | it's clearly not durable by default https://docs.openstack.org/nova/latest/configuration/config.html#oslo_messaging_rabbit.amqp_durable_queues | |
| 15:21:18 | mriedem | anything in the nova docs is likely way out of date | |
| 15:21:39 | mriedem | https://docs.openstack.org/nova/latest/reference/rpc.html?highlight=durable | |
| 15:21:40 | cfriesen | doc/source/reference/rpc.rst: | |
| 15:21:41 | mriedem | that? ^ | |
| 15:22:00 | mriedem | yeah that's all super duper old | |
| 15:22:03 | mriedem | probably predates oslo.messaging | |
| 15:22:06 | mriedem | open a bug | |
| 15:23:31 | cfriesen | okay. what about message persistance? are we always transient? | |
| 15:24:54 | mriedem | let the code be your guide young chris | |
| 15:26:08 | mriedem | i thought durable == persistent | |
| 15:28:01 | cfriesen | just to make things interesting, rabbitmq can have durable or transient exchanges and queues, and then the message itself can be persistent or transient. | |
| 15:29:39 | mriedem | idk then, ask kgiusti in #openstack-oslo? | |
| 15:29:46 | mriedem | or sileht? | |
| 15:30:11 | cfriesen | yeah, will do. I suspect we're just always transient. | |
| 15:30:17 | tssurya | mriedem: probably this has come up zillion times, but I couldn't find the right reasoning on why the power synchronization is asymmetric in nova.. why does nova not acknowledge that the vm is back up again ? | |
| 15:31:36 | tssurya | isnt' the driver the state of truth ? | |
| 15:32:08 | mriedem | tssurya: what thing are you specifically talking about? the sync_instance_power_states periodic in compute | |
| 15:32:09 | mriedem | ? | |
| 15:32:12 | tssurya | yes | |
| 15:32:18 | mriedem | that shuts down your server if the db says it's down but the driver says it's up? | |
| 15:32:27 | tssurya | exactly | |
| 15:32:36 | mriedem | because you could be getting charged for one when you told nova to shut it down | |
| 15:32:59 | tssurya | ah okay.. | |
| 15:33:11 | mriedem | it's been years since i've had to load that thing into memory | |
| 15:33:17 | mriedem | and it's always a disaster when i do | |
| 15:33:45 | tssurya | and for the ironic cases where the users may interact through the ipmi interface.. | |
| 15:33:56 | tssurya | the only solution we have is to switch the power sync off ? | |
| 15:34:21 | mriedem | nova doesn't support monkeying with the guests out of band | |
| 15:34:27 | tssurya | is there is a way we could make this behaviour configurable (a choice for deployments) | |
| 15:34:39 | tssurya | hmm | |
| 15:34:44 | mriedem | i'm sure vmware and powervc have had this same argument because the user started up the vm in vcenter or the hmc | |
| 15:34:56 | mriedem | and then nova shut it down | |
| 15:35:03 | tssurya | yes, we have the same stuff | |
| 15:35:19 | stephenfin | mriedem: Remind me: can I +W code? We've branched and everything, right? | |
| 15:35:26 | tssurya | we were just thinking if we could make this configurable.. as in tell nova not to shut it back down | |
| 15:35:32 | mriedem | stephenfin: we have not branched | |
| 15:35:38 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: guest: introduce blockStats instead of domain.blockStats https://review.openstack.org/526833 | |
| 15:35:41 | tssurya | the default could be what nova does today of course | |
| 15:35:43 | stephenfin | Well then :) | |
| 15:35:50 | mriedem | stephenfin: +W depends on the change | |
| 15:35:58 | mriedem | if it's a bug fix, then yeah maybe | |
| 15:36:02 | mriedem | if it's docs, sure | |
| 15:36:10 | stephenfin | It's a refactor so nope, it should wait | |
| 15:36:12 | mriedem | if it's a big ass refactor or something risky, likely not a great idea right now | |
| 15:36:28 | stephenfin | Yup, I'll just leave it til branch. No panic on it | |
| 15:37:02 | mriedem | tssurya: likely a better question for the ML to get wider input, from both -dev and ops lists | |
| 15:37:19 | mriedem | i'm able to devote about 5% brain to your question atm | |
| 15:37:30 | tssurya | mriedem: ack, sorry for the bad timing then :) | |
| 15:37:35 | mriedem | np | |
| 15:43:20 | gmann | stephenfin: yes, these were missed and they would not fail as they all are req files which are not verified on sample tests. let me know if you can or want me to add them and good to ref those in api-ref also. | |
| 15:51:57 | stephenfin | gmann: I don't mind. Is it a big issue? I guess it just means the api-ref will be incomplete? | |
| 15:52:20 | stephenfin | gmann: If you _do_ have time to work on them, I'll happily review it | |
| 15:59:55 | mriedem | tssurya: note you can disable that sync power state task | |
| 16:00:21 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L7439 | |
| 16:00:30 | melwitt | dangit, was just pulling that up | |
| 16:01:04 | melwitt | you win...again | |
| 16:04:48 | melwitt | https://github.com/openstack/nova/blob/master/nova/conf/compute.py#L753-L777 | |
| 16:17:20 | rdiggz | good day everyone. Could someone help me understand what this means? "A sys-admin privsep daemon has been added and needs to be included in your | |
| 16:17:22 | rdiggz | rootwrap configuration. | |
| 16:18:35 | rdiggz | Its from the release notes of pike and queens. Im seeing an issue deploying a snapshot of a CentOS7 instance that points to sys-admin and im thinking they are related. | |
| 16:26:33 | mriedem | cfriesen: is this a bug that should be upstreamed or at least reported to nova? https://github.com/starlingx-staging/stx-nova/commit/fe0c0617be857161b6bb66d632b2b35887c08772 | |
| 16:27:27 | mriedem | oh nvm it's already in nova | |
| 16:27:38 | mriedem | https://github.com/openstack/nova/commit/ce8bf6734e554b116e82e924cbe81a5968441926 | |
| 16:29:42 | cfriesen | they should've pointed to upstream if it was a backport. grr. | |
| 16:30:05 | mriedem | it's not a backport | |
| 16:42:47 | mriedem | dansmith: looks like one of your earlier is_bfv attempts made it in https://github.com/starlingx-staging/stx-nova/commit/71acfeae0d1c59fdc77704527d763bd85a276f9a#diff-b839034e35c154b8c3a1c65bf7791eefR114 | |
| 16:43:16 | dansmith | mriedem: lol | |
| 16:43:16 | mriedem | cfriesen: what are offline_cpus? | |
| 16:43:59 | melwitt | I think that's part of the second iteration of my famous root_gb=0 workaround patches | |
| 16:44:35 | cfriesen | mriedem: that's part of our cpu scaling code. you can boot with X cpus and then scale down to fewer. offline_cpus tracks which guest vCPUs are supposed to be offline. | |
| 16:44:49 | cfriesen | mriedem: (the cpu scaling stuff requires guest cooperation to do properly) | |
| 16:45:02 | cfriesen | melwitt: I think that's correct | |
| 16:45:44 | melwitt | dansmith had started with the request spec part but I found that the claims and reporting on the compute side didn't yet support is_bfv so had to add that to it too in another patch stacked on top. and in the end it looked exactly like the original workaround patch that got nack'd by everyone | |
| 16:46:04 | melwitt | so it lay abandoned again | |
| 16:46:53 | mriedem | cfriesen: how is that different from just resizing down? | |
| 16:47:01 | mriedem | the offline CPUs are reserved? | |
| 16:47:27 | cfriesen | mriedem: it's similar to the "live-resize" proposed recently | |
| 16:47:30 | mriedem | i.e. so you can scale up and down w/o worrying about claim failures? | |
| 16:47:34 | cfriesen | mriedem: the server stays running | |
| 16:48:10 | cfriesen | mriedem: the resource tracking is adjusted, but the quota isn't | |
| 16:48:24 | cfriesen | so you could fail to scale up if there are no free cpus on the host | |
| 16:49:14 | mriedem | right that's what i was saying, | |
| 16:49:24 | mriedem | you essentially boot with like 4 vcpu, | |
| 16:49:29 | mriedem | resize down to 2, | |
| 16:49:39 | mriedem | but the other 2 are "offline" so you can resize back up? | |
| 16:50:01 | mriedem | but you're still paying for 4 VCPU of quota? | |
| 16:50:32 | cfriesen | remember this is private cloud so "paying for quota" is a bit iffy. but yes. | |
| 16:51:00 | dansmith | I'm not sure people usually charge for quota either | |