| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-01-27 | |||
| 16:46:13 | tosky | and tosky wasn't able to find a proper workaround for the failures when using grenade plugins (shame on him) | |
| 16:54:58 | melwitt | tosky: do you have a change where you are working on getting plugins to work? is that here? https://review.opendev.org/639774 is there any way we could help? | |
| 16:56:46 | tosky | melwitt: I don't have a change; the problem is that I'm not able to figure out where the py2 openstackclient is installed and make sure its CLI used over its py3 counterpart | |
| 16:57:08 | melwitt | oh, interesting | |
| 16:57:26 | tosky | I would expect that starting from a py3 deployment, even if the py2 version is installed afterwards, the py3 version installed first would still be the one used for the CLI | |
| 17:00:10 | melwitt | I would expect that as well | |
| 17:01:28 | stephenfin | efried: create a blueprint for what? | |
| 17:02:43 | lyarwood | stephenfin: https://review.opendev.org/702943 - I think he was talking about that spec that had just merged. | |
| 17:03:01 | stephenfin | ah, makes sense | |
| 17:03:03 | stephenfin | will do | |
| 17:03:32 | tosky | talking about jobs, a few days ago the nova-live-migration job was failing in stable/pike; do you know if it has been solved? | |
| 17:04:38 | melwitt | tosky: it has not been solved yet. sean-k-mooney said they would try to dig into it more soon. it's not 100% failure rate but seems like it's close to that | |
| 17:06:44 | efried | stephenfin: I was talking about https://review.opendev.org/#/c/682655/ | |
| 17:07:04 | efried | It looks like you're using a new bp name for this round, but there's no bp by that name. | |
| 17:09:31 | openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Switch to the Zuulv3 grenade job https://review.opendev.org/704364 | |
| 17:10:50 | stephenfin | efried: https://blueprints.launchpad.net/nova/+spec/flavor-extra-spec-validators | |
| 17:11:00 | efried | thanks | |
| 17:15:19 | sean-k-mooney | tosky: its actully the other way around. which ever version is installed last get used but only if its the first installation | |
| 17:15:40 | sean-k-mooney | if you upgrade an existing install it does not replace the console script i think | |
| 17:16:11 | sean-k-mooney | so ify its py2 then py3 on n-1 1 and py3 only on n i thing the console scrript will point to py2 | |
| 17:17:15 | sean-k-mooney | anyway its a bit of a pain. we likely should make the py2 install in py3 mode optional in train | |
| 17:17:26 | sean-k-mooney | so we can disable it in the greade job | |
| 17:18:15 | sean-k-mooney | but yes i will try to look at nova-live-migration this week. i think i have got a downstream backport working so i should have more time to look at it this week | |
| 17:18:35 | sean-k-mooney | i might try to set aside tomorow to look at it | |
| 17:26:32 | stephenfin | gibi: RE: https://review.opendev.org/#/c/701763/4/specs/ussuri/approved/non-admin-filter-instance-by-az.rst@83, are you saying it's not possible to use policy to allow any user access to the API? | |
| 17:28:45 | gibi | stephenfin: I think the policy is not per field | |
| 17:30:12 | gibi | stephenfin: so if we want to make some fields available for filtering to the normal user then we need to change code | |
| 17:31:39 | stephenfin | Oh, so even with this spec there are fields that the admin will be able to filter on but not a normal user? | |
| 17:31:52 | stephenfin | I missed that, if so | |
| 17:33:01 | gibi | stephenfin: yeah I think so | |
| 17:33:11 | stephenfin | riiight, okay then, my bad | |
| 17:33:16 | gibi | admin can filter on admin only fields too | |
| 17:33:34 | stephenfin | lemme double that check and then I'll hit it again | |
| 17:33:34 | gibi | I mean those fields that are not returned in the GET /servers/ to non-admins | |
| 17:33:44 | stephenfin | yeah, makes sense in hindsight | |
| 17:37:14 | tosky | sean-k-mooney: thanks for both :) | |
| 19:18:32 | openstackgerrit | Merged openstack/nova stable/pike: Avoid redundant initialize_connection on source post live migration https://review.opendev.org/683008 | |
| 19:27:28 | KeithMnemonic | Eureka, that BDM patch on stable/pike finally merged. Thanks a ton for all of your endless help mriedem: , melwitt: and of course hemna: | |
| 19:39:04 | melwitt | cool KeithMnemonic | |
| 20:13:59 | openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Add Fedora based jobs to the experimental queue https://review.opendev.org/701231 | |
| 20:14:08 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Use virDomainBlockCopy to swap volumes with >= 5.10.0 https://review.opendev.org/696834 | |
| 23:18:24 | efried | dansmith: remind me (again) which direction we're allowed to have mismatched versions: controller newer than compute or vice versa? | |
| 23:22:30 | sean-k-mooney | controler newer then compute i think | |
| 23:22:54 | sean-k-mooney | we expect the contol plane to be upgraded first | |
| 23:25:04 | efried | k | |
| 23:26:44 | sean-k-mooney | in either case the sender is ment to back level to the max version supported by the reciver if memory serves. greade upgrade the contol plane first so that is what we test. | |
| 23:27:28 | efried | I'm trying to understand the upgrade implications of the extra spec validation thing. | |
| 23:27:45 | sean-k-mooney | ah ok | |
| 23:28:53 | sean-k-mooney | that does rais an intersting question. if the extra specs were ovo should you allow the flaovr to be back leveled if a unsupporte extra spec was requested. | |
| 23:29:26 | sean-k-mooney | i have not review that proposal by the way so that is jsut me specualting but i would lean towrads no | |
| 23:29:32 | efried | I think it's simpler than that. | |
| 23:30:23 | efried | Ultimately an extra spec that's used on/by the compute (which is many of them) is being validated at the API. | |
| 23:30:50 | sean-k-mooney | yes at least initally | |
| 23:31:23 | efried | and if the versions are mismatched, we could have an extra spec that the control plane (because newer) understands, but the compute won't. If the developer of that extra spec is doing their job right, they'll have tied it to a request filter so you would never land on that compute, so probably no big deal. | |
| 23:31:48 | efried | But the other way around is also possible. If we deprecate/remove support for an extra spec, the control plane thinks it's invalid while the compute still supports it. | |
| 23:32:15 | efried | I guess that means you have to wait *two* cycles before you get to add the validator that forbids the removed extra spec. | |
| 23:32:16 | efried | or something. | |
| 23:33:07 | sean-k-mooney | ya althoguh im not sure we have ever remvoed an extra spec | |
| 23:33:28 | sean-k-mooney | i can think of one or two i woudl consider removing but i dont think we have ever actully done it | |
| 23:33:50 | sean-k-mooney | and actully i have reviewd https://review.opendev.org/#/c/682655/1/specs/ussuri/approved/flavor-extra-spec-validators.rst but it was last cycle | |
| #openstack-nova - 2020-01-28 | |||
| 00:13:19 | dansmith | efried: compute is the only thing allowed to be old | |
| 00:13:25 | efried | ack | |
| 00:13:59 | dansmith | efried: the controllers are always required to enforce the version rule, so if it's something that the compute might not be able to do, it needs to check the version and refuse, downlevel, or find a compute that *can* handle it | |
| 00:14:24 | efried | right, but the point of this validation is that it's done at flavor creation time -- there's no compute in the picture. | |
| 00:14:53 | efried | sorry, you were answering my actual question, thanks. | |
| 00:15:21 | dansmith | there's compute at boot time, so you can create a newer flavor, you just need to refuse to boot an instance from it when there are no computes able to handle the request | |
| 00:16:40 | efried | right | |
| 00:17:16 | efried | we just need to be careful not to enforce a new rule that strictifies something until we're sure all the computes are upgraded. Kind of thing. | |
| 00:17:51 | dansmith | not sure what you mean exactly.. | |
| 00:18:28 | dansmith | you want to ban creation of the flavor with something if some computes are still old? that's definitely okay if it's appropriate, just not required if it's a spec that just specifies something about the instance | |
| 00:20:11 | efried | I'm saying if we decide to, I don't know, deprecate and remove the 'shared' hw:cpu_policy, it would be reasonable to think you could change the validator accordingly to bounce new flavors with hw:cpu_policy=shared. But you can't, because old computes can (and should be allowed to) still handle 'shared'. | |
| 00:20:42 | efried | (I know that's not a good example, just trying to explain the issue.) | |
| 00:22:18 | sean-k-mooney | am we proably would want a config option of some kind to to contol how deprecations are treated | |
| 00:22:48 | sean-k-mooney | so warn by default and make it an error if you opt in. | |
| 00:23:14 | sean-k-mooney | or something but i thnk we can cross that bridge when we have a example that requires it | |
| 00:24:11 | efried | Well, I definitely think we should have a granny switch that turns the whole validator on or off. | |
| 00:24:23 | efried | I don't know about per-key, if that's what you're suggesting. | |
| 00:24:42 | sean-k-mooney | i was more thinkg a strict vs compatiblity mode thing | |
| 00:24:55 | sean-k-mooney | rather then per key | |
| 00:26:34 | sean-k-mooney | if we wanted to deprecate something then we would mark it as such in the validtor and consier it an error in stict mode but just warn on it in compatibliyt mode. then if we drop support entirely we can graduate to always an error becasue we know it will not be supprotin in n+X releases | |
| 00:26:52 | sean-k-mooney | that said im not sure we need to do that | |
| 00:27:15 | sean-k-mooney | if the extra spec is not support by a compute anymore its just ignored | |
| 00:27:51 | dansmith | efried: right so that's an example where controlling the flavor definition based on compute levels would be good.. where changing the flavor impacts the way computes behave, as opposed to just something that affects the instance like a new disk controller or something | |
| 00:28:40 | efried | yikes. | |
| 00:29:12 | dansmith | yikes/ | |
| 00:29:15 | dansmith | I'm agreeing with you | |
| 00:29:24 | dansmith | I mean, I think :D | |
| 06:29:53 | HagunKim | Hello, Nova! I have a question. Sometimes my VMs are deleted unexpectedly. What conf or log should I check?? | |
| 06:29:53 | HagunKim | Here is nova-compute log when a vm is deleted. I removed date, req. id, instance id. | |
| 06:29:53 | HagunKim | INFO nova.compute.manager [instance: intance id] Terminating instance | |
| 06:29:53 | HagunKim | INFO nova.virt.libvirt.driver [-] [instance: intance id] Instance destroyed successfully. | |
| 06:29:54 | HagunKim | INFO nova.compute.manager [-] [instance: intance id] VM Stopped (Lifecycle Event) | |
| 06:29:54 | HagunKim | INFO nova.compute.manager [instance: intance id] During sync_power_state the instance has a pending task (deleting). Skip. | |
| 06:29:55 | HagunKim | INFO nova.virt.libvirt.driver [instance: intance id] Deleting instance files /var/lib/nova/instances/instance_id_del | |
| 06:29:55 | HagunKim | INFO nova.virt.libvirt.driver [instance: intance id] Deletion of /var/lib/nova/instances/instance_id_del complete | |
| 06:29:56 | HagunKim | INFO nova.compute.manager [instance: intance id] Took 23.99 seconds to destroy the instance on the hypervisor. | |
| 08:29:07 | gibi | HagunKim: with the request_id from your log I would grep the nova logs to see what other events lead to the deletion of the instance | |
| 08:29:22 | bauzas | good morning Nova | |
| 08:31:23 | gibi | bauzas: good morning | |
| 08:31:44 | bauzas | gibi: how are you ? I was a bit on and off last week | |