| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-10 | |||
| 11:45:38 | flwang1 | after installed devstack, i found the local_gb is too small because it's using my root /, though i have almost 800GB at /home | |
| 11:45:49 | jroll | ah | |
| 11:45:53 | flwang1 | but my / only has 30GB | |
| 11:46:06 | jroll | flwang1: there's a config option for where the instance data lives, you can change that | |
| 11:46:52 | flwang1 | ? | |
| 11:46:52 | flwang1 | #state_path = /opt/stack/data/nova | |
| 11:46:52 | flwang1 | #instances_path = /opt/stack/data/nova/instances | |
| 11:47:13 | jroll | yes, instances_path | |
| 11:47:34 | flwang1 | i changed them both and doesn't work | |
| 11:48:02 | jroll | did you restart nova-compute (and give it time to update resources)? | |
| 11:48:29 | flwang1 | yes, i did. i restarted all nova services | |
| 11:49:27 | jroll | hm, that's odd. I'm not sure if there's something else that needs to be done, hopefully someone else can help, sorry | |
| 11:50:53 | flwang1 | jroll: no worries, thank you for the help | |
| 11:51:01 | jroll | :) | |
| 11:52:32 | efried | Anyone happen to know where it's documented how/where we, as developers, are supposed to use translation macros in code? | |
| 11:53:39 | flwang1 | jroll: i think that's the issue, after restart the nova-compute, it's still trying to talk to the old path | |
| 11:53:42 | flwang1 | not sure why | |
| 11:54:29 | jroll | yeah, that would be my guess | |
| 11:54:49 | jroll | efried: this is a good start: https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html | |
| 11:54:59 | efried | Thanks jroll | |
| 11:55:00 | jroll | primarily "All text messages the user sees via exceptions or API calls should be marked for translation" | |
| 11:55:14 | efried | jroll: Yeah, I need something even more basic than that, for karimull. | |
| 11:55:41 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Fix all invalid obj_make_compatible test case https://review.openstack.org/574240 | |
| 11:55:47 | jroll | I'm not sure how to make that more basic :/ | |
| 11:56:52 | efried | jroll: See https://review.openstack.org/#/c/569498/10..11/nova/conductor/manager.py | |
| 11:59:22 | efried | I.e. need to describe what "marked for translation" means. I think that doc works. | |
| 11:59:37 | jroll | efried: yeah, I don't have anything better | |
| 12:00:06 | efried | karimull: After you've looked at the doc, let me know if you'd like some more background. | |
| 12:06:20 | sahid | stephenfin: thanks for your eye on this. I just replied | |
| 12:07:56 | openstackgerrit | Merged openstack/nova master: Handle compare in test_pre_live_migration_volume_backed* directly https://review.openstack.org/580702 | |
| 12:18:51 | mago_ | Hi team, I'm working on GPU Passtrough on OpenStack. I would like to limit the number of GPUs available to a user / project. There is no quotas for Passthrough resources and I was wondering why ? I was googling around but found no where a similar question. | |
| 12:22:37 | mago_ | Side question, as quotas are not only for Nova, but also for the Network and Block Storage services, is there a better place to ask my question ? | |
| 12:23:30 | openstackgerrit | Yikun Jiang (Kero) proposed openstack/python-novaclient master: Microversion 2.64 - Use new format policy in server group https://review.openstack.org/578261 | |
| 12:24:36 | efried | mago_: You mean like project-wide? Cause I would imagine there probably wouldn't be enough GPUs on a single host for quotas to make much sense. | |
| 12:24:49 | efried | mago_: but are you talking about physical or virtual GPUs? | |
| 12:26:55 | jaypipes | johnthetubaguy: https://review.openstack.org/#/c/543262/ could use your +2 restoration. | |
| 12:27:38 | mago_ | efried, well you could imagine a cluster with nodes with 4 to 8 GPU cards, if you have let's say 10 nodes, you can have up to 80 GPUs available. I think a quota would make sense. | |
| 12:27:44 | jaypipes | mago_: there is no quota class for either virtual or physical GPUs, no. | |
| 12:28:22 | mago_ | efried, I talk about GPU cards, passed to VMs using passthrough (also called Direct Device Assignment) | |
| 12:29:24 | efried | mago_: Yup, got it. I defer to jaypipes here, perhaps he can explain why there are no quotas. | |
| 12:30:08 | jaypipes | mago_: I can't explain *why* there are no quotas for GPUs. or at least, I can't explain that without some seriously opinionated rant about how our PCI manager system was created. | |
| 12:31:13 | mago_ | jaypipes, do you know why, I mean is it technically not possible because of the way quotas are implemented, or could it be imagined ? for GPUs, or for passthrough resources in general | |
| 12:32:12 | jaypipes | mago_: quotas were not considered by the creators of the pci management module in nova. | |
| 12:33:07 | jaypipes | mago_: when and if we move the management of PCI resources to the placement service, we will be able to use quotas for those PCI resource classes, but until then, it's not going to happen. | |
| 12:35:14 | mago_ | jaypipes, ok, understood. It does not seem to be an easy task, but do you know if this is already addressed / under discussion ? | |
| 12:38:13 | mdbooth | stephenfin gibi: Do we have something like a fixture which perturbs dicts? Running nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_pre_live_migration_volume_backed_encrypted fails for me literally every other time when running only that test. | |
| 12:38:20 | mdbooth | It's weird. | |
| 12:39:22 | mdbooth | I know dict order isn't guaranteed, but all things being equal I'd expect it to be consistent in practise unless something is deliberately making it inconsistent. | |
| 12:39:37 | gibi | mdbooth: I don't think we have. I think the order depends on the PYTHONHASHSEED | |
| 12:39:38 | jaypipes | like SR-IOV VFs are some of the resource types we do *not* currently track using the placement service. our work on nested resource providers, granular request groups, and the "reshaper" effort are paving the way to supporting these types of resources in the placement service. I predict that we'll have most of that work done by the end of the "T" release series, at which point the quotas-in-placement work can become a reality. | |
| 12:39:38 | jaypipes | mago_: we've been planning on moving the queries for calculation of resource usage out of the nova-api service and towards the placement service for a while (https://review.openstack.org/#/c/509042/) but until we are tracking all the different types of resources using the placement service consistently, such an effort is only ever going to yield a small benefit. PCI devices, NUMA topology and dedicated CPUs, and certain networking resources | |
| 12:39:39 | efried | mdbooth: There's an open bug and proposed fix for this. | |
| 12:39:50 | mdbooth | efried: Yeah, we discussed it earlier. | |
| 12:40:11 | efried | mdbooth: Okay, upon rereading I figured that's not what you were getting at :) | |
| 12:40:27 | mdbooth | But even knowing the bug, I'm still surprised by the behaviour in practise. | |
| 12:40:56 | efried | mdbooth: You mean that there's some apparent predictability to the unpredictability of the hash ordering? | |
| 12:41:11 | mdbooth | I mean, I'd expect potentially different ordering if we were creating objects in a different order, or running things in a different order, or doing *anything* different. | |
| 12:41:38 | mdbooth | But in 2 consecutive absolutely identical test runs I'm surprised to see a difference in behaviour unless we're intentionally perturbing it somehow. | |
| 12:41:44 | gibi | mdbooth: try the same PYTHONHASHSEED | |
| 12:41:54 | mago_ | jaypipes, efried thank you very much for the explanations. | |
| 12:43:08 | gibi | mdbooth: I think if PYTHONHASHSEED=0 then you will get the same order every time | |
| 12:43:19 | gibi | mdbooth: but by default PYTHONHASHSEED=random | |
| 12:44:06 | mdbooth | gibi: PYTHONHASHSEED=1 ./.tox/lower-constraints/bin/python -m testtools.run nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_pre_live_migration_volume_backed_encrypted | |
| 12:44:13 | mdbooth | Still inconsistent | |
| 12:44:22 | mdbooth | Meh, I was just curious | |
| 12:44:30 | mdbooth | This doesn't actually matter in the slightest :) | |
| 12:44:48 | gibi | :) | |
| 12:46:00 | jaypipes | mago_: np. sorry to not have great news! | |
| 12:53:30 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: [WIP/POC] Updating queued_for_delete from instance_destroy() https://review.openstack.org/566813 | |
| 12:53:35 | jangutter | jaypipes: for SR-IOV VF's there's a rough a proxy: Neutron Ports can under quota.... but it's very very very leaky at best. | |
| 12:54:22 | jangutter | jaypipes: very difficult to see how that model would translate to GPU, though. | |
| 12:57:38 | efried | cdent, jaypipes: +A on https://review.openstack.org/#/c/543262/ -- great stuff. | |
| 12:58:13 | efried | johnthetubaguy: FYI ^ | |
| 13:03:27 | cdent | efried: On self.compute thing, it's been long enoug since I made that change that I can't remember the details :( | |
| 13:03:49 | efried | no worries; gate-bound in any case. If it passes zuul, it's good enough for me. | |
| 13:04:33 | efried | cdent: But as I'm sure you're aware, mock.patch.object(actual object, 'method') is not quite the same thing as mock.patch.object(path to lib containing method, 'method') | |
| 13:04:47 | efried | cdent: But in this case I think both things were actually pointing to the lib ultimately. | |
| 13:04:53 | efried | so in this case I think it *was* the same. | |
| 13:05:08 | cdent | yes, that's why it ended up as it did, because the indirection of self.compute was meaningless | |
| 13:12:19 | openstackgerrit | Merged openstack/os-vif master: add noop plugin https://review.openstack.org/567924 | |
| 13:13:27 | tssurya | mriedem, dansmith: do you have some time now or shall I ping you back later ? | |
| 13:14:41 | mriedem | i'm here | |
| 13:15:50 | tssurya | mriedem: so is it documented somewhere that when a user does nova list --deleted, it silently ignores this | |
| 13:15:57 | tssurya | without complaining about policy ? | |
| 13:16:10 | mriedem | for non-admins? | |
| 13:16:21 | tssurya | yes | |
| 13:16:43 | mriedem | i'm not aware of something saying it's not an error for non-admins | |
| 13:16:54 | mriedem | https://docs.openstack.org/python-novaclient/latest/cli/nova.html#nova-list says (admin only) | |
| 13:17:02 | tssurya | hmm okay then I will open a bug | |
| 13:17:11 | mriedem | bug for what? | |
| 13:17:28 | tssurya | because currently if you do nova list --deleted as a non-admin | |
| 13:17:32 | tssurya | it just ignores the filter | |
| 13:17:39 | mriedem | it's been that way forever | |
| 13:17:39 | tssurya | and gives you all the active instances | |
| 13:18:03 | mriedem | changing that would likely be a non-trivial backward incompatible behavior change in the api | |
| 13:18:26 | tssurya | oh so why do the rest of the admin only filters give policy errors | |
| 13:18:32 | mriedem | see remove_invalid_options() | |
| 13:18:59 | mriedem | which filters for example? | |
| 13:19:09 | mriedem | also, you should probably include alex_xu and/or gmann in this discussion | |
| 13:19:16 | tssurya | --user or --all-tenants etcc | |