Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-07-10
10:12:41 stephenfin sahid: Comments left on https://review.openstack.org/#/c/580657. Fancy taking a look?
10:13:04 neha_alhat_ cdent: For some performance testing I want to create 100 RPs, for that I am writing python code.
10:13:35 neha_alhat_ cdent: 100 Resource providers*
10:14:33 neha_alhat_ cdent: using openstackclient: openstack resource provider create rp1
10:14:56 stephenfin mdbooth, lyarwood: I'm guessing this failing test is a race or the likes in our unit tests? http://logs.openstack.org/57/580657/2/check/openstack-tox-lower-constraints/607ab0c/job-output.txt.gz
10:15:14 stephenfin Looks unrelated to the actual change https://review.openstack.org/#/c/580657
10:15:22 mdbooth stephenfin: clicking
10:17:19 cdent neha_alhat_: if you do that in a loop with a different name each time, that ought to work. another way to do it is directly in python code that makes its own http requests to the service
10:17:42 neha_alhat_ cdent: But I am unable to create novaclient using openstackclient in python code. Like we can do in cinderclient: https://github.com/openstack/python-cinderclient/blob/master/cinderclient/v3/client.py#L51
10:17:46 mdbooth stephenfin: Weird!
10:18:35 stephenfin Right? It's the lower constraints job so we're using different requirements to usual but I can't see how that would be an issue
10:19:33 mdbooth stephenfin: I hit failures in those tests recently which didn't hit on python2. Any chance they run a different set of tests?
10:19:41 neha_alhat_ cdent: How can I create a http requests to the service through python code
10:19:51 stephenfin mdbooth: What do you mean a different set of tests?
10:20:00 neha_alhat_ cdent: like https://github.com/openstack/python-cinderclient/blob/master/cinderclient/v3/client.py#L51
10:20:04 cdent neha_alhat_: that's right, there is no "for use in code" placement client. This is because making your own is very straightforward. Have a look at the code around https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L270 for some ideas
10:20:08 mdbooth stephenfin: i.e. it runs unit tests which don't run in py27
10:20:26 stephenfin I don't think so https://github.com/openstack/nova/blob/master/tox.ini#L249
10:20:29 openstack Launchpad bug 1779711 in OpenStack Compute (nova) "test_pre_live_migration_volume_backed* intermittently fails serialized json compare" [Critical,Fix released] - Assigned to Matt Riedemann (mriedem)
10:20:29 gibi stephenfin: I think you see this https://bugs.launchpad.net/nova/+bug/1779711
10:20:45 stephenfin gibi: Yup, that looks like the one and the same
10:21:14 mdbooth gibi: Whoa...
10:21:28 mdbooth gibi: Ah... dict ordering
10:21:44 gibi yeah, good old dict ordering
10:21:50 stephenfin gibi: Not the first thing I've been caught out by ordering in recent days :)
10:21:54 stephenfin *first time
10:23:02 gibi python 3.7 adds stable dict key ordering https://docs.python.org/3/tutorial/datastructures.html#dictionaries
10:23:03 mdbooth Aren't all dicts ordered in the newest anguine offering?
10:23:07 stephenfin fwiw, Python 3.6+ dicts are insertion ordered, which is rather nice
10:23:10 mdbooth gibi: Snap ;)
10:23:16 stephenfin mdbooth: double snap
10:23:19 gibi :D
10:24:12 stephenfin I'm sure whoever is working on nova in 10 years or whenever we can drop Python < 3.6 support will be very happy
10:31:44 kashyap (Maybe there won't be any.)
10:35:16 neha_alhat_ cdent: yes, will check. Thanks
11:13:22 cdent johnthetubaguy: you still happy with https://review.openstack.org/#/c/543262/ ?
11:34:12 flwang1 i'm going to ask again, can anybody help understand how to set the local_gb for hypervisor? thanks
11:43:22 jroll flwang1: that's set by nova-compute when it updates its resources. it looks at the size of the block device that instances are on
11:43:53 flwang1 jroll: thanks, then my question is how can i change it?
11:44:39 jroll change local_gb? I don't believe you can. why do you want to?
11:45:38 flwang1 after installed devstack, i found the local_gb is too small because it's using my root /, though i have almost 800GB at /home
11:45:49 jroll ah
11:45:53 flwang1 but my / only has 30GB
11:46:06 jroll flwang1: there's a config option for where the instance data lives, you can change that
11:46:52 flwang1 ?
11:46:52 flwang1 #state_path = /opt/stack/data/nova
11:46:52 flwang1 #instances_path = /opt/stack/data/nova/instances
11:47:13 jroll yes, instances_path
11:47:34 flwang1 i changed them both and doesn't work
11:48:02 jroll did you restart nova-compute (and give it time to update resources)?
11:48:29 flwang1 yes, i did. i restarted all nova services
11:49:27 jroll hm, that's odd. I'm not sure if there's something else that needs to be done, hopefully someone else can help, sorry
11:50:53 flwang1 jroll: no worries, thank you for the help
11:51:01 jroll :)
11:52:32 efried Anyone happen to know where it's documented how/where we, as developers, are supposed to use translation macros in code?
11:53:39 flwang1 jroll: i think that's the issue, after restart the nova-compute, it's still trying to talk to the old path
11:53:42 flwang1 not sure why
11:54:29 jroll yeah, that would be my guess
11:54:49 jroll efried: this is a good start: https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html
11:54:59 efried Thanks jroll
11:55:00 jroll primarily "All text messages the user sees via exceptions or API calls should be marked for translation"
11:55:14 efried jroll: Yeah, I need something even more basic than that, for karimull.
11:55:41 openstackgerrit Yikun Jiang (Kero) proposed openstack/nova master: Fix all invalid obj_make_compatible test case https://review.openstack.org/574240
11:55:47 jroll I'm not sure how to make that more basic :/
11:56:52 efried jroll: See https://review.openstack.org/#/c/569498/10..11/nova/conductor/manager.py
11:59:22 efried I.e. need to describe what "marked for translation" means. I think that doc works.
11:59:37 jroll efried: yeah, I don't have anything better
12:00:06 efried karimull: After you've looked at the doc, let me know if you'd like some more background.
12:06:20 sahid stephenfin: thanks for your eye on this. I just replied
12:07:56 openstackgerrit Merged openstack/nova master: Handle compare in test_pre_live_migration_volume_backed* directly https://review.openstack.org/580702
12:18:51 mago_ Hi team, I'm working on GPU Passtrough on OpenStack. I would like to limit the number of GPUs available to a user / project. There is no quotas for Passthrough resources and I was wondering why ? I was googling around but found no where a similar question.
12:22:37 mago_ Side question, as quotas are not only for Nova, but also for the Network and Block Storage services, is there a better place to ask my question ?
12:23:30 openstackgerrit Yikun Jiang (Kero) proposed openstack/python-novaclient master: Microversion 2.64 - Use new format policy in server group https://review.openstack.org/578261
12:24:36 efried mago_: You mean like project-wide? Cause I would imagine there probably wouldn't be enough GPUs on a single host for quotas to make much sense.
12:24:49 efried mago_: but are you talking about physical or virtual GPUs?
12:26:55 jaypipes johnthetubaguy: https://review.openstack.org/#/c/543262/ could use your +2 restoration.
12:27:38 mago_ efried, well you could imagine a cluster with nodes with 4 to 8 GPU cards, if you have let's say 10 nodes, you can have up to 80 GPUs available. I think a quota would make sense.
12:27:44 jaypipes mago_: there is no quota class for either virtual or physical GPUs, no.
12:28:22 mago_ efried, I talk about GPU cards, passed to VMs using passthrough (also called Direct Device Assignment)
12:29:24 efried mago_: Yup, got it. I defer to jaypipes here, perhaps he can explain why there are no quotas.
12:30:08 jaypipes mago_: I can't explain *why* there are no quotas for GPUs. or at least, I can't explain that without some seriously opinionated rant about how our PCI manager system was created.
12:31:13 mago_ jaypipes, do you know why, I mean is it technically not possible because of the way quotas are implemented, or could it be imagined ? for GPUs, or for passthrough resources in general
12:32:12 jaypipes mago_: quotas were not considered by the creators of the pci management module in nova.
12:33:07 jaypipes mago_: when and if we move the management of PCI resources to the placement service, we will be able to use quotas for those PCI resource classes, but until then, it's not going to happen.
12:35:14 mago_ jaypipes, ok, understood. It does not seem to be an easy task, but do you know if this is already addressed / under discussion ?
12:38:13 mdbooth stephenfin gibi: Do we have something like a fixture which perturbs dicts? Running nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_pre_live_migration_volume_backed_encrypted fails for me literally every other time when running only that test.
12:38:20 mdbooth It's weird.
12:39:22 mdbooth I know dict order isn't guaranteed, but all things being equal I'd expect it to be consistent in practise unless something is deliberately making it inconsistent.
12:39:37 gibi mdbooth: I don't think we have. I think the order depends on the PYTHONHASHSEED
12:39:38 jaypipes like SR-IOV VFs are some of the resource types we do *not* currently track using the placement service. our work on nested resource providers, granular request groups, and the "reshaper" effort are paving the way to supporting these types of resources in the placement service. I predict that we'll have most of that work done by the end of the "T" release series, at which point the quotas-in-placement work can become a reality.
12:39:38 jaypipes mago_: we've been planning on moving the queries for calculation of resource usage out of the nova-api service and towards the placement service for a while (https://review.openstack.org/#/c/509042/) but until we are tracking all the different types of resources using the placement service consistently, such an effort is only ever going to yield a small benefit. PCI devices, NUMA topology and dedicated CPUs, and certain networking resources
12:39:39 efried mdbooth: There's an open bug and proposed fix for this.
12:39:50 mdbooth efried: Yeah, we discussed it earlier.
12:40:11 efried mdbooth: Okay, upon rereading I figured that's not what you were getting at :)
12:40:27 mdbooth But even knowing the bug, I'm still surprised by the behaviour in practise.
12:40:56 efried mdbooth: You mean that there's some apparent predictability to the unpredictability of the hash ordering?
12:41:11 mdbooth I mean, I'd expect potentially different ordering if we were creating objects in a different order, or running things in a different order, or doing *anything* different.
12:41:38 mdbooth But in 2 consecutive absolutely identical test runs I'm surprised to see a difference in behaviour unless we're intentionally perturbing it somehow.
12:41:44 gibi mdbooth: try the same PYTHONHASHSEED
12:41:54 mago_ jaypipes, efried thank you very much for the explanations.

Earlier   Later