| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-10-04 | |||
| 18:29:12 | andreykurilin | mriedem: so rally is quite simple and doesn't require any additional services to be installed. All the results are stored in the sql-dabase. One rally installation can check different clusters. The latest clients work perfectly with older releases of openstack. For example we have regular job which launch the scenarios at mitaka release | |
| 18:29:39 | mriedem | ok, i'm mostly interested in scheduling performance between ocata and pike, | |
| 18:30:01 | mriedem | i only have a single node devstack, so i'd have to get creative with mocking out computes | |
| 18:31:08 | mriedem | i know devstack has some flags for mocking out fake compute services when using the fake virt driver, but not sure if that works anymore with the tiered conductor stuff in devstack | |
| 18:31:16 | mriedem | i could run it in singleconductor mode to workaround that | |
| 18:36:04 | openstackgerrit | Merged openstack/nova master: Log consumer uuid when retrying claims in the scheduler https://review.openstack.org/507705 | |
| 18:36:38 | openstackgerrit | Merged openstack/nova master: [placement] gabbi tests for shared custom resource class https://review.openstack.org/485209 | |
| 18:38:38 | andreykurilin | mriedem: btw, I know some guys from huawei who are working on inner performance testing of openstack(as far as I know). I'll contact them about the hardware for such testing as you need | |
| 18:39:13 | mriedem | andreykurilin: are they in toronto? | |
| 18:39:17 | mriedem | hongbin's team? | |
| 18:39:44 | andreykurilin | mriedem: they are from China | |
| 18:39:48 | mriedem | oh | |
| 18:40:10 | mriedem | well, if you can get answers on internal openstack perf testing at huawei faster than i can, kudos to you sir | |
| 18:41:10 | andreykurilin | mriedem: heh. At least I'll try, but it will happen tomorrow, since they are slipping now | |
| 18:41:34 | mriedem | it's a chinese national holiday this week too | |
| 18:41:48 | openstackgerrit | priyaduggirala proposed openstack/nova master: Rename parameters in call() of nova/image/glance.py https://review.openstack.org/508533 | |
| 18:42:00 | melwitt | clarkb: sorry, was on a call. what did you mean by get the order sorted out between the uca package being installed and the pip package being installed? | |
| 18:43:03 | melwitt | mriedem: yeah, there's no ubuntu bug I don't think, because I think this is a unique problem to having devstack install PasteDeploy via pip and then the installation of ceph packages in the devstack plugin installing the uca packages | |
| 18:44:04 | melwitt | I thought the same, that I could probably do something like try uninstalling the uca python-pastedeploy package in the ceph devstack plugin and see if that would work | |
| 18:44:15 | clarkb | melwitt: basically make sure that apt-get happens first, then pip install, then keystone start | |
| 18:45:18 | melwitt | clarkb: ah, okay thanks. I'll see if that's possible in devstack plugin land, if there's a hook point for plugins that occurs before the pip installs that devstack does | |
| 18:46:04 | andreykurilin | mriedem: oh... yes, I forgot about it | |
| 18:46:09 | melwitt | because the devstack ceph plugin is doing the apt-get installs | |
| 18:48:17 | mriedem | melwitt: should be in this phase https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/plugin.sh#L12 | |
| 18:48:41 | mriedem | check the logs and see if that happens before or after the pip install happens | |
| 18:49:12 | melwitt | hm, yeah ... so it should be doing the right thing already. looking | |
| 18:54:07 | melwitt | says "Requirement already satisfied: PasteDeploy===1.5.2 in /usr/lib/python2.7/dist-packages" which means it skipped the pip install | |
| 18:54:13 | melwitt | (I think) | |
| 18:54:17 | melwitt | so how is keystone not finding it | |
| 18:54:24 | openstackgerrit | Merged openstack/nova master: fix nova accepting invalid availability zone name with ':' https://review.openstack.org/490722 | |
| 18:55:01 | melwitt | I had thought that because PasteDeploy was in the pip freeze output, that it had been pip installed too http://logs.openstack.org/32/508432/1/check/gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial/575e932/logs/pip2-freeze.txt.gz | |
| 18:55:17 | melwitt | but I guess that's not necessarily true | |
| 18:55:18 | mriedem | not necessarily | |
| 18:55:32 | mriedem | i remember yum installed packages show up in pip freeze output too | |
| 18:55:43 | mriedem | you just generally aren't using pip with rpms because things get wonky | |
| 18:56:03 | melwitt | yeah, huh. | |
| 18:57:25 | mriedem | right so it's setup via deb first http://logs.openstack.org/32/508432/1/check/gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial/575e932/logs/devstacklog.txt.gz | |
| 18:57:39 | mriedem | and then pip ignores it http://logs.openstack.org/32/508432/1/check/gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial/575e932/logs/devstacklog.txt.gz#_2017-10-04_01_50_59_811 | |
| 18:57:41 | clarkb | hrm pastedeploy 1.5.2 is latest | |
| 18:57:53 | clarkb | so is the ubuntu package just broken and not including all the bits? | |
| 18:57:58 | mriedem | oops first link is http://logs.openstack.org/32/508432/1/check/gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial/575e932/logs/devstacklog.txt.gz#_2017-10-04_01_49_51_453 | |
| 18:58:13 | melwitt | do you think it has anything to do with this? wsgi-file = /usr/local/bin/keystone-wsgi-public http://logs.openstack.org/32/508432/1/check/gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial/575e932/logs/etc/keystone/keystone-uwsgi-public.ini.txt.gz | |
| 18:58:16 | mriedem | we are using pike uca now | |
| 18:58:17 | mriedem | as of last week | |
| 18:58:41 | melwitt | is the /usr/local/ there causing keystone to only look under /usr/local for paste deploy? | |
| 18:59:51 | melwitt | clarkb: hm, maybe. I hadn't considered the package could be broken | |
| 19:00:10 | cfriesen | is there a way to specify running a single functional test class? | |
| 19:00:27 | clarkb | cfriesen: tox -e functional -- ClassNameHere ? | |
| 19:00:32 | melwitt | mriedem: yeah, the ceph-mgr started pulling in python-pastedeploy starting in the pike uca | |
| 19:00:35 | clarkb | that actually takes a regex (or should) | |
| 19:00:59 | cfriesen | clarkb: will try it...I had done that with the full dotted path to the class and it didn't like it. | |
| 19:01:15 | artom | cfriesen, clarkb's right, it should be a regex | |
| 19:01:22 | artom | So just ClassName should do it | |
| 19:01:30 | artom | Or even assName, because it matches as wlel | |
| 19:01:38 | mriedem | sorry make that monday https://review.openstack.org/#/c/505674/ | |
| 19:02:08 | cfriesen | artom: clarkb: nope. I ran "tox -e functional -- AggregateMigrationTestCase" and it started doing tests from nova.tests.functional.api.openstack.placement.test_placement_api | |
| 19:02:32 | melwitt | cfriesen: do you have an old checkout? there was some issues around the regex functionality that got fixed while we were at the PTG | |
| 19:02:44 | artom | cfriesen, hrmm - try without the --? | |
| 19:02:45 | mriedem | cfriesen: rebuild | |
| 19:02:48 | cfriesen | melwitt: entirely possible | |
| 19:02:49 | mriedem | tox -r -e | |
| 19:03:48 | melwitt | cfriesen: https://github.com/openstack/nova/commits/master/tox.ini I'd make sure you have the latest version of tox.ini | |
| 19:08:26 | cfriesen | combination of a fresh pull and tox -r seems to have done the trick | |
| 19:08:27 | cfriesen | thansk | |
| 19:14:40 | openstackgerrit | Matt Riedemann proposed openstack/nova master: use already loaded BDM in instance. |
|
| 19:14:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: use already loaded BDM in instance. |
|
| 19:14:41 | openstackgerrit | Matt Riedemann proposed openstack/nova master: use already loaded BDM in instance.create https://review.openstack.org/483969 | |
| 19:18:06 | krypto | hello all i have a strange problem with Mitaka release ,on one compute if i enable debug to true nova-compute is restarting continously with error "UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 335: ordinal not in range(128)" with out debug everything is working as expected ,any idea? | |
| 19:20:00 | mriedem | krypto: there is likely some debug-only log message hitting something that's not encoded properly | |
| 19:20:08 | mriedem | i probably know what / where it is too | |
| 19:20:34 | mriedem | https://bugs.launchpad.net/nova/+bug/1624277 is similar | |
| 19:20:35 | openstack | Launchpad bug 1624277 in OpenStack Compute (nova) mitaka "nova-scheduler: UnicodeDecodeError in host aggregates handling" [Medium,Confirmed] - Assigned to Nazeema Begum (nazeema123) | |
| 19:21:04 | mriedem | krypto: look at this fix in oslo.log https://review.openstack.org/#/c/371582/ which was never released in the mitaka series | |
| 19:21:19 | cfriesen | mriedem: I tweaked the code slightly to test the host aggregate migration, and confirmed that we don't soft-delete the entries in table 'aggregate_hosts'. Diff is here, with functional test output at bottom: http://paste.openstack.org/show/622702/ This shouldn't affect normal code, but it messed up one of our audits that verified DB consistency. | |
| 19:21:24 | mriedem | looks like oslo.log 3.9.0 has that fix | |
| 19:22:57 | mriedem | krypto: i've also seen one in nova-compute when logging the vif | |
| 19:23:01 | mriedem | i can't find the bug for it though | |
| 19:23:07 | krypto | i see thank you mriedem ,thanks alot i didnt know that turning debug=true could bring down nova-compute :) not sure whats wrong with my compute | |
| 19:24:28 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add alternate hosts https://review.openstack.org/486215 | |
| 19:24:29 | openstackgerrit | Ed Leafe proposed openstack/nova master: Add Selection objects https://review.openstack.org/499239 | |
| 19:24:30 | openstackgerrit | Ed Leafe proposed openstack/nova master: Return Selection objects from the scheduler driver https://review.openstack.org/495854 | |
| 19:25:24 | mriedem | krypto: you would need to capture the stacktrace from the n-cpu logs and put those in a paste to know where the actual problem needs to be fixed, or if it's already fixed and you can cherry pick that fix to your mitaka env, since mitaka is eol upstream | |
| 19:26:02 | mriedem | it might not be in nova, it could be in glanceclient, i see fixed bugs for the same thing in glanceclient from several years ago | |
| 19:26:09 | krypto | okk let me check how to do that | |
| 19:26:16 | mriedem | http://paste.openstack.org/ | |
| 19:28:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: use already loaded BDM in instance. |
|
| 19:28:09 | openstackgerrit | Matt Riedemann proposed openstack/nova master: use already loaded BDM in instance.create https://review.openstack.org/483969 | |
| 19:28:17 | cfriesen | is there a way to boot an instance from a volume-exposed-as-cdrom, install to another volume-exposed-as-disk, then unmount the CDROM volume? Or to change boot order of volumes in the instance after it's created? | |
| 19:36:29 | rybridges | Hello. Did we remove the ability to list servers by project/tenant id in the novaclient? | |
| 19:36:52 | rybridges | nova list --tenant-id <tenant_id> is not working. same if i try with openstack client | |
| 19:37:11 | rybridges | have tried all combinations of tenant_id, tenant-id, project_id, project-id in both openstack client and novaclient | |
| 19:37:25 | rybridges | but I see it in the documentation here https://developer.openstack.org/api-ref/compute/#listServers | |
| 19:37:37 | mriedem | rybridges: which version of novaclient? | |
| 19:38:30 | mriedem | nova help list | |
| 19:38:32 | mriedem | shows me: | |
| 19:38:37 | mriedem | --tenant [<tenant>] Display information from single tenant (Admin | |
| 19:38:38 | mriedem | only). | |
| 19:39:01 | melwitt | you might have to specific --all-tenants too. and make sure tenant is a UUID | |
| 19:39:04 | melwitt | *specify | |
| 19:39:18 | mriedem | nova list --tenant f7832bfb6daf4a00b6c5687b2ab756fd totally works for me | |