| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-01 | |||
| 19:21:09 | mriedem | dtroyer: well, one thing i guess is GET /os-cells would return something for cells v1 if enabled https://developer.openstack.org/api-ref/compute/#cells-os-cells-capacities | |
| 19:21:17 | mriedem | that's admin-only by default | |
| 19:21:19 | mriedem | the api i mean | |
| 19:21:36 | mriedem | but cells v1 runs the separate n-cell service, that's how devstack checks it | |
| 19:21:46 | mriedem | dtroyer: cells v2 is baked in, not a separate service or API | |
| 19:21:55 | dtroyer | mriedem: thanks. I'm trying to do graceful failures in OSC in the aggregate commands so admin-only is OK | |
| 19:21:58 | mriedem | dtroyer: so starting in ocata, you're running cells v2 by design | |
| 19:22:00 | dtroyer | ah, gotcha | |
| 19:22:10 | mriedem | even if you're running cells v1, you have cells v2 :) | |
| 19:22:14 | mriedem | consider your mind blown | |
| 19:22:45 | mriedem | dtroyer: the other thing would probably be the /os-services API | |
| 19:23:08 | mriedem | https://github.com/openstack/nova/blob/master/nova/cmd/cells.py#L41 | |
| 19:23:08 | dtroyer | mriedem: one more question… I have a note that nova-net required cells v1, that doesn't seem right to me… | |
| 19:23:23 | mriedem | if you're running cells, there should be a nova-cells service binary in the GET /os-services response | |
| 19:23:35 | mriedem | i think anyway | |
| 19:23:43 | mriedem | dtroyer: other way around | |
| 19:23:47 | mriedem | well, | |
| 19:24:01 | mriedem | cells v1 support in-tree requires nova-net, | |
| 19:24:17 | dtroyer | ok, that explains a couple of things. thanks | |
| 19:24:29 | mriedem | dtroyer: in ocata we made nova-network only start under a cells v1 config | |
| 19:24:32 | mriedem | that's probably the confusion | |
| 19:24:48 | dtroyer | I bet so | |
| 19:25:07 | mriedem | we basically just continued to disable nova-net in ocata, | |
| 19:25:15 | mriedem | and switched all jobs to use neutron except for the cells v1 job | |
| 19:31:13 | dansmith | jaypipes: I don't hate your sql, I admire it. | |
| 19:31:18 | dansmith | jaypipes: I just note that I shan't be debugging it if something goes wrong | |
| 19:31:34 | cdent | tshirt | |
| 19:39:21 | openstackgerrit | Eric Fried proposed openstack/nova-specs master: Updates for [glance]api_servers and ksa https://review.openstack.org/461481 | |
| 19:53:09 | openstackgerrit | Ed Leafe proposed openstack/nova master: Remove the can_host column https://review.openstack.org/461494 | |
| 19:56:21 | melwitt | jaypipes: fyi I pushed an update to the quotas series attempting to address your comments https://review.openstack.org/#/c/446239 | |
| 19:56:29 | jaypipes | melwitt: yup, reviewing it now :) | |
| 19:56:37 | jaypipes | while waiting for functional tests to complete.. | |
| 19:57:01 | melwitt | jaypipes: cool, thanks :) | |
| 20:02:24 | ildikov | jaypipes: around for a meeting on #openstack-meeting-3? :) | |
| 20:03:03 | jaypipes | ildikov: I'm sorry, Ildiko, I'm right in the middle of debugging a nasty little feature and really need to focus on it. | |
| 20:03:06 | jaypipes | sorry :( | |
| 20:03:16 | jaypipes | ildikov: will read back from channel later. | |
| 20:03:29 | ildikov | jaypipes: ok, let's catch up later then | |
| 20:04:52 | jaypipes | thx ildikov | |
| 21:10:04 | openstackgerrit | Eric Fried proposed openstack/nova master: nova.utils.get_service_url() https://review.openstack.org/458257 | |
| 21:16:54 | mriedem | oh boy, so my daughter got home and there is a note from the teacher in her backpack about inappropriate behavior, | |
| 21:17:09 | mriedem | e.g. a worksheet of sight words to fille out, every entry has the word "butt" in it | |
| 21:17:34 | mriedem | she at least spelled it correctly... | |
| 21:23:48 | cdent | \o/ | |
| 21:25:39 | dansmith | mriedem: sight words? | |
| 21:26:08 | mriedem | dansmith: yeah, like simple single-syllable words that they should recognize immediately | |
| 21:26:12 | openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Debug race in delete-while-booting test https://review.openstack.org/461513 | |
| 21:26:14 | mriedem | they, here, were, are, them | |
| 21:26:22 | mriedem | butt | |
| 21:26:25 | dansmith | mriedem: and butt is inappropriate? | |
| 21:26:37 | dansmith | maybe the teacher should read "everybody poops" | |
| 21:26:39 | imacdonn | I'd recognise a butt anywhere! | |
| 21:26:40 | mriedem | well, not only that, but the picture of a butt pooping | |
| 21:26:59 | dansmith | mriedem: I dunno, I think I'd go argue with the teacher | |
| 21:28:33 | cdent | tbachman++ | |
| 21:28:59 | dansmith | tbachman: that would be inappropriate (apparently) | |
| 21:30:39 | edleafe | mriedem: the apple doesn't fall far... | |
| 21:32:28 | melwitt | wat is this "UnicodeDecodeError \'ascii\' codec can\'t decode byte 0xc3 in position 3575: ordinal not in range(128)" coming from the cells v1 tempest job | |
| 21:32:29 | mriedem | edleafe: it's true, it reminds me of things i got in trouble for in school | |
| 21:32:47 | mriedem | melwitt: likely means we aren't using six.text_type somewhere | |
| 21:33:04 | mriedem | melwitt: link and stacktrace? | |
| 21:33:22 | mriedem | like, LOG.warning('something %s' % unicode) will do that | |
| 21:34:51 | melwitt | mriedem: http://logs.openstack.org/39/446239/5/check/gate-tempest-dsvm-cells-ubuntu-xenial/7061c58/logs/screen-n-cell-region.txt.gz?#_May_01_20_22_15_324264 | |
| 21:35:37 | mriedem | hmm https://github.com/openstack/nova/blob/master/nova/cells/messaging.py#L1011 | |
| 21:35:47 | cdent | edleafe: I guess you didn't run all the functional tests before pushing that can_host change. I can no longer trust you! | |
| 21:36:00 | mriedem | melwitt: i've seen something like that in n-cpu where we log an instance with plugging vifs | |
| 21:36:11 | mriedem | it dumps the entire instance and there must be something in there that has unicode in it | |
| 21:36:23 | melwitt | hm, okay. thanks | |
| 21:36:26 | cdent | melwitt: saw some melwitt I've just started seeing that too | |
| 21:36:31 | mriedem | melwitt: https://bugs.launchpad.net/nova/+bug/1580728 | |
| 21:36:31 | openstack | Launchpad bug 1580728 in OpenStack Compute (nova) "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 386: ordinal not in range(128) in nova.virt.libvirt.vif:unplug with unicode instance.display_name" [Medium,Triaged] | |
| 21:36:33 | melwitt | I don't think I did anything to cause that but maybe I did | |
| 21:37:41 | mriedem | in ^ it's unicode in the instance display name | |
| 21:37:43 | cdent | http://logs.openstack.org/13/459413/5/check/gate-tempest-dsvm-cells-ubuntu-xenial/da51ac2/console.html#_2017-05-01_20_33_50_887866 | |
| 21:37:45 | melwitt | cdent: ack. good it's not my patch maybe probably | |
| 21:38:15 | melwitt | that's weird, I wonder what made this start happening all of a sudden. implies something recently changed to unicode | |
| 21:38:55 | mriedem | check logstash | |
| 21:40:40 | mriedem | i only see 2 hits in logstash | |
| 21:41:11 | melwitt | doing. I also found earlier today that tempest.api.volume.admin.test_volume_quota_classes.VolumeQuotaClassesTest.test_update_default_quota fails intermittently. I'm guessing it's something to do with parallel tests racing | |
| 21:42:55 | cdent | mriedem: what did you search on? | |
| 21:43:01 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Handle uuid in HostAPI._find_service https://review.openstack.org/461519 | |
| 21:43:11 | mtreinish | melwitt: heh, the default quota tests are an inherent race condition | |
| 21:43:11 | mriedem | cdent: i deleted it :) sec | |
| 21:43:36 | melwitt | message:"UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 413: ordinal not in range(128)" | |
| 21:43:38 | mriedem | melwitt: the default quota compute api tests in tempest are locked to run serially | |
| 21:43:46 | cdent | cuz i've got 259 in the last 6 hours on message:"decode byte 0xc3 in position" which is clearly too general, but... | |
| 21:43:47 | melwitt | is what I searched on and see the same, only two hits | |
| 21:43:54 | mtreinish | but I think we set it to have sane defaults so we don't ever decrease the default quotas beyond where they start (which would be a big problem) | |
| 21:44:04 | mriedem | message:"UnicodeDecodeError" AND message:"instance_update_at_top" AND tags:"screen-n-cell-region.txt" | |
| 21:45:58 | edleafe | cdent: I was meaning to mark it WIP and forgot. I knew it wasn't fully tested. | |
| 21:46:15 | melwitt | mtreinish, mriedem: so they're set to run serially ... the thing I see is e.g. (u'snapshots_tempest-QosSpecsTestJSON-volume-type-1219921907', -1) not in [(u'gigabytes', 1001), (u'backups', 11), (u'snapshots_lvmdriver-1', -1), (u'volumes_lvmdriver-1', -1), (u'volumes', 11), (u'per_volume_gigabytes', -1), (u'backup_gigabytes', 1001), (u'gigabytes_lvmdriver-1', -1), (u'snapshots', 11)] | |
| 21:46:21 | cdent | edleafe: my bad really, I was being far too optimistic | |
| 21:47:29 | melwitt | mtreinish, mriedem: as though some other test had added a quota default and it gets picked up unexpectedly in another | |
| 21:48:06 | cdent | g'night all | |
| 21:50:39 | mtreinish | melwitt: do you have a link to that failure? | |
| 21:51:03 | melwitt | mtreinish: yeah, sorry I didn't include it http://logs.openstack.org/39/446239/5/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/0b6acd6/logs/testr_results.html.gz | |
| 21:57:11 | mtreinish | melwitt: that's really weird. It looks like tempest sends a PUT with the new default quota entries, cinder returns 200, and the tempest does a GET to list the default quotas and the stuff it just added isn't there | |
| 21:58:41 | mtreinish | melwitt: oh, I think you might be right about a race | |
| 21:58:44 | mriedem | dansmith: while looking at this https://github.com/openstack/nova/blob/master/nova/compute/api.py#L4508 for using uuid rather than id, i realized this could have been a direct lookup if we stored the compute node uuid on the host mapping, but we only store the ComputeNode.host field... | |
| 21:59:15 | melwitt | mtreinish: hm, what I thought is that update_body is returning an additional unexpected item that was added by a different test | |