| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-02-02 | |||
| 18:20:29 | mriedem | mdbooth: lyarwood: done with that review, only real questions are in that mount module | |
| 18:21:20 | jaypipes | johnthetubaguy: +W from me on you disk_gb patch. | |
| 18:28:53 | mriedem | is anyone working a fix for this? https://bugs.launchpad.net/nova/+bug/1659266 | |
| 18:28:53 | openstack | Launchpad bug 1659266 in OpenStack Compute (nova) "Disk allocation for instance is not good with swap" [High,Confirmed] - Assigned to Tanvir Talukder (tanvirt16) | |
| 18:29:18 | mriedem | bauzas: ^ you reported it | |
| 18:29:29 | EmilienM | 2 questions : | |
| 18:29:53 | EmilienM | 1. Where [cells] section needs to be configured (on which services, eg: nova-api? etc) | |
| 18:29:53 | mriedem | (1) who is your daddy and (2) what does he do? | |
| 18:30:04 | mriedem | EmilienM: cells config is only for cells v1 | |
| 18:30:06 | mriedem | you don't care about it | |
| 18:30:14 | EmilienM | 2. Do we *need* cells/enable= true in Ocata, | |
| 18:30:17 | mriedem | no | |
| 18:30:19 | EmilienM | amoralej: ^ | |
| 18:30:27 | mriedem | [cells] is all about cells v1 | |
| 18:30:44 | EmilienM | mriedem: ok | |
| 18:30:49 | amoralej | thanks mriedem | |
| 18:30:55 | EmilienM | yeah thx | |
| 18:31:01 | mriedem | i'll update the config option help to make that more clear | |
| 18:31:09 | EmilienM | mriedem: we're unable to spawn VMs anymore | |
| 18:31:28 | EmilienM | maybe you'll know why | |
| 18:31:47 | EmilienM | http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/b29f35b/logs/nova/nova-compute.txt.gz#_2017-02-02_12_57_00_754 | |
| 18:32:16 | amoralej | mriedem, we getg "Instance 6a26cec0-3769-44b6-851c-b77359897e07 could not be found." kind of errors in certain tempest tests | |
| 18:32:38 | EmilienM | mriedem: we're deploying trunk | |
| 18:32:41 | EmilienM | with UC | |
| 18:33:17 | johnthetubaguy | jaypipes: thanks! | |
| 18:35:56 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Clarify the [cells] config option help https://review.openstack.org/428344 | |
| 18:35:57 | mriedem | melwitt: dansmith: ^ | |
| 18:37:06 | mriedem | EmilienM: that trace says the bdm is invalid | |
| 18:37:37 | EmilienM | bdm? | |
| 18:37:40 | mriedem | EmilienM: because the volume isn't found http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/b29f35b/logs/nova/nova-compute.txt.gz#_2017-02-02_12_56_57_274 | |
| 18:37:45 | mriedem | bdm == block_device_mapping | |
| 18:37:48 | EmilienM | Block Device Mapping | |
| 18:37:49 | EmilienM | ok | |
| 18:38:01 | EmilienM | well, it worked yesterday | |
| 18:38:13 | mriedem | EmilienM: did it work with devstack? :P | |
| 18:38:19 | johnthetubaguy | mriedem: I can put something up for that bug 1659266 | |
| 18:38:19 | openstack | bug 1659266 in OpenStack Compute (nova) "Disk allocation for instance is not good with swap" [High,Confirmed] https://launchpad.net/bugs/1659266 - Assigned to Tanvir Talukder (tanvirt16) | |
| 18:38:26 | mriedem | johnthetubaguy: thanks | |
| 18:38:30 | EmilienM | mriedem: I was about to say it but well... | |
| 18:38:38 | johnthetubaguy | I have to run, soon, but lets see what can be done | |
| 18:38:50 | mriedem | johnthetubaguy: you're not allowed to leave | |
| 18:38:55 | EmilienM | mriedem: FYI we're delaying TripleO release because upgrades are not working currently from Newton to Ocata | |
| 18:38:55 | mriedem | sally will understand | |
| 18:39:07 | EmilienM | mriedem: we hope to solve blockers by next week :/ | |
| 18:39:14 | johnthetubaguy | mriedem: you haven't met the conductor, she is scary | |
| 18:41:15 | mriedem | EmilienM: my wild guess would be something related to https://review.openstack.org/#/c/427782/ | |
| 18:41:22 | mriedem | which merged last night | |
| 18:41:33 | mriedem | EmilienM: but would also mean you don't have things setup for cells v2 | |
| 18:42:07 | EmilienM | mriedem: do you run experimental job sometimes? | |
| 18:42:12 | EmilienM | it seems like you're not | |
| 18:42:44 | EmilienM | mriedem: I think we are pretty well covered for cells v2 but I'll need to confirm | |
| 18:43:00 | diana_clarke | EmilienM: I've run it, but I've yet to see it pass ;( | |
| 18:43:28 | EmilienM | diana_clarke: where? | |
| 18:43:48 | diana_clarke | EmilienM: for example: https://review.openstack.org/#/c/420186/ | |
| 18:44:10 | diana_clarke | (it's not recent, but people are running it) | |
| 18:44:52 | EmilienM | diana_clarke: cool | |
| 18:46:49 | dansmith | I run it too on occasion, but it takes a long time, fails often, and the logs are harder to sift through | |
| 18:47:03 | EmilienM | dansmith: ping me when it fails | |
| 18:47:06 | EmilienM | I'm happy to help | |
| 18:47:26 | openstackgerrit | John Garbutt proposed openstack/nova master: Stop swap allocations being wrong due to MB vs GB https://review.openstack.org/428352 | |
| 18:48:07 | EmilienM | dansmith: any idea about my bug mentioned above ^ | |
| 18:48:33 | dansmith | EmilienM: I looked a bit earlier, but I saw no debug logs to help track anything down | |
| 18:49:27 | EmilienM | dansmith: no debug on http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/b29f35b/logs/nova/nova-compute.txt.gz#_2017-02-02_12_56_57_274 ? | |
| 18:49:32 | johnthetubaguy | jaypipes: mriedem: its a quick cut and refactor from the previous fix, hopefully, at least thats what the unit tests suggested: https://review.openstack.org/#/c/428352 | |
| 18:49:58 | EmilienM | dansmith: could it be related to cells again? | |
| 18:50:05 | dansmith | EmilienM: no, I was looking at another run that had no debug, periodic maybe? | |
| 18:50:21 | EmilienM | dansmith: yeah periodic but this one has debug ^ | |
| 18:51:24 | dansmith | EmilienM: ah, I hadn't seen that volume error, and no that's not cells related AFAIK | |
| 18:51:57 | openstackgerrit | John Garbutt proposed openstack/nova master: Stop swap allocations being wrong due to MB vs GB https://review.openstack.org/428352 | |
| 18:52:05 | EmilienM | dansmith: could it be related to https://review.openstack.org/#/c/427782/ ? | |
| 18:52:55 | openstackgerrit | John Garbutt proposed openstack/nova master: DNM: hack ironic with resource providers https://review.openstack.org/426296 | |
| 18:53:44 | dansmith | EmilienM: not the volume error for sure no | |
| 18:54:31 | EmilienM | well, if you have any idea so far, please let me know. We're stuck. | |
| 18:56:19 | openstackgerrit | Maciej Szankin proposed openstack/nova master: conf: add min parameter to scheduler opts https://review.openstack.org/428355 | |
| 18:56:57 | mriedem | EmilienM: so the instance not found you were talking about is like this http://logs.openstack.org/15/424915/8/check/gate-puppet-openstack-integration-4-scenario001-tempest-centos-7/b29f35b/console.html#_2017-02-02_13_04_48_204185 | |
| 18:57:03 | mriedem | in a bunch of the tempest tests | |
| 18:57:47 | mriedem | it looks like it's waiting for the server to be active, it's gone | |
| 18:58:01 | amoralej | EmilienM, dansmith, reported in https://bugs.launchpad.net/nova/+bug/1661360 | |
| 18:58:01 | openstack | Launchpad bug 1661360 in OpenStack Compute (nova) "tempest test fails with "Instance not found" error" [Undecided,New] | |
| 18:58:04 | jroll | johnthetubaguy: there's an experimental job for resource class stuff, so I was going to abandon https://review.openstack.org/#/c/426296/ | |
| 18:58:11 | jroll | johnthetubaguy: but can keep it around if you rather | |
| 18:58:42 | mriedem | EmilienM: i can put up a debug patch if you guys can depend on it | |
| 18:58:48 | amoralej | mriedem, i've reproduced locally, nova server returns "instance not found" but it's found in "nova list" command | |
| 18:59:04 | mriedem | amoralej: that's probably a race between when we have a build request and the actual instance, | |
| 18:59:07 | mriedem | the api returns either | |
| 18:59:15 | mriedem | since they both represent a server at different stages of the lifecycle | |
| 18:59:22 | EmilienM | mriedem: yes I can | |
| 18:59:29 | EmilienM | mriedem: you can run check experimental | |
| 19:00:48 | dansmith | mriedem: what race? | |
| 19:01:02 | melwitt | mriedem: isn't that issue fixed by https://review.openstack.org/#/c/427775 this morning? | |
| 19:01:22 | dansmith | melwitt: there's no delete going on here, which is what ^ is for right? | |
| 19:01:31 | melwitt | oh. yeah the above is for delete | |
| 19:04:08 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: debugging for instance not found in tripleo https://review.openstack.org/428360 | |
| 19:04:09 | mriedem | EmilienM: amoralej: ^ | |
| 19:04:42 | cdent | edleafe: do you recall if this allocation checking is happening anywhere else because this seems important but is stuck behind something else: https://review.openstack.org/#/c/416752/ | |
| 19:05:07 | mriedem | melwitt: dansmith: even with my fix, | |
| 19:05:31 | mriedem | couldn't we have a window of time where the build request exists, the instance exists, but the instance mapping doesn't? | |
| 19:05:44 | dansmith | mriedem: sure, but then we return the buildrequest, that's the point | |
| 19:05:53 | dansmith | mriedem: in their logs, I do not see "Found 1 cells" | |
| 19:05:54 | edleafe | cdent: it's not really "stuck"; just hasn't gotten core attention | |