| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-08 | |||
| 15:41:49 | sean-k-mooney | spatel: i know you had that issue in the past but i can rememebr what it was again | |
| 15:42:29 | melwitt | mriedem: yeah, fair enough | |
| 15:59:22 | spatel | i think stats output doing something which list not doing.. that is why its difference | |
| 16:08:01 | sean-k-mooney | spatel: the hypervior stats i think only show running vms | |
| 16:08:33 | sean-k-mooney | nova list will also show shelved instnaces and ones in error or power off state | |
| 16:08:47 | spatel | if that is true then it make sense | |
| 16:09:14 | spatel | let me drill and see | |
| 16:11:04 | sean-k-mooney | oh wait you are not asking about nova list | |
| 16:11:27 | sean-k-mooney | you are asing about count in openstack hypervisor stats show vs the outpu of openstack hypervior list | |
| 16:11:47 | spatel | yes count | |
| 16:11:57 | spatel | now you are on same page :) | |
| 16:12:13 | spatel | sean-k-mooney: ^^ | |
| 16:16:28 | sean-k-mooney | spatel: try openstack hypervisor list -c "Host IP" -c "Hypervisor Type" -f value | grep QEMU | sort | uniq | wc -l | |
| 16:17:00 | spatel | 193 | |
| 16:17:33 | sean-k-mooney | ok i was wondering if you had duplicate entries | |
| 16:17:57 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Set the conductor indirection API when running nova-metadata under uwsgi https://review.openstack.org/635577 | |
| 16:18:10 | spatel | i verified already basic stuff | |
| 16:18:41 | spatel | i think openstack hypervisor stats show ( only showing hypervisor in use ) | |
| 16:19:14 | spatel | let me do experiment and see... let me empty or spin up vm and see count change or not | |
| 16:22:27 | sean-k-mooney | spatel: well teh api doces say it should the number of hyperviors | |
| 16:22:29 | sean-k-mooney | https://developer.openstack.org/api-ref/compute/?expanded=show-hypervisor-statistics-detail#show-hypervisor-statistics | |
| 16:22:47 | sean-k-mooney | so if it the number of active or in use hyperviors there is at least a docs bug | |
| 16:23:39 | spatel | let me prove it and open bug | |
| 16:23:53 | spatel | did you try to test these command in your cloud? | |
| 16:24:08 | mriedem | dansmith: melwitt: i'm +2 on the bottom 3 down cell changes in that series now https://review.openstack.org/#/c/635120/ | |
| 16:24:10 | sean-k-mooney | yes and they are the same | |
| 16:24:23 | mriedem | splitting those out from the big patch was definitely the way to go | |
| 16:24:24 | sean-k-mooney | ill disabel a compute service and see if it changes | |
| 16:25:43 | sean-k-mooney | spatel: ok i can repoduce | |
| 16:25:59 | sean-k-mooney | if the comptue node is disabled hypervior stats does not report it | |
| 16:26:19 | sean-k-mooney | so that is a bug either in the code or in the docs | |
| 16:26:54 | spatel | oh!!! | |
| 16:27:32 | melwitt | mriedem: ack | |
| 16:28:20 | spatel | sean-k-mooney: you are right!! i have around 10 or 12 disabled compute node | |
| 16:28:44 | sean-k-mooney | spatel: what release are you running again? | |
| 16:28:50 | spatel | pike | |
| 16:28:54 | spatel | sorry queens | |
| 16:29:35 | sean-k-mooney | i am not sure if we changed it in rocky or queens but we used to auto disable compute nodes after 3-5 failed builts without a successful build | |
| 16:29:49 | sean-k-mooney | if you manually did not disable them that coudl be the cause | |
| 16:30:35 | spatel | i didn't remember when and who did disable it.. but at least clear that stats not showing disable | |
| 16:31:04 | spatel | Do you know how to list disable hypervisor from command line? | |
| 16:32:57 | dansmith | mriedem: okay cool, glad to see that, I'll look a bit later | |
| 16:47:25 | sean-k-mooney | spatel: openstack compute service list | |
| 16:48:13 | spatel | thanks | |
| 16:51:47 | spatel | sean-k-mooney: 179 + 14 (disable) = 193 | |
| 16:52:09 | spatel | I got my answer :) thanks | |
| 16:55:47 | sean-k-mooney | spatel: can you file a bug just so we dont forget | |
| 16:56:09 | sean-k-mooney | as i said we either should update the docs or the code | |
| 17:02:09 | spatel | i think we should update code and add one more row ( disabled compute node ) ;) | |
| 17:21:00 | mriedem | ew another subunit parser giant log capture http://logs.openstack.org/27/619527/16/check/openstack-tox-py35/ab8a233/job-output.txt.gz#_2019-02-08_14_57_03_415588 | |
| 17:21:41 | mriedem | gross that's from o.vo | |
| 17:22:06 | mriedem | so i suppose we need to set log levels to debug for oslo.versionedobjects and oslo.messaging in our test runs | |
| 17:22:16 | mriedem | s/debug/warning/ | |
| 17:22:35 | mriedem | although for oslo.messaging we're tracing exceptions in those conductor unit tests... | |
| 17:22:44 | sean-k-mooney | mriedem: one question do we know what the cause is? i was wondering if it could be a locale issue or something like that | |
| 17:24:12 | melwitt | sean-k-mooney: I don't think so. see the comments in this earlier bug for more info https://bugs.launchpad.net/cinder/+bug/1728640 | |
| 17:24:14 | openstack | Launchpad bug 1728640 in Cinder "py35 unit test subunit.parser failures" [Critical,Fix released] - Assigned to Sean McGinnis (sean-mcginnis) | |
| 17:25:08 | mriedem | i think the summary is we're sending a shit load of content to the subunit output stream capture which blows it up | |
| 17:25:52 | sean-k-mooney | ya that was the other ting i was wondering but i guess that means we are not mocking out the loggers enough in the unit tests | |
| 17:26:14 | melwitt | yeah. from mtreinish: "we may be exceeding the max attachment size in subunit" | |
| 17:26:35 | melwitt | (from a comment in that bug) | |
| 17:27:21 | sean-k-mooney | on the plus side if we do reduce the log output it will both help with space on logs.openstack.org and maybe speed up the tests | |
| 17:27:36 | sean-k-mooney | all that io is proably slowing them down | |
| 17:34:38 | mriedem | well normally i don't think you get a lot of this output unless a test fails | |
| 17:34:52 | mriedem | or the subunit parser blows up | |
| 17:35:38 | openstackgerrit | Elod Illes proposed openstack/nova stable/queens: Handle IndexError in _populate_neutron_binding_profile https://review.openstack.org/635897 | |
| 17:59:31 | mnaser | mriedem, cdent, dansmith: https://review.openstack.org/#/c/635852/ has an openstack ansible cross repo job (which you can see installs from the zuul cloned nova -- http://logs.openstack.org/52/635852/1/check/nova-openstack-ansible-cross-repo/9738be9/logs/ara-report/result/113b6602-4a28-4512-964b-4174593eb507/ "Processing /home/zuul/src/git.openstack.org/openstack/nova") | |
| 17:59:55 | mnaser | i will add another tox env which will run it making sure that we *skip* placement deploy to merge that eventually | |
| 18:00:15 | mnaser | and then the normal functional test will run *with* out-of-repo placement | |
| 18:00:26 | cdent | awesome, thanks | |
| 18:02:53 | cdent | I'll look more closely soon, probably monday | |
| 18:06:53 | dansmith | mriedem: should I not nit out over the presumptive microversion comment? https://review.openstack.org/#/c/635121/4 | |
| 19:10:54 | aspiers | is there a reason that the trait string values in this dict are not referenced using os_traits constants? http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/utils.py#n49 | |
| 19:11:39 | aspiers | wouldn't it be helpful to have the compile-time checks resulting from using os_traits? or am I misunderstanding something? | |
| 19:13:39 | cdent | aspiers: I think it's a matter of "didn't happen yet" | |
| 19:13:52 | aspiers | cdent: so a patch would be gratefully received? :) | |
| 19:14:10 | cdent | ask fried_rice or leakypipes to confirm, but I suspect so | |
| 19:14:16 | cdent | I'm not savvy to that section of the coe | |
| 19:14:16 | aspiers | OK thanks | |
| 19:14:20 | cdent | and it is dinner time | |
| 19:16:50 | openstackgerrit | Corey Bryant proposed openstack/nova master: xenapi/agent: Change openssl error handling https://review.openstack.org/635533 | |
| 19:22:25 | cfriesen | If I were to request a release of os-traits (in order to make use of the new vTPM traits) would I ask for release 0.10.1 or 0.11.0? | |
| 19:23:03 | melwitt | new traits = "new features" so a minor version bump | |
| 19:23:35 | aspiers | melwitt: speaking of traits, do you have any thoughts on the above? I can submit a review easily | |
| 19:25:13 | leakypipes | aspiers: no reason other than legacy I think... I'd welcome a cleanup patch. | |
| 19:25:28 | melwitt | ok, good. I was about to say, I don't know and I'd ask jay | |
| 19:25:36 | aspiers | :) | |
| 19:25:50 | aspiers | leakypipes, melwitt: coming in about 15 seconds ... | |
| 19:25:51 | cfriesen | aspiers: I see you just added some stuff to os-traits...were you planning on requesting a new release? | |
| 19:26:13 | openstackgerrit | Adam Spiers proposed openstack/nova master: Convert CPU_TRAITS_MAPPING to use os_traits https://review.openstack.org/635919 | |
| 19:26:16 | aspiers | cfriesen: don't need one urgently yet - still working on the code to consume the new trait :) | |
| 19:26:39 | cfriesen | aspiers: I'm in the same boat, just don't want to get caught waiting too long | |
| 19:26:52 | aspiers | cfriesen: fair enough | |
| 19:27:33 | aspiers | leakypipes, melwitt: added you as reviewers | |
| 19:28:01 | mriedem | dansmith: yeah good point, that should move | |
| 19:28:10 | dansmith | I shall convert | |
| 19:28:27 | dansmith | I shall wait until you drop your +2 | |
| 19:28:31 | melwitt | aspiers: ack | |
| 19:28:44 | mriedem | dansmith: already done | |
| 19:28:55 | dansmith | mriedem: I shall be already done as well | |