| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-19 | |||
| 23:07:29 | efried | dustinc: See the predecessor patch at https://review.openstack.org/#/c/643664/5/nova/utils.py@1239 | |
| 23:08:11 | efried | dustinc: So the only change needed to have ironic make use of that was here: https://review.openstack.org/#/c/642899/16/nova/virt/ironic/driver.py@188 | |
| 23:08:31 | efried | dustinc: And tbc, this works, which you can tell by looking at the CI results, if you know what you're looking for: | |
| 23:08:55 | efried | The job called ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa only succeeds if ^ is copacetic | |
| 23:09:08 | efried | Here's the last run (before the latest rebase): http://logs.openstack.org/99/642899/15/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/c032d49/ | |
| 23:09:14 | efried | http://logs.openstack.org/99/642899/15/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/c032d49/testr_results.html.gz | |
| 23:09:42 | efried | The other red stuff in the CI report is the unit and functional tests - for those, it's the actual tests that need to be fixed, not the code. | |
| 23:09:58 | efried | and most of the test fixage needs to happen in the predecessor patch - which is what I've been working on. | |
| 23:10:11 | efried | made a little progress today, but definitely didn't finish. | |
| 23:10:16 | efried | a couple of serious mysteries in there still. | |
| #openstack-nova - 2019-03-20 | |||
| 01:55:03 | eandersson | What is the difference between disk_available_least and disk_free_gb ? | |
| 02:23:26 | openstackgerrit | Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765 | |
| 02:26:38 | eandersson | It looks like maybe disk_available_least takes into account the qcow images on the host? | |
| 02:28:11 | eandersson | because the host has 79GB available, the flavor uses 74GB, but the scheduler thinks its out of disk space so refuses to schedule there | |
| 02:28:33 | eandersson | but if we schedule there manually using --availability-zone nova:<compute-name> | |
| 02:28:35 | eandersson | it works | |
| 02:29:01 | eandersson | and it now has ~4GB disk free according to the nova.compute logs | |
| 02:30:38 | eandersson | > does not have 76800 MB usable disk, it only has 44031.0 MB | |
| 02:31:10 | eandersson | So confusing that the disk space used to schedule isn't reported anywhere, or at the very least not the number the scheduler uses to make decisions upon isn't reported. | |
| 02:40:57 | eandersson | After scheduling manually I can see that disk_available_least is -33 | |
| 02:41:07 | eandersson | but free_disk_gb is at 4 | |
| 03:56:23 | openstackgerrit | Merged openstack/nova master: Remove "Fixing the Scheduler DB model" from schedule evolution doc https://review.openstack.org/643615 | |
| 04:55:34 | eandersson | maybe jaypipes ? ^ | |
| 05:09:14 | openstackgerrit | Merged openstack/nova master: Add docs for compute capabilities as traits https://review.openstack.org/644293 | |
| 05:17:35 | openstackgerrit | Yongli He proposed openstack/nova master: Clean up orphan instances https://review.openstack.org/627765 | |
| 07:15:46 | openstackgerrit | Seyeong Kim proposed openstack/nova stable/rocky: Share snapshot image membership with instance owner https://review.openstack.org/643853 | |
| 09:02:54 | openstackgerrit | Michael Still proposed openstack/nova master: Complete remove fake_libvirt_utils. https://review.openstack.org/643897 | |
| 09:02:55 | openstackgerrit | Michael Still proposed openstack/nova master: Remove fake_libvirt_utils users in functional testing. https://review.openstack.org/644793 | |
| 09:28:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add known issue for minimum bandwidth resource leak https://review.openstack.org/644694 | |
| 09:42:06 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Misc cleanups https://review.openstack.org/644616 | |
| 10:08:53 | mdbooth | stephenfin: Hey, so it turns out sphinx also imports nova/api/openstack/__init__.py | |
| 10:09:15 | mdbooth | stephenfin: My patch is adding monkey patching there, so sphinx is still borked | |
| 10:10:11 | mdbooth | stephenfin: I moved monkey patching there for the wsgi entry point because I don't think it can be anywhere else, so I'm a bit stumped. | |
| 10:10:48 | stephenfin | mdbooth: Tried using the mock setting I was on about? | |
| 10:10:59 | mdbooth | stephenfin: In sphinx? | |
| 10:11:05 | stephenfin | aye | |
| 10:11:21 | mdbooth | Is that a valid thing to do? | |
| 10:11:32 | mdbooth | I didn't understand what it did, tbh. | |
| 10:11:39 | stephenfin | http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_mock_imports | |
| 10:12:24 | mdbooth | Oh, so autodoc_mock_imports = ["nova.monkey_patch"] ? | |
| 10:12:28 | stephenfin | tbh, I'm not sure how it works under the hood either. You could try mocking the 'eventlet' module though and see what happens | |
| 10:12:32 | stephenfin | Or that, yeah | |
| 10:12:52 | mdbooth | Where does the sphinx config live again? | |
| 10:13:00 | stephenfin | If that doesn't help, we're going to have to rip out the stuff that imports code and figure out which one is causing the issue | |
| 10:13:22 | stephenfin | Those modules being 'oslo_config.sphinxext', 'oslo_policy.sphinxext' and 'sphinx.ext.apidoc' | |
| 10:13:25 | sean-k-mooney | mdbooth: i think in docs/sources/conf.py | |
| 10:13:28 | stephenfin | mdbooth: doc/source/conf.py | |
| 10:13:39 | sean-k-mooney | i was so close :) | |
| 10:14:36 | openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Update Network Bandwidth resource provider spec https://review.openstack.org/644810 | |
| 10:19:12 | mdbooth | stephenfin: autodoc_mock_imports = ["nova.monkey_patch"] does not appear to mock nova.monkey_patch | |
| 10:20:25 | sean-k-mooney | mdbooth: whatabout autodoc_mock_imports = ["eventlet"] ? | |
| 10:20:57 | mdbooth | Just trying, but I am disappoint | |
| 10:21:52 | stephenfin | mdbooth: Hmm, out of curiosity, doesn't this mean that simply importing 'nova' will trigger the monkey patching so, regardless of whether it's Sphinx importing it or something else? | |
| 10:22:02 | mdbooth | That also appears not to be mocked. | |
| 10:22:25 | mdbooth | stephenfin: Well *anything* importing one of these classes will apply monkey patching | |
| 10:22:31 | mdbooth | But this has always been true | |
| 10:22:51 | mdbooth | In this specific instance, I've moved the wsgi entry point, though | |
| 10:23:02 | stephenfin | and what's changed? | |
| 10:23:11 | sean-k-mooney | stephenfin: the classes that triggered the auto monkey patching in the past did not contain any reuable busines loginc | |
| 10:23:14 | mdbooth | It used to be in nova/api/openstack/wsgi_app.py, it's now in nova/api/openstack/__init__.py | |
| 10:23:25 | mdbooth | However, it has *always* been in nova/cmd/__init__.py | |
| 10:23:34 | sean-k-mooney | ya | |
| 10:24:05 | sean-k-mooney | nova/cmd/__init__.py was done so that when we use setup tools to generate the console scipts | |
| 10:24:16 | mdbooth | sean-k-mooney: The reusable aspect of my patch is a side effect | |
| 10:24:23 | sean-k-mooney | the import of the commands monkey patched it automatically | |
| 10:24:43 | mdbooth | Primarily it *fixes* the non-wsgi and wsgi imports to do monkey patching before importing anything else | |
| 10:24:54 | mdbooth | It's a bugfix, not a cleanup | |
| 10:24:58 | sean-k-mooney | mdbooth: yes | |
| 10:25:09 | sean-k-mooney | but the point i was try to make was | |
| 10:25:24 | sean-k-mooney | if you wrote a tool that just imported our config definiton | |
| 10:25:34 | sean-k-mooney | say to do a config migration or to generate the files | |
| 10:25:48 | sean-k-mooney | then before it would not monkey patch your tool | |
| 10:26:09 | sean-k-mooney | depending on where you put the call that may or may not change | |
| 10:26:48 | mdbooth | stephenfin: How would you feel if I set an environment variable in sphinx tox.ini which nova.monkey_patch used to skip monkey patching? | |
| 10:26:57 | mdbooth | Or conf.py | |
| 10:27:40 | stephenfin | mdbooth: I'm all for it | |
| 10:29:07 | stephenfin | With a giant NOTE explaining why we do this craziness | |
| 10:29:32 | mdbooth | stephenfin: Heh, of course | |
| 10:29:47 | mdbooth | stephenfin: nova.monkey_patch is itself now a giant NOTE | |
| 10:30:20 | sean-k-mooney | stephenfin: im hoping in train we can remove the need to monkey patch the wsgi api code. | |
| 10:30:28 | stephenfin | U, you mean? | |
| 10:30:36 | sean-k-mooney | no in trian | |
| 10:30:48 | stephenfin | Oh, sorry, just the WSGI piece | |
| 10:30:52 | sean-k-mooney | it was a regression in the multi cell list feither | |
| 10:30:58 | sean-k-mooney | *feature | |
| 10:30:58 | mdbooth | Indeed | |
| 10:31:05 | sean-k-mooney | ya just the wsgi piece | |
| 10:31:34 | mdbooth | Although, tbh it's flaky as hell and we need to remove the whole thing. | |
| 10:31:41 | sean-k-mooney | doing it in general is proably not worth it but eventlets and wsgi are not partcally happy bedfellows | |
| 10:32:16 | mdbooth | Our entry points are not sufficiently well defined to do this. Specifically, monkey patching on import is a bad idea, but our entry points leave us with no other choice. | |
| 10:32:19 | sean-k-mooney | mdbooth: ya that proably a U thing but futuriest actully has eveything i waned by asyncio form python3 | |
| 10:32:56 | sean-k-mooney | mdbooth: ya | |
| 10:33:53 | sean-k-mooney | fortunetly or not depending on your perspective we dont alter those entry points frequently so the sting and ductape we have been using in the past has been enough | |
| 10:59:32 | aspiers | is it just me or is tempest-full-py broken? "The error was: 'gabbi_tempest_path' is undefined\n\nThe error appears to have been in '/var/lib/zuul/builds/42292da2453549e68ae6c1e63bfca00b/untrusted/project_5/git.openstack.org/openstack/tempest/roles/setup-tempest-run-dir/tasks/main.yaml'" | |
| 11:00:10 | cdent | blargh, that sounds like it is at least partially my fault | |
| 11:00:27 | cdent | (because of gabbi in the name) but I haven't changed anything recently | |
| 11:00:32 | aspiers | cdent: failure is in https://review.openstack.org/#/c/644554/ | |
| 11:00:35 | cdent | thanks | |
| 11:03:06 | cdent | aspiers: looks like it has already been fixed | |
| 11:03:11 | aspiers | oh nice | |