| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-22 | |||
| 19:38:48 | melwitt | yeah I agree stay consistent with other python-novaclient things | |
| 19:39:21 | efried | mriedem: it looks like logging is enabled by default? | |
| 19:41:29 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Try to repro bug 1823251 with mysql logs https://review.opendev.org/678051 | |
| 19:41:30 | openstack | bug 1823251 in OpenStack Compute (nova) "Spike in TestNovaMigrationsMySQL.test_walk_versions/test_innodb_tables failures since April 1 2019 on limestone-regionone" [High,Confirmed] https://launchpad.net/bugs/1823251 | |
| 19:41:54 | mnaser | sean-k-mooney: do you happen to know if you're able to backport the os-vif bug to stable/stein ? | |
| 19:42:14 | efried | mriedem: if it's writing to "the console"... is that the same console that tests are being run in? | |
| 19:44:03 | mriedem | efried: no, | |
| 19:44:11 | mriedem | likely the journal? | |
| 19:44:19 | mriedem | i.e. mysql systemd journal log | |
| 19:44:34 | mriedem | if i had a devstack handy i'd poke at this but i don't | |
| 19:45:00 | mriedem | efried: if this were an IMM or VIOS, wwyd?! | |
| 19:45:14 | efried | mriedem: How destructive is the poking? I have one you can jump on | |
| 19:46:06 | openstackgerrit | Jonathan Rosser proposed openstack/os-vif stable/stein: only disable mac ageing for ovs hybrid plug https://review.opendev.org/678098 | |
| 19:47:33 | efried | I don't know what IMM is. If it were VIOS I would phone a friend. | |
| 19:47:33 | efried | The problem is that I have basically no understanding of the chain of events that's leading to mysqld running and/or being used in this environment. | |
| 19:48:38 | mnaser | sean-k-mooney: jrosser has backported the os-vif issue which fixes deployments letting vms see other vms traffic :) | |
| 19:49:53 | mriedem | integrated management module | |
| 19:49:58 | mriedem | like the CMM in a bladecenter | |
| 19:50:00 | mriedem | DOY | |
| 19:50:25 | mriedem | where is dansmith when i need him for ibm jokes | |
| 19:51:16 | mriedem | efried: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/test-setup is the role for hitting the test-setup script in the unit test jobs | |
| 19:51:23 | efried | by the time we run tools/test-setup.sh, mysqld is already running, right? | |
| 19:51:52 | mriedem | has to be | |
| 19:51:56 | mriedem | since we're issuing commands against it | |
| 19:52:05 | efried | so whatever we're doing we need to do well before that | |
| 19:52:29 | efried | unless it's possible to restart mysqld from that script, is that what you were suggesting above? | |
| 19:56:29 | melwitt | mnaser: isn't that a pretty serious security issue? looked in the bug and it didn't look treated as such | |
| 19:57:33 | openstackgerrit | Merged openstack/nova master: Remove deprecated [neutron]/url option https://review.opendev.org/677755 | |
| 20:01:04 | mriedem | efried: the mysql docs seem to suggest you can change logging config at runtime and flush the settings | |
| 20:01:09 | mriedem | but w/o an env to play with idk | |
| 20:01:35 | mriedem | efried: otherwise yeah i'd think you could restart from within the script just by doing: sudo systemctl restart mysqld | |
| 20:05:26 | efried | I'm looking at a default-y devstack and /var/log/mysql has files in it | |
| 20:07:49 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Fail compute service delete if resource provider delete fails https://review.opendev.org/678100 | |
| 20:08:52 | mriedem | efried: huh yeah i guess my devstack patch works | |
| 20:08:52 | mriedem | https://ccb62560c9b6fbee62ac-3bdb45e74ea7431452edb68e5fcaf6be.ssl.cf5.rackcdn.com/678053/1/check/neutron-tempest-linuxbridge/d0d8b63/controller/logs/mysql/ | |
| 20:09:00 | mriedem | https://review.opendev.org/#/c/678053/ | |
| 20:09:12 | mriedem | maybe the ubuntu packages changed where the log files go? | |
| 20:09:15 | mriedem | since...trusty? | |
| 20:11:27 | efried | mriedem: Presumably that ^ will just work for dsvm jobs anyway since by default tox jobs don't copy *any* logs? | |
| 20:13:33 | openstackgerrit | Merged openstack/nova master: Limit get_sdk_adapter to requested service type https://review.opendev.org/674678 | |
| 20:13:41 | openstackgerrit | Merged openstack/nova master: Add a hacking rule for useless assertions https://review.opendev.org/677610 | |
| 20:13:49 | openstackgerrit | Merged openstack/nova master: doc: remove confusing docs about aggregate allocation ratios https://review.opendev.org/677472 | |
| 20:28:49 | mnaser | melwitt: it is a very serious secuity issue | |
| 20:28:55 | mnaser | with linuxbridge, a vm can see traffic to other vms | |
| 20:29:49 | melwitt | ok. argh, so by default we get no OSSA handling with it because of the way os-vif project is tagged? feels like we need to fix that (I don't know how) | |
| 20:29:52 | melwitt | I commented on the lp bug | |
| 20:36:39 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Find instance in another cell during floating IP re-association https://review.opendev.org/656594 | |
| 20:36:45 | mriedem | efried: tox jobs copy logs, like the tox logs and testr results | |
| 20:42:33 | openstackgerrit | Michael Still proposed openstack/nova master: Remove zVM requirement from the top level requirements. https://review.opendev.org/677870 | |
| 21:06:47 | mriedem | dansmith: i looked at mdbooths thing here https://review.opendev.org/#/c/677736/ and it seems sane, just some nits and such, and maybe some unnecessary test code | |
| 21:06:54 | mriedem | guess we're just waiting on a customer feedback | |
| 21:07:53 | openstackgerrit | Merged openstack/nova master: Update docstring of 'revert_resize' function https://review.opendev.org/665976 | |
| 21:21:12 | jrosser | mnaser: melwitt i can already see that this is failing https://review.opendev.org/#/c/678098/ | |
| 21:21:28 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Introduce live_migration_claim() https://review.opendev.org/635669 | |
| 21:21:28 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects for NUMA live migration https://review.opendev.org/634827 | |
| 21:21:29 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: add support for augmenting migrate_data with info from claims https://review.opendev.org/634828 | |
| 21:21:29 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: add support for updating NUMA-related XML on the source https://review.opendev.org/635229 | |
| 21:21:30 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: RPC changes to prepare for NUMA live migration https://review.opendev.org/634605 | |
| 21:21:30 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: NUMA live migration support https://review.opendev.org/634606 | |
| 21:21:31 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration https://review.opendev.org/640021 | |
| 21:21:31 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Functional test for NUMA live migration https://review.opendev.org/672595 | |
| 21:23:09 | melwitt | jrosser: hm, yeah I see it too. will probably need sean-k-mooney help to fix | |
| 21:23:10 | jrosser | whilst the cherry-pick was trivial, i'm not really familiar with the code and imminent PTO means someone else really needs to take a look | |
| 21:23:42 | artom | dansmith, ^^ finally have something | |
| 21:23:53 | melwitt | jrosser: ack. sean-k-mooney or I or someone can get it into shape while you're away | |
| 21:24:08 | artom | I posted a note in the middle that only the patches below it have had feedback addressed | |
| 21:24:17 | artom | But I wanted to let CI do its thing | |
| 21:24:29 | jrosser | melwitt: excellent thankyou - we have some anxious folk on the OSA side with stein deploys affected by this | |
| 21:25:17 | melwitt | jrosser: understood. this is bad for everyone, will want to fix asap | |
| 21:27:03 | openstackgerrit | Merged openstack/nova master: Address nits from privsep series https://review.opendev.org/677210 | |
| 21:41:01 | melwitt | fun, I just learned that the reverts_task_state decorator tries to update the resource tracker and if that fails (e.g. libvirtd stopped), it blows up and prevents further decorators from running, namely wrap_instance_fault | |
| 21:41:44 | melwitt | so then we get no server fault in the instance | |
| 21:52:35 | mriedem | efried: sean-k-mooney: alex_xu: dansmith (in absentia): i've replied on the vpmem persistence ML thread, enjoy | |
| 21:52:45 | mriedem | i can't say i read all of sean-k-mooney's legalize reply | |
| 21:52:57 | mriedem | i got to like bullet 4-a-ii and my head exploded | |
| 21:53:05 | efried | agreed | |
| 21:53:59 | mriedem | with that i need to get the hell outside for awhile | |
| 21:58:26 | sean-k-mooney | thanks and ya sorry my reply were a bit dense on info | |
| 22:01:39 | sean-k-mooney | melwitt: i can take a look and fix up the backport | |
| 22:01:52 | openstackgerrit | Eric Fried proposed openstack/nova master: Introduces SDK to IronicDriver and uses for node.get https://review.opendev.org/642899 | |
| 22:01:52 | openstackgerrit | Eric Fried proposed openstack/nova master: Use SDK for node.list https://review.opendev.org/656027 | |
| 22:01:53 | openstackgerrit | Eric Fried proposed openstack/nova master: Use SDK for validating instance and node https://review.opendev.org/656028 | |
| 22:01:53 | melwitt | sean-k-mooney: thanks! | |
| 22:01:53 | openstackgerrit | Eric Fried proposed openstack/nova master: Use SDK for setting instance id https://review.opendev.org/659690 | |
| 22:01:54 | openstackgerrit | Eric Fried proposed openstack/nova master: Use SDK for add/remove instance info from node https://review.opendev.org/659691 | |
| 22:01:54 | openstackgerrit | Eric Fried proposed openstack/nova master: Use SDK for getting network metadata from node https://review.opendev.org/670213 | |
| 22:03:36 | sean-k-mooney | melwitt: im pretty sure that add is missing here https://github.com/openstack/os-vif/blob/stable/stein/os_vif/internal/command/ip/__init__.py | |
| 22:04:09 | sean-k-mooney | we remove a layer of indirection in train | |
| 22:04:26 | sean-k-mooney | so ther is 1 addtion plays we need to add the parmater in stien | |
| 22:04:26 | melwitt | you mean it does not accept a ageing kwarg? | |
| 22:04:36 | melwitt | oh | |
| 22:05:40 | sean-k-mooney | correct it does not currently the patch that was backported did not have to update that file because we removed it in master | |
| 22:19:31 | sean-k-mooney | melwitt: mnaser in attion to adding the aging parmater to 1 more location i will need to update the functional test or backport a second patch | |
| 22:19:45 | sean-k-mooney | but yes we shoudl be able to make this work | |
| 22:20:26 | dansmith | mriedem_away: I assume by "outside" you mean either (a) out in super mario world or (b) three feet from your door to plant something | |
| 22:21:11 | melwitt | super mario world? | |
| 22:23:36 | dansmith | melwitt: i.e. some video game as opposed to actually going outside | |
| 22:24:24 | melwitt | haha, ok. that's an old super nintendo game | |
| 22:24:40 | sean-k-mooney | it is a good game. unfortunetly i dont have my snes anymore. i do have a saga meadrive with sonic plug into by tv | |
| 22:25:00 | melwitt | I didn't know mriedem_away is a SNES person | |