| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-09-28 | |||
| 18:27:28 | mriedem | well i think that's for live migration with ceph shared storage, so probably not the actual thing i'm trying to fix, but still | |
| 18:28:29 | mtreinish | mriedem: do you want that block_migration flag to be true or false? | |
| 18:28:53 | mriedem | it needs to be true | |
| 18:28:58 | mriedem | but hold up | |
| 18:29:32 | mriedem | i'm confused as to where the post-test-hook is called | |
| 18:29:41 | mtreinish | mriedem: it's called in devstack gate | |
| 18:30:16 | mtreinish | the second tempest run is done by devstack gate instead of grenade | |
| 18:30:37 | mtreinish | and looking at that post test hook code you're telling devstack gate to not run tempest, so the hook can modify the tempest config and run it itself | |
| 18:31:24 | mriedem | the reason the hook is setting block_migration=False is because the tests that come after that are for shared storage (nfs and ceph) | |
| 18:31:40 | mriedem | so i'm confused as to why that's configuring tempest before the tests are run | |
| 18:32:03 | mriedem | because i can see from the failed tempest log, that tempest is passing block_migration=False b/c that's what's in the config | |
| 18:32:59 | mriedem | starts running here http://logs.openstack.org/87/463987/20/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/ae8875f/console.html#_2017-09-26_14_25_30_596259 | |
| 18:33:32 | mriedem | at that point, things pass | |
| 18:34:05 | mriedem | http://logs.openstack.org/87/463987/20/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/ae8875f/console.html#_2017-09-26_14_26_36_296539 | |
| 18:34:37 | mriedem | oh ffs | |
| 18:34:41 | mriedem | it is the branch thing | |
| 18:34:54 | mriedem | it passes the first run with live block migration, and then fails the ceph one | |
| 18:35:00 | mriedem | presumably because we just suck with ceph still | |
| 18:36:52 | sean-k-mooney | mriedem: right so it failing later here http://logs.openstack.org/87/463987/20/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/ae8875f/console.html#_2017-09-26_14_28_50_096937 | |
| 18:39:04 | mriedem | well, so bug 1691769 isn't a problem anymore | |
| 18:39:05 | openstack | bug 1691769 in OpenStack Compute (nova) "gate-grenade-dsvm-neutron-multinode-live-migration-nv fails in pike: "Failed to restart |
|
| 18:42:08 | mtreinish | email address hidden? | |
| 18:42:30 | mtreinish | hah systemd units | |
| 18:43:30 | mriedem | aha | |
| 18:43:33 | mriedem | another discovery | |
| 18:43:44 | mriedem | which dansmith might remember | |
| 18:43:51 | mriedem | grenade runs in singleconductor more | |
| 18:43:52 | mriedem | *mode | |
| 18:43:54 | mriedem | but, | |
| 18:44:09 | mriedem | this live migration test hook is configuring /etc/nova/nova-cpu.conf | |
| 18:44:10 | mriedem | http://logs.openstack.org/87/463987/20/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/ae8875f/logs/subnode-2/etc/nova/nova-cpu.conf.txt.gz | |
| 18:44:45 | mriedem | and it's not running with that one | |
| 18:44:46 | mriedem | ['--config-file', '/etc/nova/nova.conf'] | |
| 18:46:35 | mriedem | Ibc4b21089ef86ab2430874c39d63174528c9a83e | |
| 18:47:18 | sean-k-mooney | right its doing that here http://logs.openstack.org/87/463987/20/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/ae8875f/console.html#_2017-09-26_14_27_39_368886 and running and here its using /etc/nova/nova.conf http://logs.openstack.org/87/463987/20/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/ae8875f/console.html#_2017-09-26_14_27_56_301409 | |
| 18:48:07 | mriedem | yup, the issue is that https://review.openstack.org/#/c/458537/ isn't aware of grenade being singleconductor | |
| 18:48:27 | mriedem | because, jesus, live migration + grenade + superconductor...crazy | |
| 18:52:05 | sean-k-mooney | so the fix is the run nova-compute with /etc/nova/nova-cpu.conf ? | |
| 18:52:38 | mriedem | no | |
| 18:55:29 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix live migration grenade ceph setup https://review.openstack.org/508271 | |
| 18:55:30 | mriedem | i think this is the fix ^ | |
| 18:58:04 | sean-k-mooney | hum that should work however it may be better to always use a seperate config for nova compute so that it does not chage regardless of the config. | |
| 18:58:09 | mriedem | i don't think zuul is running any jobs right now though so, i guess we'll find out later | |
| 18:58:33 | mriedem | sean-k-mooney: oh i'm sure this is air tight and will never require changes again :) | |
| 18:58:52 | sean-k-mooney | :) | |
| 18:59:25 | sean-k-mooney | by the way https://github.com/openstack/nova/search?utf8=%E2%9C%93&q=TODO%28+Queens&type= that was not the only todo with remove in queens ... | |
| 19:00:37 | sean-k-mooney | stephenfin: i belive this one is all yours https://github.com/openstack/nova/blob/762c89f85a4aeceb7bed5e80edc70a239872b757/nova/cmd/manage.py#L141 | |
| 19:01:04 | tssurya | mriedem, dansmith, melwitt : random off-topic : saw the nova cellv2 meeting logs from yesterday, so sorry I could not attend (was afk due to bad time zone timings) it and for wasting some precious seconds during the meeting; but thanks for the ping. Also from next time I will keep you posted earlier on if I am not attending. | |
| 19:01:31 | sean-k-mooney | wait i just confused myself queens comes after pike... | |
| 19:01:44 | dansmith | tssurya: not a problem and no need to keep us notified of your schedule, I just made a point of pinging you because we had fooled you with our hangout the week prior | |
| 19:02:05 | melwitt | no worries tssurya. yep, what dansmith said | |
| 19:02:55 | tssurya | dansmith, melwitt : cool :) thanks! | |
| 19:05:03 | mriedem | tssurya: i got to thinking, | |
| 19:05:09 | mriedem | cern is upgrading to pike right now right? | |
| 19:05:14 | mriedem | and cells v2 was required in ocata, | |
| 19:05:34 | mriedem | so it would be nice if you could give us some info on how cern upgraded to ocata and rolled out cellsv2 even though they are using cells v1 | |
| 19:05:47 | mriedem | like, was cern in ocata just a single giant cells v2 cell? | |
| 19:06:03 | tssurya | mriedem : no cern is still on newton, it is going to upgrade to ocata | |
| 19:06:04 | mriedem | or, is cern actually migrating to cells v2 | |
| 19:06:16 | mriedem | i saw something the other day saying they were upgrading glance to pike | |
| 19:06:19 | mriedem | maybe that's just glance> | |
| 19:06:20 | mriedem | ? | |
| 19:06:27 | tssurya | mriedem : yes its just glance | |
| 19:06:31 | mriedem | gah! | |
| 19:06:37 | tssurya | mriedem : nova is still on newton | |
| 19:06:59 | mriedem | is that because of the complications with migrating nova to ocata? | |
| 19:07:01 | sean-k-mooney | tssurya: so cern is migrating to cellsv2 now also | |
| 19:07:04 | tssurya | mriedem : however we will soon upgrade to cells v2 :) | |
| 19:07:24 | mriedem | tssurya: ok, i hope there will be copious amounts of blog posts and such on what the plan was and how it goes | |
| 19:07:56 | tssurya | mriedem , sean-k-mooney : yes belmiro is about to come up with a plan for doing this soon | |
| 19:08:20 | sean-k-mooney | mriedem: we are waiting for cern to fully migrate to cellsv2 before removing cellsv1 and nova networks correct | |
| 19:08:33 | dansmith | not just cern | |
| 19:08:45 | dansmith | until pike it wasn't even a possibility for multiple cells, | |
| 19:09:24 | mriedem | sean-k-mooney: at the ptg we said if we have dan's efficient instance list stuff in queens, we remove cells v1 and nova-net in rocky | |
| 19:09:28 | dansmith | and many will have issues with multiple cells until queens | |
| 19:09:30 | sean-k-mooney | well yes but as the bigest user of cells they are a good litmus test that such a migration works at scale | |
| 19:09:31 | mriedem | and we're on a good path to doing that | |
| 19:10:21 | sean-k-mooney | mriedem: ah ok. part of my interest is there is some legacy nova network datamodels in os-vif | |
| 19:10:26 | sean-k-mooney | https://bugs.launchpad.net/os-vif/+bug/1720175 | |
| 19:10:27 | dansmith | sean-k-mooney: get in line :) | |
| 19:10:27 | openstack | Launchpad bug 1720175 in os-vif "Move "ips" field from Subnet object to VIF object" [High,Triaged] | |
| 19:10:56 | sean-k-mooney | currently nothing uses it but its from when we imported the network models for nova | |
| 19:13:39 | sean-k-mooney | dansmith: gladly, i just have to keep remining people that os-vif is used with nova net too so we cant break it eventhough we dont gate on it for the os-vif repo :) | |
| 19:14:13 | dansmith | sean-k-mooney: I'm just saying there's a lot of stuff we get to remove when we drop n-net, and you're late to that party :P | |
| 19:15:30 | efried | mriedem I have reason to believe that it is impossible to use anything other than the public interface for cinderclient in Nova. | |
| 19:15:56 | sean-k-mooney | haha yes though not that late i have wanted to kill nova-net since neutron was still quantum but in fairness to nova net it proably was a better choice for large deployment back then. | |
| 19:16:15 | efried | (mriedem or point me to someone else who would have the inclination to walk through this with me) | |
| 19:17:19 | mriedem | efried: ok? | |
| 19:17:52 | efried | mriedem D'oh, nope, buried in the bowels, the 'interface' kwarg overrides 'endpoint_type'. | |
| 19:18:10 | efried | It's just way non-obvious from the first four layers of calls. | |
| 19:18:15 | efried | Carry on. | |
| 19:18:16 | mriedem | sean-k-mooney: dansmith: don't forget that if we wait long enough, edge will require nova-net again | |
| 19:18:37 | sean-k-mooney | edge? | |
| 19:18:55 | sean-k-mooney | as in cloud edge computing | |
| 19:19:10 | mriedem | yes | |
| 19:20:48 | sean-k-mooney | ah well it might actullly work well in that model in multihost mode untill a telco trys to add sfc to nova net | |
| 19:36:39 | mriedem | stvnoyes: ok comments inline https://review.openstack.org/#/c/463987/ | |
| 19:36:45 | mriedem | once those test things are cleaned up i'm +2 | |
| 20:00:01 | stvnoyes | cool. thx | |
| 20:05:13 | catintheroof | hi guys, quick question, when CoreFilter is enabled on the scheduler nodes, is cpu_allocation_ratio allowed per compute node ? | |