Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-07-26
21:14:59 mriedem 25481973760
21:15:12 dansmith yup
21:15:22 dansmith convert to GB of course
21:16:41 mriedem yup; even shorter with json-pretty
21:16:41 mriedem $ sudo ceph df --format=json-pretty | grep total_avail_bytes | egrep -o '[0-9]+'
21:16:42 mriedem 25481887744
21:27:01 mriedem oh do we have to pull from the vms pool specifically?
21:27:02 mriedem i suppose huh
21:28:29 dansmith I assume it's all the same space since it's the same size
21:28:35 melwitt rc todo etherpad https://etherpad.openstack.org/p/nova-rocky-release-candidate-todo
21:28:37 dansmith and you're only going to update it once (and it's just inventory)
21:28:42 dansmith so, probably meh
21:31:19 mriedem the max available is the same for the 3 pools
21:32:06 dansmith yeah, I dunno what that's showing really
21:32:06 mriedem oh i see
21:32:06 mriedem /dev/loop0 24G 228M 24G 1% /var/lib/ceph
21:32:18 dansmith could be that if you allocate some from the images you'll lose max on the vms
21:32:22 dansmith like ZFS in a pool
21:32:27 mriedem yeah so it's the same 24GB block shared across 3 pools?
21:32:31 dansmith yeah I'm sure
21:33:02 dansmith hard to say what the right thing to do is, but for testing just saying it's 24G is probably fine
21:33:35 mriedem was trying to think what we should put for reserved
21:33:37 mriedem or just leave that 0
21:33:43 dansmith it won't be right regardless,
21:33:58 dansmith other than maybe reserving enough for whatever images and snaps we do in a regular tempest run
21:34:02 dansmith but still, it's just a gamble
21:35:19 mriedem yeah so i can really just use CEPH_LOOPBACK_DISK_SIZE in devstack, and don't need to parse ceph df output then..
21:36:33 dansmith hah
21:38:29 mriedem but this has been fun
22:03:49 mriedem easy bug https://bugs.launchpad.net/placement-osc-plugin/+bug/1783896
22:03:49 openstack Launchpad bug 1783896 in placement-osc-plugin "openstack resource provider aggregate set <name> should be <uuid>" [Medium,Triaged]
22:05:15 melwitt mriedem: this patch for the metadata API bp, it looks good to me but I wasn't 100% sure the vmware CI fail on it is legit https://review.openstack.org/580742 forgot to mention it this morning
22:05:46 melwitt the other third party CI fails on it look unrelated
22:06:00 mriedem can't really dig into that atm
22:06:05 mriedem already shouldn't be working anymore
22:06:14 melwitt ok
22:11:04 melwitt mriedem: so we're Depends-On https://review.openstack.org/566813 and https://review.openstack.org/564452 for r-3, is there anything else that you know of?
22:11:35 mriedem dansmith: melwitt: https://review.openstack.org/586363
22:11:55 melwitt oh yeah, guh.
22:12:03 mriedem those are the 2 i'm watching
22:12:19 mriedem no, https://review.openstack.org/#/c/586363/ is the ceph job using a shared provider in placement for disk
22:12:22 mriedem efried: ^
22:13:01 openstackgerrit OpenStack Release Bot proposed openstack/python-novaclient stable/rocky: Update .gitreview for stable/rocky https://review.openstack.org/586364
22:13:03 openstackgerrit OpenStack Release Bot proposed openstack/python-novaclient stable/rocky: Update UPPER_CONSTRAINTS_FILE for stable/rocky https://review.openstack.org/586365
22:13:05 openstackgerrit OpenStack Release Bot proposed openstack/python-novaclient master: Update reno for stable/rocky https://review.openstack.org/586366
22:13:39 melwitt mriedem: oh, coincidence, ok. but you reminded me, we need your test to determine whether or not to reno a warning not to use the shared aggregate thing. does that have to be done before my EOD or just before RC1?
22:14:16 mriedem we don't have to warn about that in a reno
22:14:20 mriedem it's totally undocumented
22:14:25 mriedem we don't need to say "yup this still doesn't work"
22:14:51 melwitt ok, thanks
22:25:37 dansmith we have it in specs,
22:25:43 dansmith and before this, setting it would do nothing
22:25:50 dansmith people do read and try stuff that are in specs
22:26:05 dansmith so I think we should just patch it out of the driver if we're not going to mention it personally (assuming it's broken like we expect)
22:26:07 dansmith just MHO
22:26:37 dansmith if we start claiming in stein that it works, people will try it in older releases too
22:30:45 melwitt mriedem_afk: fyi nova r-3 release patch https://review.openstack.org/586368
22:31:38 dansmith first hit: https://docs.openstack.org/nova/latest/reference/update-provider-tree.html
22:33:39 melwitt ok ... thinking. the change was done as a bug fix, https://bugs.launchpad.net/nova/+bug/1771728 which was found via someone setting MISC_SHARES_VIA_AGGREGATE
22:33:39 openstack Launchpad bug 1771728 in OpenStack Compute (nova) "libvirt: Shared Resource Provider (RP) DISK_GB is NOT taken into account if it's configured with Compute Node RPs" [Undecided,Fix released] - Assigned to Bhagyashri Shewale (bhagyashri-shewale)
22:35:02 melwitt if we patch the bug fix out, then it goes back to setting it does nothing. but that still seems weird because people think they can use it (because of this doc, possibly)
22:36:53 dansmith yeah, so someone thought they could use it and reported it did nothing,
22:36:59 melwitt I dunno, maybe just a doc update to this page saying MISC_SHARES_VIA_AGGREGATE isn't tested in tempest?
22:37:09 melwitt like a warning?
22:37:17 dansmith but if that's the case and it actually breaks resource reporting and probably migrations...
22:37:48 dansmith if it totally doesn't do anything other than break things we should probably not do it until the rest of the feature works
22:37:55 dansmith but at least a known issue reno, IMHO
22:38:18 dansmith not tested in tempest is different than "totally broken and we know it"
22:38:20 melwitt what I mean is, was the previous state of it doing nothing actually doing nothing? or could it have been breaking things underneath?
22:38:49 dansmith no, it was just being ignored when people expected it to magically make the compute node do the shared storage accounting right
22:39:12 dansmith now, setting it will cause the compute node to not expose disk inventory,
22:39:31 dansmith and likely in a migration situation we will clobber the allocation against the shared pool,
22:39:37 dansmith which will result in instances with no disk allocation,
22:40:02 melwitt okay ... so maybe, revert the bug fix, add a warning to the doc to mention MISC_SHARES_VIA_AGGREGATE isn't usable in nova yet, and then don't allow the bug fix back in until we have tempest setup with ceph etc?
22:40:05 dansmith which we'll have to fix via migration somehow, and potentially result in oversubscribing the pool since anything that has migrated once will have no disk allocation
22:40:28 dansmith that's certainly one approach yeah
22:40:44 dansmith the other being just a reno that warns that it's half implemented and known broken
22:41:20 melwitt I'm worried people will miss that reno, I guess
22:41:35 dansmith yup, and they'll corrupt their data if they do and try it
22:41:39 melwitt and the price for missing it is high
22:42:16 melwitt efried: are you around? what do you think about ^
22:48:01 mriedem rather than revert, you could just comment out the one line in the libvirt driver
22:48:12 dansmith yeah, revert is too old I'm sure
22:48:20 dansmith just have to mangle it manually
22:50:19 melwitt yeah, okay. I think we should because from what dansmith has explained, using this in its current state could badly corrupt a deployment and then we'd have to come up with some code to migrate people out of it if it happens to them
22:50:40 dansmith it'd be heal_allocations I guess
22:50:43 dansmith but still
22:54:49 dansmith might be able to functional test our way
22:55:30 melwitt I'll put this on the rc1 todo list https://etherpad.openstack.org/p/nova-rocky-release-candidate-todo
22:55:41 melwitt and get efried's thoughts tomorrow morning
23:22:34 rm_work hey nova folks! the deprecation discussed here: https://review.openstack.org/#/c/522027/ / https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/deprecate-file-injection.html -- is that what we're using here? https://github.com/openstack/octavia/blob/master/octavia/compute/drivers/nova_driver.py#L150
23:26:04 melwitt rm_work: yes, that is file injection, deprecated with microversion 2.57 https://github.com/openstack/python-novaclient/blob/master/novaclient/v2/servers.py#L1210-L1215
23:26:29 rm_work johnsom: ^^
23:27:08 johnsom melwitt What? You removed the ability to load data via config drive? That will break a ton of services.
23:27:41 melwitt johnsom: no, the ability to specify files to inject into the server. the supported way to do that is user data
23:28:33 johnsom melwitt but user_data was supposed to go away given it's non-standard and significantly limited in size.
23:29:21 melwitt I think you might be reversing the two? user_data isn't going away and is supported by cloud-init
23:29:24 johnsom Wow, this went under the radar. I know it impacts Octavia and Trove, probably manila and a few others
23:30:11 johnsom The file method we were using is supported by cloud-init via config drive.
23:31:31 johnsom What we are talking about is in the "content" folder in config drive

Earlier   Later