| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-26 | |||
| 20:53:13 | mriedem | dansmith: yup thar she blar http://paste.openstack.org/show/726714/ | |
| 20:53:44 | dansmith | nice | |
| 21:01:49 | melwitt | mriedem: novaclient release proposed https://review.openstack.org/586350 | |
| 21:08:05 | mriedem | my awk skills are not going to be good enough to parse out that df output | |
| 21:08:11 | mriedem | i can parse out: ceph status | grep usage | |
| 21:08:45 | dansmith | is that a challenge? | |
| 21:08:49 | mriedem | it is | |
| 21:09:25 | melwitt | mriedem: you can pass --format=json to make it return json if that helps | |
| 21:09:42 | mriedem | ooo | |
| 21:09:42 | mriedem | $ sudo ceph df | awk 'NR==3{print $1}END{$1}' | |
| 21:09:43 | mriedem | 24564M | |
| 21:10:20 | mriedem | melwitt: doesn't really help me in bash | |
| 21:10:43 | mriedem | although, | |
| 21:10:50 | mriedem | pipe that into python -m json.tool | |
| 21:10:56 | mriedem | "stats": { | |
| 21:10:56 | mriedem | "total_avail_bytes": 25482182656, | |
| 21:10:56 | mriedem | "total_bytes": 25757220864, | |
| 21:10:56 | mriedem | "total_used_bytes": 275038208 | |
| 21:10:56 | mriedem | } | |
| 21:11:16 | dansmith | echo $((`cat ceph | grep vms | awk '{print $5}' | egrep -o '[0-9]+'` / 1024)) | |
| 21:11:16 | dansmith | 22 | |
| 21:11:45 | dansmith | oh json format is nice | |
| 21:11:51 | dansmith | is jq on the worker nodes? I bet it is | |
| 21:12:03 | mriedem | except the comma | |
| 21:12:03 | mriedem | $ sudo ceph df --format=json | python -m json.tool | grep total_avail_bytes | awk 'END{print $2}' | |
| 21:12:03 | mriedem | 25482182656, | |
| 21:12:22 | dansmith | hold up | |
| 21:13:03 | mriedem | haha | |
| 21:13:03 | mriedem | $ sudo ceph df --format=json | python -m json.tool | grep total_avail_bytes | awk 'END{print $2}' | egrep -o '[0-9]+' | |
| 21:13:03 | mriedem | 25481973760 | |
| 21:13:35 | dansmith | ceph df --format=json | | |
| 21:13:43 | dansmith | ceph df --format=json | jq .total_bytes | |
| 21:13:49 | dansmith | er | |
| 21:13:57 | dansmith | ceph df --format=json | jq stats.total_bytes | |
| 21:13:59 | dansmith | I think | |
| 21:14:14 | mriedem | don't have jq | |
| 21:14:28 | dansmith | ceph df --format=json | jq .stats.total_bytes | |
| 21:14:30 | dansmith | ^ works | |
| 21:14:37 | dansmith | on the worker nodes? | |
| 21:14:40 | dansmith | that sucks | |
| 21:14:43 | mriedem | $ sudo ceph df --format=json | jq .stats.total_bytes | |
| 21:14:43 | mriedem | 25757220864 | |
| 21:14:47 | mriedem | i mean, we can install it... | |
| 21:14:58 | mriedem | but this also works | |
| 21:14:58 | mriedem | $ sudo ceph df --format=json | python -m json.tool | grep total_avail_bytes | egrep -o '[0-9]+' | |
| 21:14:59 | mriedem | 25481973760 | |
| 21:15:12 | dansmith | yup | |
| 21:15:22 | dansmith | convert to GB of course | |
| 21:16:41 | mriedem | yup; even shorter with json-pretty | |
| 21:16:41 | mriedem | $ sudo ceph df --format=json-pretty | grep total_avail_bytes | egrep -o '[0-9]+' | |
| 21:16:42 | mriedem | 25481887744 | |
| 21:27:01 | mriedem | oh do we have to pull from the vms pool specifically? | |
| 21:27:02 | mriedem | i suppose huh | |
| 21:28:29 | dansmith | I assume it's all the same space since it's the same size | |
| 21:28:35 | melwitt | rc todo etherpad https://etherpad.openstack.org/p/nova-rocky-release-candidate-todo | |
| 21:28:37 | dansmith | and you're only going to update it once (and it's just inventory) | |
| 21:28:42 | dansmith | so, probably meh | |
| 21:31:19 | mriedem | the max available is the same for the 3 pools | |
| 21:32:06 | dansmith | yeah, I dunno what that's showing really | |
| 21:32:06 | mriedem | oh i see | |
| 21:32:06 | mriedem | /dev/loop0 24G 228M 24G 1% /var/lib/ceph | |
| 21:32:18 | dansmith | could be that if you allocate some from the images you'll lose max on the vms | |
| 21:32:22 | dansmith | like ZFS in a pool | |
| 21:32:27 | mriedem | yeah so it's the same 24GB block shared across 3 pools? | |
| 21:32:31 | dansmith | yeah I'm sure | |
| 21:33:02 | dansmith | hard to say what the right thing to do is, but for testing just saying it's 24G is probably fine | |
| 21:33:35 | mriedem | was trying to think what we should put for reserved | |
| 21:33:37 | mriedem | or just leave that 0 | |
| 21:33:43 | dansmith | it won't be right regardless, | |
| 21:33:58 | dansmith | other than maybe reserving enough for whatever images and snaps we do in a regular tempest run | |
| 21:34:02 | dansmith | but still, it's just a gamble | |
| 21:35:19 | mriedem | yeah so i can really just use CEPH_LOOPBACK_DISK_SIZE in devstack, and don't need to parse ceph df output then.. | |
| 21:36:33 | dansmith | hah | |
| 21:38:29 | mriedem | but this has been fun | |
| 22:03:49 | mriedem | easy bug https://bugs.launchpad.net/placement-osc-plugin/+bug/1783896 | |
| 22:03:49 | openstack | Launchpad bug 1783896 in placement-osc-plugin "openstack resource provider aggregate set <name> should be <uuid>" [Medium,Triaged] | |
| 22:05:15 | melwitt | mriedem: this patch for the metadata API bp, it looks good to me but I wasn't 100% sure the vmware CI fail on it is legit https://review.openstack.org/580742 forgot to mention it this morning | |
| 22:05:46 | melwitt | the other third party CI fails on it look unrelated | |
| 22:06:00 | mriedem | can't really dig into that atm | |
| 22:06:05 | mriedem | already shouldn't be working anymore | |
| 22:06:14 | melwitt | ok | |
| 22:11:04 | melwitt | mriedem: so we're Depends-On https://review.openstack.org/566813 and https://review.openstack.org/564452 for r-3, is there anything else that you know of? | |
| 22:11:35 | mriedem | dansmith: melwitt: https://review.openstack.org/586363 | |
| 22:11:55 | melwitt | oh yeah, guh. | |
| 22:12:03 | mriedem | those are the 2 i'm watching | |
| 22:12:19 | mriedem | no, https://review.openstack.org/#/c/586363/ is the ceph job using a shared provider in placement for disk | |
| 22:12:22 | mriedem | efried: ^ | |
| 22:13:01 | openstackgerrit | OpenStack Release Bot proposed openstack/python-novaclient stable/rocky: Update .gitreview for stable/rocky https://review.openstack.org/586364 | |
| 22:13:03 | openstackgerrit | OpenStack Release Bot proposed openstack/python-novaclient stable/rocky: Update UPPER_CONSTRAINTS_FILE for stable/rocky https://review.openstack.org/586365 | |
| 22:13:05 | openstackgerrit | OpenStack Release Bot proposed openstack/python-novaclient master: Update reno for stable/rocky https://review.openstack.org/586366 | |
| 22:13:39 | melwitt | mriedem: oh, coincidence, ok. but you reminded me, we need your test to determine whether or not to reno a warning not to use the shared aggregate thing. does that have to be done before my EOD or just before RC1? | |
| 22:14:16 | mriedem | we don't have to warn about that in a reno | |
| 22:14:20 | mriedem | it's totally undocumented | |
| 22:14:25 | mriedem | we don't need to say "yup this still doesn't work" | |
| 22:14:51 | melwitt | ok, thanks | |
| 22:25:37 | dansmith | we have it in specs, | |
| 22:25:43 | dansmith | and before this, setting it would do nothing | |
| 22:25:50 | dansmith | people do read and try stuff that are in specs | |
| 22:26:05 | dansmith | so I think we should just patch it out of the driver if we're not going to mention it personally (assuming it's broken like we expect) | |
| 22:26:07 | dansmith | just MHO | |