| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-07-24 | |||
| 17:33:13 | sean-k-mooney | this is the funtion that actully creates teh file https://github.com/ceph/ceph/blob/nautilus/src/os/bluestore/BlueStore.cc#L5934 | |
| 17:33:21 | melwitt | v15.1.0 is Octopus | |
| 17:34:02 | sean-k-mooney | if the block file is not present it creates it https://github.com/ceph/ceph/blob/nautilus/src/os/bluestore/BlueStore.cc#L5931-L5934 | |
| 17:34:36 | melwitt | and somehow 'size' is passed in from the config option I assume | |
| 17:34:49 | sean-k-mooney | that is what im currently trying to find yes | |
| 17:36:36 | sean-k-mooney | this maybe https://github.com/ceph/ceph/blob/nautilus/src/os/bluestore/BlueStore.cc#L5943-L5944 | |
| 17:38:33 | sean-k-mooney | ah no its here | |
| 17:38:36 | sean-k-mooney | https://github.com/ceph/ceph/blob/nautilus/src/os/bluestore/BlueStore.cc#L6050-L6052 | |
| 17:38:41 | sean-k-mooney | so in the mkfs call | |
| 17:39:25 | sean-k-mooney | so when we do this | |
| 17:39:27 | sean-k-mooney | https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/lib/ceph#L482-L483 | |
| 17:39:29 | melwitt | ah yup, and it's pulling the conf option | |
| 17:39:43 | sean-k-mooney | it cause the mkfs function to be invoked on the backend store | |
| 17:39:59 | sean-k-mooney | whic for bluestore uses that config option to create the 10G file | |
| 17:40:14 | melwitt | the interesting thing is, I wonder why it uses the legacy option and not the new one. I don't understand how that works in their code. cause in nautilus they have both the 10G and 100G default in the legacy conf vs the non | |
| 17:40:24 | sean-k-mooney | if var/lib/ceph/osd/ceph-0/block is not a symplink to a device | |
| 17:41:07 | sean-k-mooney | melwitt: its legacy on master | |
| 17:41:15 | sean-k-mooney | it might not be on nautalius | |
| 17:41:37 | melwitt | oh, I thought you mentioned earlier that CI is using nautilus | |
| 17:41:57 | sean-k-mooney | actully its alos here on master https://github.com/ceph/ceph/blob/master/src/common/options.cc#L4127-L4131 | |
| 17:42:05 | sean-k-mooney | melwitt: yes it is | |
| 17:42:25 | sean-k-mooney | actully just above that | |
| 17:42:27 | sean-k-mooney | https://github.com/ceph/ceph/blob/master/src/common/options.cc#L4122-L4125 | |
| 17:42:49 | melwitt | yeah I'm saying it's weird that it's not defaulting to 100G like that is showing | |
| 17:42:57 | melwitt | the old default was 10G | |
| 17:43:02 | sean-k-mooney | yep | |
| 17:43:57 | sean-k-mooney | we are pulling 14.2.2 https://github.com/ceph/ceph/blob/v14.2.2/src/common/options.cc#L4339 | |
| 17:44:02 | sean-k-mooney | which is 10 | |
| 17:44:32 | sean-k-mooney | they backported the 100G change to nautilus | |
| 17:44:44 | sean-k-mooney | but its not in the tag we are pulling | |
| 17:45:23 | sean-k-mooney | i think legacy_config_opts.h is just an old way to define config options | |
| 17:45:27 | melwitt | ohhh | |
| 17:45:37 | melwitt | good find. ok at least everything makes sense now | |
| 17:45:39 | sean-k-mooney | rather then deprected by the way | |
| 17:45:58 | sean-k-mooney | ya so i guess we just set that config option to say 20G? | |
| 17:46:02 | sean-k-mooney | in ceph.conf | |
| 17:46:10 | melwitt | yeah, seems like it | |
| 17:46:34 | sean-k-mooney | which we can do here https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/lib/ceph#L415-L428 | |
| 17:47:23 | melwitt | yarp. just have to double check whether it's a "global" or what. are those config groups or? | |
| 17:47:32 | sean-k-mooney | i like how this is basically undocumeted other then in the source code | |
| 17:47:54 | sean-k-mooney | i think in global yes | |
| 17:48:05 | melwitt | yeah, I know. they have a bluestore config doc but zero mention of this https://docs.ceph.com/docs/mimic/rados/configuration/bluestore-config-ref | |
| 17:48:49 | sean-k-mooney | iniset -sudo ${CEPH_CONF_FILE} global "bluestore_block_size" "20" | |
| 17:48:58 | sean-k-mooney | is that right? | |
| 17:49:53 | sean-k-mooney | i was search for 10_G but i think _G is a user defied suffix | |
| 17:49:57 | sean-k-mooney | so now i need to find that | |
| 17:50:32 | sean-k-mooney | yep https://github.com/ceph/ceph/blob/8c1a077e560248760ac441f315b84304aa693e72/src/common/options.cc#L343-L345 | |
| 17:51:36 | melwitt | oh, is the unit GB or something else? | |
| 17:52:20 | sean-k-mooney | its in bytes i think | |
| 17:52:32 | sean-k-mooney | 10_G is doing 10 << 32 | |
| 17:52:55 | sean-k-mooney | its a c++ 11 user defied literal https://en.cppreference.com/w/cpp/language/user_literal | |
| 17:53:26 | sean-k-mooney | actuly its << 30 not 32 | |
| 17:53:33 | sean-k-mooney | but ya still bytes | |
| 17:54:35 | sean-k-mooney | unsigned long long .... im glad they also defined a bettere way to name integers in c++11 so you dont have toe use that c way of naming types | |
| 17:54:41 | melwitt | ok so you can't just put "20" in the conf | |
| 17:54:53 | sean-k-mooney | i think we have to do 20<<30 | |
| 17:55:14 | sean-k-mooney | so 21474836480 | |
| 17:55:41 | melwitt | right | |
| 17:59:44 | sean-k-mooney | ill pretend tehy are not potting a unsigned long long into a size_t variant without asserting it fits | |
| 18:00:03 | melwitt | :) | |
| 18:16:21 | dansmith | have ya'll fixed it yet? | |
| 18:18:17 | sean-k-mooney | im looking at a linux bridge issue from the neutron channel currently but it looks liek we jsut need one more line here to set the config option https://github.com/openstack/devstack-plugin-ceph/blob/master/devstack/lib/ceph#L429 | |
| 18:18:39 | sean-k-mooney | dansmith: can you test it with your local setup | |
| 18:19:08 | sean-k-mooney | just add iniset -sudo ${CEPH_CONF_FILE} global "bluestore_block_size" "21474836480" | |
| 18:19:16 | dansmith | yup | |
| 18:19:26 | dansmith | oh wait, I can't | |
| 18:19:29 | dansmith | because mine doesn't use blue | |
| 18:19:37 | dansmith | but I can float a patch and get jobs going | |
| 18:19:47 | sean-k-mooney | ya that works | |
| 18:20:21 | sean-k-mooney | i dont have a ceph env currently i could set one up but its almost half past 7 on a friday so dont want to wait for it to stack :) | |
| 18:24:36 | dansmith | sean-k-mooney: dude, you need to cut yourself off :) | |
| 18:25:20 | dansmith | https://review.opendev.org/#/c/742961/ | |
| 18:27:43 | dansmith | I think the nova team needs to have the keys to sean-k-mooney's irc bouncer so we can turn it off when it's time for him to sleep | |
| 18:28:10 | sean-k-mooney | hehe i dont use one i just dont trun my laptop off :P | |
| 18:28:11 | dansmith | like giving car keys to the bartender | |
| 18:28:50 | melwitt | sean-k-mooney laptop and dev box permanently ON | |
| 18:29:20 | dansmith | sean-k-mooney: well, then an ssh account to your laptop I guess | |
| 18:29:49 | sean-k-mooney | melwitt: yes they more or less are. | |
| 18:30:00 | dansmith | melwitt: more like sean-k-mooney permanently ON | |
| 18:30:02 | melwitt | man, what was I doing earlier | |
| 18:30:10 | melwitt | true | |
| 18:30:20 | dansmith | laptop sleep timer be like "jesus when is he going to go to bed, I'm exhausted" | |
| 18:30:29 | melwitt | haha yeah | |
| 18:31:23 | artom | dansmith, I think we'll need remote access to his fuse box... | |
| 18:31:38 | artom | First we'll need to invent an SSHable fuse box... | |
| 18:31:44 | dansmith | artom: he's a property owner now, so we can't go to the landlord | |
| 18:32:27 | dansmith | artom: no need to cut the power just reset his luks key | |
| 18:33:04 | artom | But then they'd have a crappy web UI with 'password' hardcoded as the admin password | |
| 18:33:07 | artom | So maybe not | |
| 18:33:17 | melwitt | yeah, I would not be surprised | |
| 18:34:32 | sean-k-mooney | dansmith: oh your mean resting the luks key on my laptop would be a pain to fix | |
| 18:34:47 | dansmith | sean-k-mooney: no, we can reset it back to the one you know when you should be online | |
| 18:34:54 | sean-k-mooney | ah ok | |
| 18:35:00 | dansmith | hah | |
| 18:35:02 | sean-k-mooney | speaking of which o/ | |
| 18:35:07 | dansmith | good :) | |
| 18:35:27 | melwitt | wait, weren't you on pto today too? what the heck | |
| 18:35:56 | sean-k-mooney | yesterday | |
| 18:36:04 | sean-k-mooney | well untill today | |
| 18:36:08 | dansmith | maybe /melwitt/ needs the sleep | |