| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-21 | |||
| 09:29:46 | kashyap | bauzas: Really, that's not required. The guest OSes that don't fsync() are really really really horribly old and we don't support | |
| 09:30:10 | bauzas | kashyap: I mean, we're making an opinionated choice, right? If so, let's just capture it in the relnote | |
| 09:30:22 | bauzas | and not just in the commit msg, which is bad for operators | |
| 09:30:42 | bauzas | and technically, I accepted it as a fix, but this isn't really a fix | |
| 09:30:46 | kashyap | bauzas: Huh. That's what the rel note says | |
| 09:30:58 | bauzas | it's a performance improvement | |
| 09:30:59 | kashyap | bauzas: I don't follow you. What is not really a fix? | |
| 09:31:06 | kashyap | Well, it's both. | |
| 09:31:23 | bauzas | let's take an example | |
| 09:31:28 | bauzas | I have a car which is running | |
| 09:31:29 | kashyap | (1) Fixing wrong assumption in Nova; (2) And (1) brings performance improvement | |
| 09:31:41 | bauzas | for sure, the engine is running | |
| 09:32:01 | bauzas | but I can make sure that I can improve the engine by changing some, say, injector | |
| 09:32:23 | bauzas | that will possibly make my engine having trouble when I stop, for example, but that's depending on my driving | |
| 09:32:36 | bauzas | so, I like to know that I can improve my engine by changing my driving | |
| 09:33:07 | bauzas | kashyap: again, the relnote doesn't mention a bit about guest OSes | |
| 09:33:28 | bauzas | it's just litterally saying "heh, we're changing a default value" | |
| 09:33:48 | bauzas | there is no other contextual information | |
| 09:34:14 | bauzas | if you say "ok, I'm going to amend the relnote in a fup", fine with me | |
| 09:34:22 | bauzas | or I could just do it | |
| 09:34:36 | bauzas | but I just want to make sure operators know about the reasons we decided | |
| 09:34:55 | kashyap | bauzas: Mentioning guest OSes really confuses at this point. | |
| 09:35:09 | kashyap | If an operator has longer attention span, they will carefully read the documentation | |
| 09:35:49 | sean-k-mooney | kashyap: if it was just for perfromance we would not be changing the default however right | |
| 09:36:08 | sean-k-mooney | that is just a side benifit | |
| 09:36:11 | kashyap | Huh, who said it was just for performance? | |
| 09:36:33 | bauzas | kashyap: don't expect operators to load contect on every relnote | |
| 09:36:36 | bauzas | context* | |
| 09:36:51 | bauzas | it's a massive amount of information they have to digest | |
| 09:37:07 | bauzas | and we suffered some pain in the past by overexpecting their knowledge | |
| 09:37:27 | kashyap | bauzas: I know very well about that. I take great care in not assuming people will have context. | |
| 09:37:56 | gibi | stephenfin: do you have any suggestion about the table issue in https://review.openstack.org/#/c/644810/1/specs/stein/approved/bandwidth-resource-provider.rst@624 | |
| 09:37:57 | bauzas | anyway, I don't want to spend more time on it, will spin a new revision | |
| 09:38:12 | bauzas | gibi: stephenfin is mostly eaten by meetings all week | |
| 09:38:31 | gibi | bauzas: thanks I will try to fight sphinx alone then :) | |
| 09:38:36 | bauzas | kashyap: let me just provide a new revision | |
| 09:38:47 | kashyap | bauzas: Hang on | |
| 09:38:53 | sean-k-mooney | kashyap could you update the release not with the explaning you gave to me about the 3 booleans that the cachemodes set to explain why its safe to ues writeback | |
| 09:39:00 | kashyap | bauzas: Just post your new text as a comment | |
| 09:39:34 | sean-k-mooney | kashyap: that is the context that ops will be missing. it was the context i was missing when i saw the commit first | |
| 09:39:59 | sean-k-mooney | ops wont see the commit message so all they have is the release note | |
| 09:40:35 | bauzas | sean-k-mooney: yeah I agree | |
| 09:41:01 | bauzas | sean-k-mooney: kashyap did a great work on documenting it in a docstring https://review.openstack.org/#/c/641981/13/nova/virt/libvirt/driver.py@410 | |
| 09:41:11 | bauzas | but we don't take the opportunity to explain it to ops | |
| 09:41:28 | bauzas | while it could be scary "heh, wtf, now I could loose data integrity?" | |
| 09:42:27 | sean-k-mooney | right well quickly looking at that it could be more or less copied into the release note | |
| 09:42:55 | sean-k-mooney | it might be a little verbose but that is not a bad thing | |
| 09:43:39 | sean-k-mooney | i think all of the context is already in the patch just need to duplicate some of it into the release note | |
| 09:55:18 | bauzas | mdbooth: kashyap: sean-k-mooney: so, I provided an example of what I'd like to see in the relnote | |
| 09:55:48 | bauzas | if operators don't want to expect modern OSes, it's their choices | |
| 09:55:51 | mdbooth | bauzas: I saw that. Honestly, though, this hasn't been an issue for at least a decade, maybe 2. | |
| 09:56:11 | bauzas | mdbooth: not really, you can enable caching on some OSes that don't flush, right? | |
| 09:56:13 | mdbooth | I think it just increases cognitive load on the reader for no practical benefit. | |
| 09:56:28 | sean-k-mooney | mdbooth: there are still people running centos 5 | |
| 09:56:28 | bauzas | from my recall, W2K is one of those | |
| 09:56:41 | mdbooth | sean-k-mooney: Centos 5 definitely flushes! | |
| 09:56:42 | sean-k-mooney | mdbooth: didnt it have issue with flushing | |
| 09:56:45 | sean-k-mooney | ok | |
| 09:57:16 | sean-k-mooney | that was the distro that i always worried about for this kind of thing | |
| 09:57:33 | mdbooth | If your OS doesn't flush data on request, you're going to find out about that real soon. | |
| 09:58:06 | sean-k-mooney | i know it does not partcally like virtio disks as i useulaly had to revert to useing other disk bus like ide to avoid kernel issues | |
| 09:58:32 | mdbooth | I remember having to be very careful shutting down my old 286-clone Tandy running DOS, though. | |
| 09:58:56 | bauzas | oh, I remember some PITAs I got with Win2K guests in the past with OpenStack Essex | |
| 09:59:03 | bauzas | I'm pretty sure they weren't flushing | |
| 09:59:23 | sean-k-mooney | im sure that at least 50% of all openstack workloads :P | |
| 09:59:26 | bauzas | but this could have been fixed in a SP or something else | |
| 09:59:35 | sean-k-mooney | dos gaming in the cloud | |
| 10:00:12 | bauzas | actually, that's a good question now we have containerizing OSes like Atomic | |
| 10:00:25 | bauzas | do we still flush on disk ? | |
| 10:00:40 | sean-k-mooney | with containers? | |
| 10:00:45 | sean-k-mooney | that shoudl not change | |
| 10:01:03 | mdbooth | bauzas: It's really an application issue: your application needs to flush. | |
| 10:01:23 | mdbooth | If the application flushes, the OS should also flush. | |
| 10:01:53 | mdbooth | The only thing writeback vs writethrough is going to fix is if your application flushes but the OS does not. | |
| 10:02:32 | mdbooth | If the application doesn't flush, the OS can't flush it anyway, as it may not have the data. | |
| 10:03:57 | bauzas | hum, reading https://lifehacker.com/do-i-really-need-to-eject-usb-drives-before-removing-th-5863810 | |
| 10:04:08 | bauzas | man, why is this so complicated ? | |
| 10:04:31 | bauzas | anyway, I agree, it's maybe just a strawman | |
| 10:04:43 | bauzas | but I still feel we need to explain it | |
| 10:05:10 | mdbooth | bauzas: USB drives don't need O_SYNC either if your application flushes and you wait for your application to finish before unplugging. | |
| 10:05:44 | mdbooth | Eject flushes the whole disk. | |
| 10:08:43 | bauzas | yeah, just reading http://msdn.microsoft.com/en-us/library/windows/desktop/aa364218%28v=vs.85%29.aspx | |
| 10:08:56 | bauzas | anyway, I'm balanced | |
| 10:09:21 | bauzas | see, we're talking for 30 mins about something needing more than just a glance | |
| 10:09:41 | bauzas | the fact that mdbooth and kashyap know the situation makes reasonable the change | |
| 10:09:58 | bauzas | but I just feel we need to capture this knowledge | |
| 10:10:08 | bauzas | and explain it thru the relnote | |
| 10:34:37 | stephenfin | gibi: Looks like you solved it :) | |
| 10:35:03 | gibi | stephenfin: yeah, I managed. | |
| 10:51:56 | sean-k-mooney | bauzas: for the record we can talk for 30 mins about just about any topic :) | |
| 11:41:39 | kashyap | bauzas: Just back; ah, I mistook your formatting comment | |
| 11:52:12 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fix links to neutron QoS minimum bandwidth doc https://review.openstack.org/645084 | |
| 12:01:03 | kashyap | sean-k-mooney: CentOS-5 is perfectly fine w.r.t. flushes. Again, as noted only *horribly* old OSes that don't matter (like RHEL-4 or even older versions of Windoze) | |
| 12:01:21 | kashyap | ... that had problems. And they're not relevant. | |
| 12:02:00 | sean-k-mooney | kashyap: ya | |
| 12:02:17 | kashyap | sean-k-mooney: /me reworks the rel note; and you're right - some of the info I could add there | |
| 12:02:30 | kashyap | But as Matt noted, I was being careful of not adding needless cognitive load to the reader. | |
| 12:02:49 | sean-k-mooney | i just have had data curption issue with centos 5 and centos 5 based vnf in the past and was not sure if that was due to flushing | |
| 12:05:20 | sean-k-mooney | that said it could equally have been down to the use of ext2/3? i cant rememebr but all i recal was we had to be very care to ensure we shutdown cleanly or we woudl have to do a rebuild with a clean image and/or do a rescue to fix the file system | |
| 12:15:10 | kashyap | sean-k-mooney: bauzas: Before I upload, does this read better? -- https://kashyapc.fedorapeople.org/PS14-writeback-cache-mode-for-guests-a7e4d2806c956164.yaml.txt | |