Re: [NF] For those interested in Big Data

Author: Malcolm Greene

Posted: 2011-09-24 at 09:15:36

Paul,

> I think most people just store the image data, video data, whatever, as a node in the database. You want to limit the number of things to search for and pulling the pointer from the database and then the file from the filesystem wouldn't be as fast as just pulling the blob from the db to begin with.

I respectfully disagree with your position. There was a recent thread on

the SQLite mailing list that included some benchmarks that argued

against use of blobs for storing files. While much of the discussion was

SQLite specific, I think that many of the principles would also apply to

other databases.

Some generic reasons to store files outside of a database include the

ability to use lower cost/speed file systems for file storage, faster

database backups, and faster database maintenance activities when

schemas change. Also much less database fragmentation.

The only reason I can think to store files inside a database is for

convenience, eg. having a single container for all your data.

Curious to hear other opinions on this topic.

Malcolm

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://leafe.com/mailman/listinfo/profox

OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/1316870136.3146.140258147237085@webmail.messagingengine.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2011 Malcolm Greene