Malcolm,

Been there, done that in terms of storing outside the database. Fine when it

is small amount of data but a nightmare from a security point of view with

large amounts of data.

We moved soma 600Gb of scanned & audio files from a VFP database (stored

only a link to the file) because of thise. Unfortunately if your folder

structure gets fouled up or slow to access say because you have more than

about 5,000 files in a folder then it totally screws you. Also people are

tempted to meddle with data in folders and a simple inadvertand rename of a

folder or file can render your data useless.

Now at least I sleep knowing the data is safe in SQL Server and is being

mirrored, snapshot and replicated without me doing anything. Also access is

of course quick and doesn't degrade. Currently the initial 600Gb has

increased to over 3Tb as the amount of paperwork has been reduced in the

company and it still runs as fast. Also, nobody can get at it from a

security point of view without knowing how to access SQL, so our company

security auditors are happy.

Dave

-----Original Message-----

From: profox-bounces@leafe.com [mailto:profox-bounces@leafe.com] On Behalf

Of Malcolm Greene

Sent: 24 September 2011 14:16

To: profox@leafe.com

Subject: Re: [NF] For those interested in Big Data

Paul,

> I think most people just store the image data, video data, whatever, as a

node in the database. You want to limit the number of things to search for

and pulling the pointer from the database and then the file from the

filesystem wouldn't be as fast as just pulling the blob from the db to begin

with.

I respectfully disagree with your position. There was a recent thread on

the SQLite mailing list that included some benchmarks that argued

against use of blobs for storing files. While much of the discussion was

SQLite specific, I think that many of the principles would also apply to

other databases.

Some generic reasons to store files outside of a database include the

ability to use lower cost/speed file systems for file storage, faster

database backups, and faster database maintenance activities when

schemas change. Also much less database fragmentation.

The only reason I can think to store files inside a database is for

convenience, eg. having a single container for all your data.

Curious to hear other opinions on this topic.

Malcolm

[excessive quoting removed by server]

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://leafe.com/mailman/listinfo/profox

OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/000301cc7bdc$adfca8e0$09f5faa0$@co.uk

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.