Also, I would have thought that the later versions of FP would have

enhanced Rushmore performance - or would that not apply because you are

most likely going without indexes and therefor non-optimizable? Also, maybe

the enhanced performance is eaten away by the Windows overhead?

Just some thoughts....

mjar

At 11:39 PM 8/29/99 -0700, you wrote:

>VFP (5) exports to FPW, but will VFP export to FPD? If so, maybe the VFP6

>with DBC rules might be an interesting combined solution.

>

>mjar

>

>

>At 10:19 PM 8/29/99 -0700, you wrote:

>>

>>Thanks for the input, Bill:

>>

>>Since the data would probably require a couple of steps' worth of massaging

>>I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

>>there (I'm not sure, not having seen the whole project) unless I learn that

>>low-level file functions are significantly faster. Unless they want to

>>change from FPD, I don't get any .DBC capabilities, either, so a middle

>>step of massaging seems likely anyway.

>>

>>Russ Scott

>>russsct@ix.netcom.com

>>

>>====================================

>>At 10:17 PM 8/29/99 -0400, you wrote:

>>>Russ,

>>>

>>>This is a matter of great concern to me too, and I posted a question and

got

>>>some interesting responses not too long ago on the subject. I have a

monthly

>>>job that imports a few million records spread across almost a thousand

>>>tables (ZIP4 tables for the entire US). Because the data has to be massaged

>>>on the way into tables, I originally setup the import process to (low level

>>>I/O) read the ASCII fixed fields table, do the work and then APPEND

BLANK to

>>>add a record to the database.

>>>

>>>I sped up the process a little by APPENDing blank records in 1,000 record

>>>"blocks" and then replacing the blank fields of each until the 1,000 empty

>>>records was filled and repeated the process again.

>>>

>>>Working this way, the import takes something like 2 days on a P133 machine.

>>>A faster machine probably would help, but relegating the job to a 'spare'

>>>machine that just chugs away seems to be the least bothersome. One thing I

>>>did do is design the process so multiple machines could participate in a

>>>time crunch, and that does work very well when needed.

>>>

>>>Suggestions I received for speeding up the process included using APPEND

>>>TYPE SDF, which would bring in each table en masse, but that would have to

>>>be followed with another step to do the massaging needed to prepare the

>>>tables for use, since UDFs can't be used on individual fields with APPEND

>>>TYPE SDF (but that would only partially address the data massaging

>>>requirements anyway). Another suggestion, by Anders, that seems to have a

>>>lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

>>>rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

>>>I'm anxious to try this, but can't get to it yet.

>>>

>>>In general, processing each input record and then adding it to a table

using

>>>APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

>>>

>>>As for FPD versus VFP in terms of speed, I know some people think that FPD

>>>if faster, but my experience so far with VFP shows that VFP (6, anyway,

with

>>>SP3 if that means anything) it's actually faster than equivalent FPD

apps. I

>>>attribute the difference to better memory management in VFP, but that's

only

>>>my gut feeling (wish I had better measurement tools; I like the Profiler,

>>>but can't find a way to track I/O utilization). I have not yet tested the

>>>bulk-import under VFP, the tests I'm referring to are different, but they

>>>are long running processor and I/O intensive jobs.

>>>

>>>Bill

>>>

>>>

>>>-----Original Message-----

>>>From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

>>>Sent: Sunday, August 29, 1999 9:57 PM

>>>To: Multiple recipients of ProFox

>>>Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

>>>processing

>>>

>>>

>>>

>>>Hey, all:

>>>

>>>I have a chance to investigate a data conversion job. This would involve

>>>importing from a flat file into FP tables, formatting/ massaging it, then

>>>outputting it to another set of flat files. The company uses FPD (not sure

>>>which version) instead of VFP because "FPD is faster". To give you an idea

>>>of the size of the jobs involved, they take from ~1 hour to multiple days.

>>>

>>>This causes me to ask some questions:

>>>

>>>Would FPD really be faster for data conversion if you compared VFP under

>>>Win9x/NT to FPD running on the same machine booted to DOS only? Does the

>>>version of DOS make a significant difference? Does anyone have any hard

>>>data, educated guesses or gut feelings about this?

>>>

>>>Are there considerations besides speed (reliablility for example) to take

>>>into consideration?

>>>

>>>On a related note, has anyone clocked APPEND FROM... and COPY TO... against

>>>low-level file commands?

>>>

>>>I don't have to talk to these people for a couple of days and was curious

>>>if anyone else had looked into these issues.

>>>

>>>Any ideas?

>>>

>>>

>>>Thanks:

>>>

>>>Russ Scott

>>>russsct@ix.netcom.com

>>>

>>>

>>>

>>>

>>>

>>>

>>>

>>>

>>

>>

>>

>>

>

>

>

>