VFP (5) exports to FPW, but will VFP export to FPD? If so, maybe the VFP6

with DBC rules might be an interesting combined solution.

mjar

At 10:19 PM 8/29/99 -0700, you wrote:

>

>Thanks for the input, Bill:

>

>Since the data would probably require a couple of steps' worth of massaging

>I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

>there (I'm not sure, not having seen the whole project) unless I learn that

>low-level file functions are significantly faster. Unless they want to

>change from FPD, I don't get any .DBC capabilities, either, so a middle

>step of massaging seems likely anyway.

>

>Russ Scott

>russsct@ix.netcom.com

>

>====================================

>At 10:17 PM 8/29/99 -0400, you wrote:

>>Russ,

>>

>>This is a matter of great concern to me too, and I posted a question and got

>>some interesting responses not too long ago on the subject. I have a monthly

>>job that imports a few million records spread across almost a thousand

>>tables (ZIP4 tables for the entire US). Because the data has to be massaged

>>on the way into tables, I originally setup the import process to (low level

>>I/O) read the ASCII fixed fields table, do the work and then APPEND BLANK to

>>add a record to the database.

>>

>>I sped up the process a little by APPENDing blank records in 1,000 record

>>"blocks" and then replacing the blank fields of each until the 1,000 empty

>>records was filled and repeated the process again.

>>

>>Working this way, the import takes something like 2 days on a P133 machine.

>>A faster machine probably would help, but relegating the job to a 'spare'

>>machine that just chugs away seems to be the least bothersome. One thing I

>>did do is design the process so multiple machines could participate in a

>>time crunch, and that does work very well when needed.

>>

>>Suggestions I received for speeding up the process included using APPEND

>>TYPE SDF, which would bring in each table en masse, but that would have to

>>be followed with another step to do the massaging needed to prepare the

>>tables for use, since UDFs can't be used on individual fields with APPEND

>>TYPE SDF (but that would only partially address the data massaging

>>requirements anyway). Another suggestion, by Anders, that seems to have a

>>lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

>>rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

>>I'm anxious to try this, but can't get to it yet.

>>

>>In general, processing each input record and then adding it to a table using

>>APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

>>

>>As for FPD versus VFP in terms of speed, I know some people think that FPD

>>if faster, but my experience so far with VFP shows that VFP (6, anyway, with

>>SP3 if that means anything) it's actually faster than equivalent FPD apps. I

>>attribute the difference to better memory management in VFP, but that's only

>>my gut feeling (wish I had better measurement tools; I like the Profiler,

>>but can't find a way to track I/O utilization). I have not yet tested the

>>bulk-import under VFP, the tests I'm referring to are different, but they

>>are long running processor and I/O intensive jobs.

>>

>>Bill

>>

>>

>>-----Original Message-----

>>From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

>>Sent: Sunday, August 29, 1999 9:57 PM

>>To: Multiple recipients of ProFox

>>Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

>>processing

>>

>>

>>

>>Hey, all:

>>

>>I have a chance to investigate a data conversion job. This would involve

>>importing from a flat file into FP tables, formatting/ massaging it, then

>>outputting it to another set of flat files. The company uses FPD (not sure

>>which version) instead of VFP because "FPD is faster". To give you an idea

>>of the size of the jobs involved, they take from ~1 hour to multiple days.

>>

>>This causes me to ask some questions:

>>

>>Would FPD really be faster for data conversion if you compared VFP under

>>Win9x/NT to FPD running on the same machine booted to DOS only? Does the

>>version of DOS make a significant difference? Does anyone have any hard

>>data, educated guesses or gut feelings about this?

>>

>>Are there considerations besides speed (reliablility for example) to take

>>into consideration?

>>

>>On a related note, has anyone clocked APPEND FROM... and COPY TO... against

>>low-level file commands?

>>

>>I don't have to talk to these people for a couple of days and was curious

>>if anyone else had looked into these issues.

>>

>>Any ideas?

>>

>>

>>Thanks:

>>

>>Russ Scott

>>russsct@ix.netcom.com

>>

>>

>>

>>

>>

>>

>>

>>

>

>

>

>