You can change a FPD table to a VFP format table as simply as ADD TABLE x

-Anders

----- Original Message -----

From: Russ Scott <russsct@ix.netcom.com>

To: Multiple recipients of ProFox <profox@leafe.com>

Sent: mÃ

’ndag den 30 augusti 1999 07:19

Subject: RE: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

processing

|

| Thanks for the input, Bill:

|

| Since the data would probably require a couple of steps' worth of

massaging

| I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

| there (I'm not sure, not having seen the whole project) unless I learn

that

| low-level file functions are significantly faster. Unless they want to

| change from FPD, I don't get any .DBC capabilities, either, so a middle

| step of massaging seems likely anyway.

|

| Russ Scott

| russsct@ix.netcom.com

|

| ====================================

| At 10:17 PM 8/29/99 -0400, you wrote:

| >Russ,

| >

| >This is a matter of great concern to me too, and I posted a question and

got

| >some interesting responses not too long ago on the subject. I have a

monthly

| >job that imports a few million records spread across almost a thousand

| >tables (ZIP4 tables for the entire US). Because the data has to be

massaged

| >on the way into tables, I originally setup the import process to (low

level

| >I/O) read the ASCII fixed fields table, do the work and then APPEND BLANK

to

| >add a record to the database.

| >

| >I sped up the process a little by APPENDing blank records in 1,000 record

| >"blocks" and then replacing the blank fields of each until the 1,000

empty

| >records was filled and repeated the process again.

| >

| >Working this way, the import takes something like 2 days on a P133

machine.

| >A faster machine probably would help, but relegating the job to a 'spare'

| >machine that just chugs away seems to be the least bothersome. One thing

I

| >did do is design the process so multiple machines could participate in a

| >time crunch, and that does work very well when needed.

| >

| >Suggestions I received for speeding up the process included using APPEND

| >TYPE SDF, which would bring in each table en masse, but that would have

to

| >be followed with another step to do the massaging needed to prepare the

| >tables for use, since UDFs can't be used on individual fields with APPEND

| >TYPE SDF (but that would only partially address the data massaging

| >requirements anyway). Another suggestion, by Anders, that seems to have a

| >lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

| >rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

| >I'm anxious to try this, but can't get to it yet.

| >

| >In general, processing each input record and then adding it to a table

using

| >APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

| >

| >As for FPD versus VFP in terms of speed, I know some people think that

FPD

| >if faster, but my experience so far with VFP shows that VFP (6, anyway,

with

| >SP3 if that means anything) it's actually faster than equivalent FPD

apps. I

| >attribute the difference to better memory management in VFP, but that's

only

| >my gut feeling (wish I had better measurement tools; I like the Profiler,

| >but can't find a way to track I/O utilization). I have not yet tested the

| >bulk-import under VFP, the tests I'm referring to are different, but they

| >are long running processor and I/O intensive jobs.

| >

| >Bill

| >

| >

| >-----Original Message-----

| >From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

| >Sent: Sunday, August 29, 1999 9:57 PM

| >To: Multiple recipients of ProFox

| >Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

| >processing

| >

| >

| >

| >Hey, all:

| >

| >I have a chance to investigate a data conversion job. This would involve

| >importing from a flat file into FP tables, formatting/ massaging it, then

| >outputting it to another set of flat files. The company uses FPD (not

sure

| >which version) instead of VFP because "FPD is faster". To give you an

idea

| >of the size of the jobs involved, they take from ~1 hour to multiple

days.

| >

| >This causes me to ask some questions:

| >

| >Would FPD really be faster for data conversion if you compared VFP under

| >Win9x/NT to FPD running on the same machine booted to DOS only? Does the

| >version of DOS make a significant difference? Does anyone have any hard

| >data, educated guesses or gut feelings about this?

| >

| >Are there considerations besides speed (reliablility for example) to take

| >into consideration?

| >

| >On a related note, has anyone clocked APPEND FROM... and COPY TO...

against

| >low-level file commands?

| >

| >I don't have to talk to these people for a couple of days and was curious

| >if anyone else had looked into these issues.

| >

| >Any ideas?

| >

| >

| >Thanks:

| >

| >Russ Scott

| >russsct@ix.netcom.com

| >

| >

| >

| >

| >

| >

| >

| >

|

|

|