Hey, all:

I have a chance to investigate a data conversion job. This would involve

importing from a flat file into FP tables, formatting/ massaging it, then

outputting it to another set of flat files. The company uses FPD (not sure

which version) instead of VFP because "FPD is faster". To give you an idea

of the size of the jobs involved, they take from ~1 hour to multiple days.

This causes me to ask some questions:

Would FPD really be faster for data conversion if you compared VFP under

Win9x/NT to FPD running on the same machine booted to DOS only? Does the

version of DOS make a significant difference? Does anyone have any hard

data, educated guesses or gut feelings about this?

Are there considerations besides speed (reliablility for example) to take

into consideration?

On a related note, has anyone clocked APPEND FROM... and COPY TO... against

low-level file commands?

I don't have to talk to these people for a couple of days and was curious

if anyone else had looked into these issues.

Any ideas?

Thanks:

Russ Scott

russsct@ix.netcom.com

Russ,

This is a matter of great concern to me too, and I posted a question and got

some interesting responses not too long ago on the subject. I have a monthly

job that imports a few million records spread across almost a thousand

tables (ZIP4 tables for the entire US). Because the data has to be massaged

on the way into tables, I originally setup the import process to (low level

I/O) read the ASCII fixed fields table, do the work and then APPEND BLANK to

add a record to the database.

I sped up the process a little by APPENDing blank records in 1,000 record

"blocks" and then replacing the blank fields of each until the 1,000 empty

records was filled and repeated the process again.

Working this way, the import takes something like 2 days on a P133 machine.

A faster machine probably would help, but relegating the job to a 'spare'

machine that just chugs away seems to be the least bothersome. One thing I

did do is design the process so multiple machines could participate in a

time crunch, and that does work very well when needed.

Suggestions I received for speeding up the process included using APPEND

TYPE SDF, which would bring in each table en masse, but that would have to

be followed with another step to do the massaging needed to prepare the

tables for use, since UDFs can't be used on individual fields with APPEND

TYPE SDF (but that would only partially address the data massaging

requirements anyway). Another suggestion, by Anders, that seems to have a

lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

I'm anxious to try this, but can't get to it yet.

In general, processing each input record and then adding it to a table using

APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

As for FPD versus VFP in terms of speed, I know some people think that FPD

if faster, but my experience so far with VFP shows that VFP (6, anyway, with

SP3 if that means anything) it's actually faster than equivalent FPD apps. I

attribute the difference to better memory management in VFP, but that's only

my gut feeling (wish I had better measurement tools; I like the Profiler,

but can't find a way to track I/O utilization). I have not yet tested the

bulk-import under VFP, the tests I'm referring to are different, but they

are long running processor and I/O intensive jobs.

Bill

-----Original Message-----

From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

Sent: Sunday, August 29, 1999 9:57 PM

To: Multiple recipients of ProFox

Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

processing

Hey, all:

I have a chance to investigate a data conversion job. This would involve

importing from a flat file into FP tables, formatting/ massaging it, then

outputting it to another set of flat files. The company uses FPD (not sure

which version) instead of VFP because "FPD is faster". To give you an idea

of the size of the jobs involved, they take from ~1 hour to multiple days.

This causes me to ask some questions:

Would FPD really be faster for data conversion if you compared VFP under

Win9x/NT to FPD running on the same machine booted to DOS only? Does the

version of DOS make a significant difference? Does anyone have any hard

data, educated guesses or gut feelings about this?

Are there considerations besides speed (reliablility for example) to take

into consideration?

On a related note, has anyone clocked APPEND FROM... and COPY TO... against

low-level file commands?

I don't have to talk to these people for a couple of days and was curious

if anyone else had looked into these issues.

Any ideas?

Thanks:

Russ Scott

russsct@ix.netcom.com

Thanks for the input, Bill:

Since the data would probably require a couple of steps' worth of massaging

I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

there (I'm not sure, not having seen the whole project) unless I learn that

low-level file functions are significantly faster. Unless they want to

change from FPD, I don't get any .DBC capabilities, either, so a middle

step of massaging seems likely anyway.

Russ Scott

russsct@ix.netcom.com

====================================

At 10:17 PM 8/29/99 -0400, you wrote:

>Russ,

>

>This is a matter of great concern to me too, and I posted a question and got

>some interesting responses not too long ago on the subject. I have a monthly

>job that imports a few million records spread across almost a thousand

>tables (ZIP4 tables for the entire US). Because the data has to be massaged

>on the way into tables, I originally setup the import process to (low level

>I/O) read the ASCII fixed fields table, do the work and then APPEND BLANK to

>add a record to the database.

>

>I sped up the process a little by APPENDing blank records in 1,000 record

>"blocks" and then replacing the blank fields of each until the 1,000 empty

>records was filled and repeated the process again.

>

>Working this way, the import takes something like 2 days on a P133 machine.

>A faster machine probably would help, but relegating the job to a 'spare'

>machine that just chugs away seems to be the least bothersome. One thing I

>did do is design the process so multiple machines could participate in a

>time crunch, and that does work very well when needed.

>

>Suggestions I received for speeding up the process included using APPEND

>TYPE SDF, which would bring in each table en masse, but that would have to

>be followed with another step to do the massaging needed to prepare the

>tables for use, since UDFs can't be used on individual fields with APPEND

>TYPE SDF (but that would only partially address the data massaging

>requirements anyway). Another suggestion, by Anders, that seems to have a

>lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

>rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

>I'm anxious to try this, but can't get to it yet.

>

>In general, processing each input record and then adding it to a table using

>APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

>

>As for FPD versus VFP in terms of speed, I know some people think that FPD

>if faster, but my experience so far with VFP shows that VFP (6, anyway, with

>SP3 if that means anything) it's actually faster than equivalent FPD apps. I

>attribute the difference to better memory management in VFP, but that's only

>my gut feeling (wish I had better measurement tools; I like the Profiler,

>but can't find a way to track I/O utilization). I have not yet tested the

>bulk-import under VFP, the tests I'm referring to are different, but they

>are long running processor and I/O intensive jobs.

>

>Bill

>

>

>-----Original Message-----

>From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

>Sent: Sunday, August 29, 1999 9:57 PM

>To: Multiple recipients of ProFox

>Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

>processing

>

>

>

>Hey, all:

>

>I have a chance to investigate a data conversion job. This would involve

>importing from a flat file into FP tables, formatting/ massaging it, then

>outputting it to another set of flat files. The company uses FPD (not sure

>which version) instead of VFP because "FPD is faster". To give you an idea

>of the size of the jobs involved, they take from ~1 hour to multiple days.

>

>This causes me to ask some questions:

>

>Would FPD really be faster for data conversion if you compared VFP under

>Win9x/NT to FPD running on the same machine booted to DOS only? Does the

>version of DOS make a significant difference? Does anyone have any hard

>data, educated guesses or gut feelings about this?

>

>Are there considerations besides speed (reliablility for example) to take

>into consideration?

>

>On a related note, has anyone clocked APPEND FROM... and COPY TO... against

>low-level file commands?

>

>I don't have to talk to these people for a couple of days and was curious

>if anyone else had looked into these issues.

>

>Any ideas?

>

>

>Thanks:

>

>Russ Scott

>russsct@ix.netcom.com

>

>

>

>

>

>

>

>

Russ, the low level file functions would speed things up if they could

target a table, but having to do APPEND BLANKs kills it, for some reason

that I don't understand. Even without indexes, the operation crawls. I don't

remember where I saw it, or perhaps I'm just making it up, but there

shouldn't be any reason why a 'low level' routine (maybe this would be a

good use for C) can't load a .dbf table (mimicking header and stuff).

Imagine loading tables with the speed of, say, PKZIP?

Bill

-----Original Message-----

From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

Sent: Monday, August 30, 1999 1:20 AM

To: Multiple recipients of ProFox

Subject: RE: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

processing

Thanks for the input, Bill:

Since the data would probably require a couple of steps' worth of massaging

I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

there (I'm not sure, not having seen the whole project) unless I learn that

low-level file functions are significantly faster. Unless they want to

change from FPD, I don't get any .DBC capabilities, either, so a middle

step of massaging seems likely anyway.

Russ Scott

russsct@ix.netcom.com

VFP (5) exports to FPW, but will VFP export to FPD? If so, maybe the VFP6

with DBC rules might be an interesting combined solution.

mjar

At 10:19 PM 8/29/99 -0700, you wrote:

>

>Thanks for the input, Bill:

>

>Since the data would probably require a couple of steps' worth of massaging

>I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

>there (I'm not sure, not having seen the whole project) unless I learn that

>low-level file functions are significantly faster. Unless they want to

>change from FPD, I don't get any .DBC capabilities, either, so a middle

>step of massaging seems likely anyway.

>

>Russ Scott

>russsct@ix.netcom.com

>

>====================================

>At 10:17 PM 8/29/99 -0400, you wrote:

>>Russ,

>>

>>This is a matter of great concern to me too, and I posted a question and got

>>some interesting responses not too long ago on the subject. I have a monthly

>>job that imports a few million records spread across almost a thousand

>>tables (ZIP4 tables for the entire US). Because the data has to be massaged

>>on the way into tables, I originally setup the import process to (low level

>>I/O) read the ASCII fixed fields table, do the work and then APPEND BLANK to

>>add a record to the database.

>>

>>I sped up the process a little by APPENDing blank records in 1,000 record

>>"blocks" and then replacing the blank fields of each until the 1,000 empty

>>records was filled and repeated the process again.

>>

>>Working this way, the import takes something like 2 days on a P133 machine.

>>A faster machine probably would help, but relegating the job to a 'spare'

>>machine that just chugs away seems to be the least bothersome. One thing I

>>did do is design the process so multiple machines could participate in a

>>time crunch, and that does work very well when needed.

>>

>>Suggestions I received for speeding up the process included using APPEND

>>TYPE SDF, which would bring in each table en masse, but that would have to

>>be followed with another step to do the massaging needed to prepare the

>>tables for use, since UDFs can't be used on individual fields with APPEND

>>TYPE SDF (but that would only partially address the data massaging

>>requirements anyway). Another suggestion, by Anders, that seems to have a

>>lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

>>rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

>>I'm anxious to try this, but can't get to it yet.

>>

>>In general, processing each input record and then adding it to a table using

>>APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

>>

>>As for FPD versus VFP in terms of speed, I know some people think that FPD

>>if faster, but my experience so far with VFP shows that VFP (6, anyway, with

>>SP3 if that means anything) it's actually faster than equivalent FPD apps. I

>>attribute the difference to better memory management in VFP, but that's only

>>my gut feeling (wish I had better measurement tools; I like the Profiler,

>>but can't find a way to track I/O utilization). I have not yet tested the

>>bulk-import under VFP, the tests I'm referring to are different, but they

>>are long running processor and I/O intensive jobs.

>>

>>Bill

>>

>>

>>-----Original Message-----

>>From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

>>Sent: Sunday, August 29, 1999 9:57 PM

>>To: Multiple recipients of ProFox

>>Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

>>processing

>>

>>

>>

>>Hey, all:

>>

>>I have a chance to investigate a data conversion job. This would involve

>>importing from a flat file into FP tables, formatting/ massaging it, then

>>outputting it to another set of flat files. The company uses FPD (not sure

>>which version) instead of VFP because "FPD is faster". To give you an idea

>>of the size of the jobs involved, they take from ~1 hour to multiple days.

>>

>>This causes me to ask some questions:

>>

>>Would FPD really be faster for data conversion if you compared VFP under

>>Win9x/NT to FPD running on the same machine booted to DOS only? Does the

>>version of DOS make a significant difference? Does anyone have any hard

>>data, educated guesses or gut feelings about this?

>>

>>Are there considerations besides speed (reliablility for example) to take

>>into consideration?

>>

>>On a related note, has anyone clocked APPEND FROM... and COPY TO... against

>>low-level file commands?

>>

>>I don't have to talk to these people for a couple of days and was curious

>>if anyone else had looked into these issues.

>>

>>Any ideas?

>>

>>

>>Thanks:

>>

>>Russ Scott

>>russsct@ix.netcom.com

>>

>>

>>

>>

>>

>>

>>

>>

>

>

>

>

Also, I would have thought that the later versions of FP would have

enhanced Rushmore performance - or would that not apply because you are

most likely going without indexes and therefor non-optimizable? Also, maybe

the enhanced performance is eaten away by the Windows overhead?

Just some thoughts....

mjar

At 11:39 PM 8/29/99 -0700, you wrote:

>VFP (5) exports to FPW, but will VFP export to FPD? If so, maybe the VFP6

>with DBC rules might be an interesting combined solution.

>

>mjar

>

>

>At 10:19 PM 8/29/99 -0700, you wrote:

>>

>>Thanks for the input, Bill:

>>

>>Since the data would probably require a couple of steps' worth of massaging

>>I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

>>there (I'm not sure, not having seen the whole project) unless I learn that

>>low-level file functions are significantly faster. Unless they want to

>>change from FPD, I don't get any .DBC capabilities, either, so a middle

>>step of massaging seems likely anyway.

>>

>>Russ Scott

>>russsct@ix.netcom.com

>>

>>====================================

>>At 10:17 PM 8/29/99 -0400, you wrote:

>>>Russ,

>>>

>>>This is a matter of great concern to me too, and I posted a question and

got

>>>some interesting responses not too long ago on the subject. I have a

monthly

>>>job that imports a few million records spread across almost a thousand

>>>tables (ZIP4 tables for the entire US). Because the data has to be massaged

>>>on the way into tables, I originally setup the import process to (low level

>>>I/O) read the ASCII fixed fields table, do the work and then APPEND

BLANK to

>>>add a record to the database.

>>>

>>>I sped up the process a little by APPENDing blank records in 1,000 record

>>>"blocks" and then replacing the blank fields of each until the 1,000 empty

>>>records was filled and repeated the process again.

>>>

>>>Working this way, the import takes something like 2 days on a P133 machine.

>>>A faster machine probably would help, but relegating the job to a 'spare'

>>>machine that just chugs away seems to be the least bothersome. One thing I

>>>did do is design the process so multiple machines could participate in a

>>>time crunch, and that does work very well when needed.

>>>

>>>Suggestions I received for speeding up the process included using APPEND

>>>TYPE SDF, which would bring in each table en masse, but that would have to

>>>be followed with another step to do the massaging needed to prepare the

>>>tables for use, since UDFs can't be used on individual fields with APPEND

>>>TYPE SDF (but that would only partially address the data massaging

>>>requirements anyway). Another suggestion, by Anders, that seems to have a

>>>lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

>>>rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

>>>I'm anxious to try this, but can't get to it yet.

>>>

>>>In general, processing each input record and then adding it to a table

using

>>>APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

>>>

>>>As for FPD versus VFP in terms of speed, I know some people think that FPD

>>>if faster, but my experience so far with VFP shows that VFP (6, anyway,

with

>>>SP3 if that means anything) it's actually faster than equivalent FPD

apps. I

>>>attribute the difference to better memory management in VFP, but that's

only

>>>my gut feeling (wish I had better measurement tools; I like the Profiler,

>>>but can't find a way to track I/O utilization). I have not yet tested the

>>>bulk-import under VFP, the tests I'm referring to are different, but they

>>>are long running processor and I/O intensive jobs.

>>>

>>>Bill

>>>

>>>

>>>-----Original Message-----

>>>From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

>>>Sent: Sunday, August 29, 1999 9:57 PM

>>>To: Multiple recipients of ProFox

>>>Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

>>>processing

>>>

>>>

>>>

>>>Hey, all:

>>>

>>>I have a chance to investigate a data conversion job. This would involve

>>>importing from a flat file into FP tables, formatting/ massaging it, then

>>>outputting it to another set of flat files. The company uses FPD (not sure

>>>which version) instead of VFP because "FPD is faster". To give you an idea

>>>of the size of the jobs involved, they take from ~1 hour to multiple days.

>>>

>>>This causes me to ask some questions:

>>>

>>>Would FPD really be faster for data conversion if you compared VFP under

>>>Win9x/NT to FPD running on the same machine booted to DOS only? Does the

>>>version of DOS make a significant difference? Does anyone have any hard

>>>data, educated guesses or gut feelings about this?

>>>

>>>Are there considerations besides speed (reliablility for example) to take

>>>into consideration?

>>>

>>>On a related note, has anyone clocked APPEND FROM... and COPY TO... against

>>>low-level file commands?

>>>

>>>I don't have to talk to these people for a couple of days and was curious

>>>if anyone else had looked into these issues.

>>>

>>>Any ideas?

>>>

>>>

>>>Thanks:

>>>

>>>Russ Scott

>>>russsct@ix.netcom.com

>>>

>>>

>>>

>>>

>>>

>>>

>>>

>>>

>>

>>

>>

>>

>

>

>

>

INSERT IN TO is faster, fewer locks for one thing, and INSERT .. FROM ARRAY

would probably be faster still.

-Anders

----- Original Message -----

From: Bill Arnold <bill@wjarnold.com>

To: Multiple recipients of ProFox <profox@leafe.com>

Sent: mÃ

’ndag den 30 augusti 1999 08:27

Subject: RE: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

processing

| Russ, the low level file functions would speed things up if they could

| target a table, but having to do APPEND BLANKs kills it, for some reason

| that I don't understand. Even without indexes, the operation crawls. I

don't

| remember where I saw it, or perhaps I'm just making it up, but there

| shouldn't be any reason why a 'low level' routine (maybe this would be a

| good use for C) can't load a .dbf table (mimicking header and stuff).

| Imagine loading tables with the speed of, say, PKZIP?

|

| Bill

|

|

| -----Original Message-----

| From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

| Sent: Monday, August 30, 1999 1:20 AM

| To: Multiple recipients of ProFox

| Subject: RE: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

| processing

|

|

|

| Thanks for the input, Bill:

|

| Since the data would probably require a couple of steps' worth of

massaging

| I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

| there (I'm not sure, not having seen the whole project) unless I learn

that

| low-level file functions are significantly faster. Unless they want to

| change from FPD, I don't get any .DBC capabilities, either, so a middle

| step of massaging seems likely anyway.

|

| Russ Scott

| russsct@ix.netcom.com

|

|

|

|

You can change a FPD table to a VFP format table as simply as ADD TABLE x

-Anders

----- Original Message -----

From: Russ Scott <russsct@ix.netcom.com>

To: Multiple recipients of ProFox <profox@leafe.com>

Sent: mÃ

’ndag den 30 augusti 1999 07:19

Subject: RE: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

processing

|

| Thanks for the input, Bill:

|

| Since the data would probably require a couple of steps' worth of

massaging

| I was planning on a blanket APPEND FROM ... TYPE SDF then continuing from

| there (I'm not sure, not having seen the whole project) unless I learn

that

| low-level file functions are significantly faster. Unless they want to

| change from FPD, I don't get any .DBC capabilities, either, so a middle

| step of massaging seems likely anyway.

|

| Russ Scott

| russsct@ix.netcom.com

|

| ====================================

| At 10:17 PM 8/29/99 -0400, you wrote:

| >Russ,

| >

| >This is a matter of great concern to me too, and I posted a question and

got

| >some interesting responses not too long ago on the subject. I have a

monthly

| >job that imports a few million records spread across almost a thousand

| >tables (ZIP4 tables for the entire US). Because the data has to be

massaged

| >on the way into tables, I originally setup the import process to (low

level

| >I/O) read the ASCII fixed fields table, do the work and then APPEND BLANK

to

| >add a record to the database.

| >

| >I sped up the process a little by APPENDing blank records in 1,000 record

| >"blocks" and then replacing the blank fields of each until the 1,000

empty

| >records was filled and repeated the process again.

| >

| >Working this way, the import takes something like 2 days on a P133

machine.

| >A faster machine probably would help, but relegating the job to a 'spare'

| >machine that just chugs away seems to be the least bothersome. One thing

I

| >did do is design the process so multiple machines could participate in a

| >time crunch, and that does work very well when needed.

| >

| >Suggestions I received for speeding up the process included using APPEND

| >TYPE SDF, which would bring in each table en masse, but that would have

to

| >be followed with another step to do the massaging needed to prepare the

| >tables for use, since UDFs can't be used on individual fields with APPEND

| >TYPE SDF (but that would only partially address the data massaging

| >requirements anyway). Another suggestion, by Anders, that seems to have a

| >lot of merit, but I didn't try it yet, is to use APPEND TYPE SDF but with

| >rules in the target DBC (has to be VFP and target has to be DBC, tho) ...

| >I'm anxious to try this, but can't get to it yet.

| >

| >In general, processing each input record and then adding it to a table

using

| >APPEND BLANK is very slow compared to a bulk-add using APPEND TYPE SDF.

| >

| >As for FPD versus VFP in terms of speed, I know some people think that

FPD

| >if faster, but my experience so far with VFP shows that VFP (6, anyway,

with

| >SP3 if that means anything) it's actually faster than equivalent FPD

apps. I

| >attribute the difference to better memory management in VFP, but that's

only

| >my gut feeling (wish I had better measurement tools; I like the Profiler,

| >but can't find a way to track I/O utilization). I have not yet tested the

| >bulk-import under VFP, the tests I'm referring to are different, but they

| >are long running processor and I/O intensive jobs.

| >

| >Bill

| >

| >

| >-----Original Message-----

| >From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Russ Scott

| >Sent: Sunday, August 29, 1999 9:57 PM

| >To: Multiple recipients of ProFox

| >Subject: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

| >processing

| >

| >

| >

| >Hey, all:

| >

| >I have a chance to investigate a data conversion job. This would involve

| >importing from a flat file into FP tables, formatting/ massaging it, then

| >outputting it to another set of flat files. The company uses FPD (not

sure

| >which version) instead of VFP because "FPD is faster". To give you an

idea

| >of the size of the jobs involved, they take from ~1 hour to multiple

days.

| >

| >This causes me to ask some questions:

| >

| >Would FPD really be faster for data conversion if you compared VFP under

| >Win9x/NT to FPD running on the same machine booted to DOS only? Does the

| >version of DOS make a significant difference? Does anyone have any hard

| >data, educated guesses or gut feelings about this?

| >

| >Are there considerations besides speed (reliablility for example) to take

| >into consideration?

| >

| >On a related note, has anyone clocked APPEND FROM... and COPY TO...

against

| >low-level file commands?

| >

| >I don't have to talk to these people for a couple of days and was curious

| >if anyone else had looked into these issues.

| >

| >Any ideas?

| >

| >

| >Thanks:

| >

| >Russ Scott

| >russsct@ix.netcom.com

| >

| >

| >

| >

| >

| >

| >

| >

|

|

|

Anders, at one point, I did try INSERT INTO, having read it was faster, but

for my operation, it didn't yield much of a difference. I didn't benchmark

it, but certainly didn't see the hours gain I expected. That was in FPD,

maybe different in VFP. Maybe when I revisit that code, I'll re-enable the

commented out instructions to try it under VFP, just to see.

Bill

-----Original Message-----

From: profox@leafe.com [mailto:profox@leafe.com]On Behalf Of Anders

Altberg

Sent: Monday, August 30, 1999 6:41 AM

To: Multiple recipients of ProFox

Subject: Re: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

processing

INSERT IN TO is faster, fewer locks for one thing, and INSERT .. FROM ARRAY

would probably be faster still.

-Anders

----- Original Message -----

From: Bill Arnold <bill@wjarnold.com>

To: Multiple recipients of ProFox <profox@leafe.com>

Sent: mÃ

’ndag den 30 augusti 1999 08:27

Subject: RE: FoxPro DOS v. Visual Foxpro : Speed of data conversion /

processing

Russ,

I have compare the low level file read and insert into <tablename> from

memvar against an append from (on a network) and was amazed at the

results. Low level was faster even including some data massaging not

possible in the append. I posted those results to the FoxShare list some

time ago.

Joe Yoder

jayent@a1usa.net

On Sunday, August 29, 1999 9:56 PM, russsct@ix.netcom.com (Russ Scott)

wrote:

>

>Hey, all:

>

>I have a chance to investigate a data conversion job. This would involve

>importing from a flat file into FP tables, formatting/ massaging it, then

>outputting it to another set of flat files. The company uses FPD (not sure

>which version) instead of VFP because "FPD is faster". To give you an idea

>of the size of the jobs involved, they take from ~1 hour to multiple days.

>

>This causes me to ask some questions:

>

>Would FPD really be faster for data conversion if you compared VFP under

>Win9x/NT to FPD running on the same machine booted to DOS only? Does the

>version of DOS make a significant difference? Does anyone have any hard

>data, educated guesses or gut feelings about this?

>

>Are there considerations besides speed (reliablility for example) to take

>into consideration?

>

>On a related note, has anyone clocked APPEND FROM... and COPY TO... against

>low-level file commands?

>

>I don't have to talk to these people for a couple of days and was curious

>if anyone else had looked into these issues.

>

>Any ideas?

>

>

>Thanks:

>

>Russ Scott

>russsct@ix.netcom.com