Author: MB Software Solutions, LLC
Posted: 2011-08-09 12:18:50 Link
I've got an inherited app that processes flat files--some big ones
too---and I'm looking for areas to clear out memory because after I
process one file, it then chokes on the ALINES line for subsequent runs
says "There is not enough memory to complete this operation."
Here's the line: IF ALINES(aChunks,FILETOSTR(This.InputFile)) > 0
Wondering if Ed Leafe's SetMemory would be good here? Other ideas for
clearing out memory? My first line of attack is to do a CLOSE DATA ALL
before processing, as this doesn't use input tables, but again processes
flat files.
tia,
--Mike
--
Mike Babcock, MCP
MB Software Solutions, LLC
President, Chief Software Architect
http://mbsoftwaresolutions.com
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/4E415DEA.3050705@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: Tracy Pearson
Posted: 2011-08-09 12:23:10 Link
MB Software Solutions, LLC wrote on 2011-08-09:
> I've got an inherited app that processes flat files--some big ones
> too---and I'm looking for areas to clear out memory because after I
> process one file, it then chokes on the ALINES line for subsequent runs
> says "There is not enough memory to complete this operation."
>
> Here's the line: IF ALINES(aChunks,FILETOSTR(This.InputFile)) > 0
>
> Wondering if Ed Leafe's SetMemory would be good here? Other ideas for
> clearing out memory? My first line of attack is to do a CLOSE DATA ALL
> before processing, as this doesn't use input tables, but again processes
> flat files.
>
> tia,
> --Mike
>
Mike,
Are you releasing the array by calling
RELEASE aChucks
Tracy Pearson
PowerChurch Software
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/001001cc56b0$a140e7c0$e3c2b740$@powerchurch.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: MB Software Solutions, LLC
Posted: 2011-08-09 12:28:14 Link
On 8/9/2011 12:23 PM, Tracy Pearson wrote:
> MB Software Solutions, LLC wrote on 2011-08-09:
>> I've got an inherited app that processes flat files--some big ones
>> too---and I'm looking for areas to clear out memory because after I
>> process one file, it then chokes on the ALINES line for subsequent runs
>> says "There is not enough memory to complete this operation."
>>
>> Here's the line: IF ALINES(aChunks,FILETOSTR(This.InputFile))> 0
>>
>> Wondering if Ed Leafe's SetMemory would be good here? Other ideas for
>> clearing out memory? My first line of attack is to do a CLOSE DATA ALL
>> before processing, as this doesn't use input tables, but again processes
>> flat files.
>>
>> tia,
>> --Mike
>>
>
> Mike,
>
> Are you releasing the array by calling
> RELEASE aChucks
Well, it was a locally defined variable, but I just added the explicit
RELEASE to see if that helps. Thanks for the idea!
--
Mike Babcock, MCP
MB Software Solutions, LLC
President, Chief Software Architect
http://mbsoftwaresolutions.com
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/4E41601E.4070502@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: Jarvis, Matthew
Posted: 2011-08-09 12:32:23 Link
>
> I've got an inherited app that processes flat files--some big ones
> too---and I'm looking for areas to clear out memory because after I
> process one file, it then chokes on the ALINES line for subsequent
runs
> says "There is not enough memory to complete this operation."
You say you inherited the app - did it process files this size before?
If so, what's changed?
Or did you go in there and 'fix' something.... <g>
Thanks,
Matthew Jarvis || Business Systems Analyst
IT Department
McKenzie-Willamette Medical Center
1460 G Street, Springfield, OR 97477 || Ph: 541-744-6092 || Fax:
541-744-6145
--------------------------------------------------------------------------
Disclaimer: This electronic message may contain information that is
Proprietary, Confidential, or legally privileged or protected. It
is intended only for the use of the individual(s) and entity named
in the message. If you are not an intended recipient of this
message, please notify the sender immediately and delete the
material from your computer. Do not deliver, distribute or copy
this message and do not disclose its contents or take any action in
reliance on the information it contains.
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/69F310C05DD83C48A84BA3769CE1ECF8057C9D7A@TNTRIEXEVS02.triadhospitals.net
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: MB Software Solutions, LLC
Posted: 2011-08-09 13:11:40 Link
On 8/9/2011 12:32 PM, Jarvis, Matthew wrote:
>>
>> I've got an inherited app that processes flat files--some big ones
>> too---and I'm looking for areas to clear out memory because after I
>> process one file, it then chokes on the ALINES line for subsequent
> runs
>> says "There is not enough memory to complete this operation."
>
>
> You say you inherited the app - did it process files this size before?
>
> If so, what's changed?
>
> Or did you go in there and 'fix' something....<g>
lol...well, I don't think my mods would have caused this (...famous
developer's words!)
I'm not sure it ever processed files this large. Here's my latest idea:
Instead of reading the file completely into an array, I process it
line by line with LLFF. Think that's a worthy idea?
--
Mike Babcock, MCP
MB Software Solutions, LLC
President, Chief Software Architect
http://mbsoftwaresolutions.com
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/4E416A4C.105@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: Allen
Posted: 2011-08-09 13:16:51 Link
I had something like this before, turned out to be a continual while loop
that would not exit.
Al
-----Original Message-----
From: profoxtech-bounces@leafe.com [mailto:profoxtech-bounces@leafe.com] On
Behalf Of MB Software
I've got an inherited app that processes flat files--some big ones
too---and I'm looking for areas to clear out memory because after I
process one file, it then chokes on the ALINES line for subsequent runs
says "There is not enough memory to complete this operation."
Here's the line: IF ALINES(aChunks,FILETOSTR(This.InputFile)) > 0
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/00b301cc56b8$215a9ee0$640fdca0$@com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: John Weller
Posted: 2011-08-09 13:23:56 Link
I think that is a much better idea! You have more control and it is only
reading a line at a time so there will not be any memory issues (famous last
words :-) )
John Weller
01380 723235
07976 393631
> I'm not sure it ever processed files this large. Here's my latest idea:
> Instead of reading the file completely into an array, I process it
> line by line with LLFF. Think that's a worthy idea?
>
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/01d801cc56b9$1f4c3bd0$5de4b370$@co.uk
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: MB Software Solutions, LLC
Posted: 2011-08-09 13:38:03 Link
On 8/9/2011 1:23 PM, John Weller wrote:
> I think that is a much better idea! You have more control and it is only
> reading a line at a time so there will not be any memory issues (famous last
> words :-) )
The last time I wrote any LLFF code, it was in BASIC and/or PASCAL over
15 years ago! lol
Looks like I'll use FOPEN to get the handle and FREAD to read each line.
How do I determine the number of lines (without using ALINES)...would
be it the number of bytes returned from FREAD divided by the record
length? Also, would I use FSEEK to increment along the file inside a
loop like so?
liRecLength = 550
liHandle = fopen("myfile.txt")
do while not feof(liHandle)
lcLine = fread(liHandle,liRecLength)
* now do whatever processing on the lcLine variable
enddo
Does that look about right? tia!
--
Mike Babcock, MCP
MB Software Solutions, LLC
President, Chief Software Architect
http://mbsoftwaresolutions.com
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/4E41707B.7030701@mbsoftwaresolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: Jarvis, Matthew
Posted: 2011-08-09 13:47:12 Link
>
> The last time I wrote any LLFF code, it was in BASIC and/or PASCAL
over
> 15 years ago! lol
>
Is speed a big issue with this processing?
Maybe another work around might be to import into a table and process it
that way instead...
I hear VFP does tables pretty well... <g>
Thanks,
Matthew Jarvis || Business Systems Analyst
IT Department
McKenzie-Willamette Medical Center
1460 G Street, Springfield, OR 97477 || Ph: 541-744-6092 || Fax:
541-744-6145
--------------------------------------------------------------------------
Disclaimer: This electronic message may contain information that is
Proprietary, Confidential, or legally privileged or protected. It
is intended only for the use of the individual(s) and entity named
in the message. If you are not an intended recipient of this
message, please notify the sender immediately and delete the
material from your computer. Do not deliver, distribute or copy
this message and do not disclose its contents or take any action in
reliance on the information it contains.
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/69F310C05DD83C48A84BA3769CE1ECF8057C9E54@TNTRIEXEVS02.triadhospitals.net
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Author: Charlie Coleman
Posted: 2011-08-09 13:54:59 Link
At 11:32 AM 8/9/2011 -0500, Jarvis, Matthew wrote:
> >
> > I've got an inherited app that processes flat files--some big ones
> > too---and I'm looking for areas to clear out memory because after I
> > process one file, it then chokes on the ALINES line for subsequent
>runs
> > says "There is not enough memory to complete this operation."
...
Shot in the dark.
I think 65,000 array elements may still be a max. So if the file has more
than 65,000 lines.....
-Charlie
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/5.2.0.9.2.20110809124954.031294f0@incoming.verizon.net
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.