Index
2014-02-23 10:25 – Ted Roche : VFP - web app interchange format?
2014-02-23 10:28 – Tom Slayton : RE: VFP - web app interchange format?
2014-02-23 14:33 – Alan Bourke : Re: VFP - web app interchange format?
2014-02-23 16:33 – Ted Roche : Re: VFP - web app interchange format?
2014-02-24 13:18 – Paul McNett : Re: VFP - web app interchange format?
2014-02-26 09:24 – Bill Arnold : RE: VFP - web app interchange format?
2014-02-26 09:41 – Ted Roche : Re: VFP - web app interchange format?
2014-02-26 10:04 – Ted Roche : Re: VFP - web app interchange format?
2014-02-26 10:15 – Ted Roche : Re: VFP - web app interchange format?
2014-02-26 10:38 – Bill Arnold : RE: VFP - web app interchange format?
Back to top
VFP - web app interchange format?

Author: Ted Roche

Posted: 2014-02-23 10:25:09   Link

How 'bout that? An on-topic question!

I need to upload/download small (10-100 records) batches of data between a

web application (likely Ruby, maybe PHP) and a VFP application. Data is

straight-forward, 3 simple tables of data: customers names and addresses,

order details and some numeric constant-ishes. The idea is that the web app

assigns a batch of customers, the sales guy downloads the data to their

portable VFP apps, load it up, and visit the customer sites, where internet

of any form (cellular, wireless, wired, modem) is not available. Data is

gathered/updated, and later (when internet available) tabular data loaded

back up to the mothership, overwriting existing records.

VFP is a bridging technology; I want the import/export to be generic enough

that other devices (iPad, Android, WinTab) might be used in future versions

with minimal recoding. My thoughts are that the two compatible formats

these days are JSON and XML.

Questions:

1. Anyone doing something like this? What techniques/technologies are you

using?

I see Sweet Potato (yay, Craig!) and Google Code both have JSON

translators. Experiences?

VFP can do CursorToXML() and the reverse. Problems?

2. Problems with translating character data between web app (typically

UTF-8) and FoxPro (ASCII/ANSI Windows code pages)? Customer data will

likely be Euro-American English (accents, tildes, umlauts, etc.) suitable

for US Postal delivery, so not Persian, Cyrilic or Asian character sets.

I'm pretty sure CursorToXML should handle encoding back-and-forth pretty

well.

--

Ted Roche

Ted Roche & Associates, LLC

http://www.tedroche.com

--- StripMime Report -- processed MIME parts ---

multipart/alternative

text/plain (text body -- kept)

text/html

---

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/CACW6n4sAYkVVb8UN5PNwBHRRqizAnt0xmQpxJn7=m8Ssq_XmrA@mail.gmail.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Ted Roche
Back to top
RE: VFP - web app interchange format?

Author: Tom Slayton

Posted: 2014-02-23 10:28:25   Link

I found that using ActiveVFP on the server and VFP on the client was pretty slick using the CURSORTO XML and XMLTOCURSOR. Basically VFP web services awesome!

-----Original Message-----

From: ProFox [mailto:profox-bounces@leafe.com] On Behalf Of Ted Roche

Sent: Sunday, February 23, 2014 11:25 AM

To: profox@leafe.com

Subject: VFP - web app interchange format?

How 'bout that? An on-topic question!

I need to upload/download small (10-100 records) batches of data between a web application (likely Ruby, maybe PHP) and a VFP application. Data is straight-forward, 3 simple tables of data: customers names and addresses, order details and some numeric constant-ishes. The idea is that the web app assigns a batch of customers, the sales guy downloads the data to their portable VFP apps, load it up, and visit the customer sites, where internet of any form (cellular, wireless, wired, modem) is not available. Data is gathered/updated, and later (when internet available) tabular data loaded back up to the mothership, overwriting existing records.

VFP is a bridging technology; I want the import/export to be generic enough that other devices (iPad, Android, WinTab) might be used in future versions with minimal recoding. My thoughts are that the two compatible formats these days are JSON and XML.

Questions:

1. Anyone doing something like this? What techniques/technologies are you using?

I see Sweet Potato (yay, Craig!) and Google Code both have JSON translators. Experiences?

VFP can do CursorToXML() and the reverse. Problems?

2. Problems with translating character data between web app (typically

UTF-8) and FoxPro (ASCII/ANSI Windows code pages)? Customer data will likely be Euro-American English (accents, tildes, umlauts, etc.) suitable for US Postal delivery, so not Persian, Cyrilic or Asian character sets.

I'm pretty sure CursorToXML should handle encoding back-and-forth pretty well.

--

Ted Roche

Ted Roche & Associates, LLC

http://www.tedroche.com

--- StripMime Report -- processed MIME parts ---

multipart/alternative

text/plain (text body -- kept)

text/html

---

[excessive quoting removed by server]

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/41fa58bff3534218bedad3255074069c@BLUPR05MB007.namprd05.prod.outlook.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Tom Slayton
Back to top
Re: VFP - web app interchange format?

Author: Alan Bourke

Posted: 2014-02-23 14:33:02   Link

I would advocate using JSON over XML. JSON is less wordy (important over

the wire) and is essentially serialised JavaScript. If you are working

with web services on other platforms in the future I would say that

JavaScript will be the lingua franca.

In terms of VFP and JSON, I have used Craig's library in a production

environment. That project involved a Visual FoxPro EXE running as a

Windows service, which serialised DBF data and sent it to a web service.

It worked 95% of the time but (IIRC) I kept hitting edge cases that

would cause it to fail. As a result I ended up just rolling my own JSON

and calling the 'sanitisation' routines in Craig's library to encode it

properly for the wire. Ultimately I redid the whole thing using C# and

JSON.Net. If I had to use VFP again for a similar thing I would probably

look at the JSON functionality in West Wind Software's West Wind Web

Connection. As great as Craig's stuff is he seems to be otherwise

engaged these last few years, and I imagine you would get better help

and support from a paid solution.

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/1393187582.8765.86819873.214C40D7@webmail.messagingengine.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Alan Bourke
Back to top
Re: VFP - web app interchange format?

Author: Ted Roche

Posted: 2014-02-23 16:33:22   Link

Tom and Alan:

Thanks for the replies. Good answers.

The web side with definitely be a Linux machine, likely Postgres, and

probably Ruby on Rails. An app like that speaks JSON and XML just fine, and

can even do the encoding/decoding if it makes more sense to do that

server-side. So, if we pick XML for the VFP interface, it's fairly simple

to support a second JSON interface later on. The VFP client will be the

first pass, first-years budget, but the client would really like to get

into smaller smart devices (tablets, phones) if that makes sense (cost

reliability, ease of development on native platform, etc.)

So, the VFP app is really a single-function app: collect specific order

data. (and the auxilliary functions of up- and down-load.) It collects some

specific metrics, runs a custom pricing module, spits out an estimate and

closes the sale.

I'm tempted by the simplicity of CursorToXML() and XMLToCursor() since

their built-in, know about encoding, and can produce valid XML. The VFP can

read the files in, and generate them out with no 3rd party dependency. The

web app would provide a secure up-and down-load.

It will be wordier than JSON since attributes or properties will be

repeated for each row, but the quantity of data is limited to to a day's

work, so that's likely never more than 40 or 50 rows, so I don't think the

bandwidth consumed should be a big concern.

On Sun, Feb 23, 2014 at 3:33 PM, Alan Bourke <alanpbourke@fastmail.fm>wrote:

> I would advocate using JSON over XML.

--

Ted Roche

Ted Roche & Associates, LLC

http://www.tedroche.com

--- StripMime Report -- processed MIME parts ---

multipart/alternative

text/plain (text body -- kept)

text/html

---

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/CACW6n4vamh3Nk3d8Pzx1TX-Q08uEOHcjOrYNRedicBYsqU47Ng@mail.gmail.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Ted Roche
Back to top
Re: VFP - web app interchange format?

Author: Paul McNett

Posted: 2014-02-24 13:18:26   Link

On 2/23/14, 8:25 AM, Ted Roche wrote:

> How 'bout that? An on-topic question!

>

> I need to upload/download small (10-100 records) batches of data between a

> web application (likely Ruby, maybe PHP) and a VFP application.

On the one hand, JSON is great for interchange because of its compaction

ability (if you do away with readability), its readability (if you do

away with compactness), its general acceptance, and its flexibility. On

the other hand, perhaps CSV should be considered as well, for ease

importing the data into VFP. I don't know, there's probably some VFP

json library available now, but VFP's import/export from/to CSV I

remember being pretty excellent, easy, and fast.

> Data is

> straight-forward, 3 simple tables of data: customers names and addresses,

> order details and some numeric constant-ishes. The idea is that the web app

> assigns a batch of customers, the sales guy downloads the data to their

> portable VFP apps, load it up, and visit the customer sites, where internet

> of any form (cellular, wireless, wired, modem) is not available. Data is

> gathered/updated, and later (when internet available) tabular data loaded

> back up to the mothership, overwriting existing records.

Seems like you are also going to want some sort of merging smartness so

that one salesperson can't overwrite another's history just because he

added a phone number (extreme example).

> VFP is a bridging technology; I want the import/export to be generic enough

> that other devices (iPad, Android, WinTab) might be used in future versions

> with minimal recoding. My thoughts are that the two compatible formats

> these days are JSON and XML.

JSON or CSV. I wouldn't even consider XML for this. Client database

should be SQLite (connect with ODBC from VFP), perhaps even doing that

now to avoid headaches later, like with conflicting encodings (SQLite is

UTF-8 by default even on Windows, none of the legacy ansi mess).

> 1. Anyone doing something like this? What techniques/technologies are you

> using?

I have a python desktop app in the field that has a single-user mode.

Client database is SQLite with data merging with the mothership's MySQL

data using a custom web service that transfers SQL statements. Yeah, I

know. I did this in 2007, what can I say. Today, I'd use JSON as the

interchange format. It works very well.

If you can, use UUID primary keys, so that your salesguy can create new

records that have no chance of colliding with records on the mothership.

> VFP can do CursorToXML() and the reverse. Problems?

Yeah, the XML. ;)

> 2. Problems with translating character data between web app (typically

> UTF-8) and FoxPro (ASCII/ANSI Windows code pages)?

You WILL run into encoding/decoding issues if you don't address this up

front. Can VFP tables store in utf-8? If not, choose a database that

can, especially since it'll be a requirement later on (sqlite on devices).

> Customer data will

> likely be Euro-American English (accents, tildes, umlauts, etc.) suitable

> for US Postal delivery, so not Persian, Cyrilic or Asian character sets.

> I'm pretty sure CursorToXML should handle encoding back-and-forth pretty

> well.

I'm not sure about this, but you might be correct. I guess you could

find out. :)

Paul

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/530B9B02.6010602@mcnettware.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Paul McNett
Back to top
RE: VFP - web app interchange format?

Author: Bill Arnold

Posted: 2014-02-26 09:24:59   Link

A mention of IBM's Watson might be helpful because it uses JSON.

https://www.ibmdw.net/watson/docs/developing-watson-apis/using-question-answ

er-api-access-watson-capabilities/

Bill

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/001d01cf3306$e9368b40$bba3a1c0$@h2officesolutions.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Bill Arnold
Back to top
Re: VFP - web app interchange format?

Author: Ted Roche

Posted: 2014-02-26 09:41:38   Link

On Mon, Feb 24, 2014 at 2:18 PM, Paul McNett <paul@mcnettware.com> wrote:

>

> On the one hand,... On the other hand,...

>

>

Yeah, me, too. Just typing in (and simplifying) the problem to form the

question let me look at my biases.

Minimal data, a days batch, shouldn't be a bandwidth issue, so less

concerned with wordiness than ease of debugging. Can always compact later

if needed.

CSV is okay if used right (commas, quotes, etc.) but an amazing number of

products don't understand the simple rules.

> Seems like you are also going to want some sort of merging smartness so

> that one salesperson can't overwrite another's history just because he

> added a phone number (extreme example).

This is a really simple case with pessimistic locking business rules: the

mothership "checks out" a record to the field guy, so no updates while it

is checked out. SImple "Last Save Wins" rules for data merging.

> JSON or CSV. I wouldn't even consider XML for this. Client database should

> be SQLite (connect with ODBC from VFP), perhaps even doing that now to

> avoid headaches later, like with conflicting encodings (SQLite is UTF-8 by

> default even on Windows, none of the legacy ansi mess).

XML is just JSON with too many pointy-brackets :) CursorToXML and its

obverse are built-in and testing indicates support for two-way translation

of UTF8 to Windows ANSI (Emilè Cöté appears correct on both platforms) so

no futzing with the horrors of STRTRAN() or STRCONV() to try to get simple

accented characters.

> Client database is SQLite with data merging with the mothership's MySQL

> data using a custom web service that transfers SQL statements. Yeah, I

> know. I did this in 2007, what can I say.

SQL Server has a fancy-schmantzy replication scheme called 'log shipping'

which is essentially this, wrapped in a proprietary binary format. It's not

an awful solution. I have something similar for a backchannel update on a

publish-and-subscribe mechanism myself.

> Today, I'd use JSON as the interchange format. It works very well.

>

The issue I'm seeing is the couple of JSON libs I've found for VFP seem to

hand-code "foreign" character translations (they're all foreign, folks!) I

don't want the whole house of cards to fall apart the first time we run

into an Å.

> If you can, use UUID primary keys, so that your salesguy can create new

> records that have no chance of colliding with records on the mothership.

Good idea.

ATM, the barebones design is for the VFP app to generate export files to

disk, and then the operator manually upload later when docked. Since they

could upload last nights Netflix stream instead, we're looking at

quarantining and doing some sanity checks and possibly individual reviews

before actual import to production tables. Volume is low enough that this

is feasible, and we are replacing a huge about of manual keypunching from

pencil-filled forms, so this is still an order-of-magnitude time-saver. As

the app matures, I'm sure we'll automate this.

> VFP can do CursorToXML() and the reverse. Problems?

>

> Yeah, the XML. ;)

Yeah, now I have two problems :) But they're known problems, rather than

unknown unknowns :)

> I'm not sure about this, but you might be correct. I guess you could find

> out. :)

>

Yeah, all will need to be sanity checked. After all, in theory, theory and

practice are the same :)

--

Ted Roche

Ted Roche & Associates, LLC

http://www.tedroche.com

--- StripMime Report -- processed MIME parts ---

multipart/alternative

text/plain (text body -- kept)

text/html

---

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/CACW6n4sNy9O_7E9T2Zzci9djhNTzUwS2HP4HOD6hHc2ZK5yb3A@mail.gmail.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Ted Roche
Back to top
Re: VFP - web app interchange format?

Author: Ted Roche

Posted: 2014-02-26 10:04:18   Link

On Wed, Feb 26, 2014 at 10:24 AM, Bill Arnold <

billjarnold@h2officesolutions.com> wrote:

> A mention of IBM's Watson might be helpful because it uses JSON.

>

Hi, Bill! Long time no chat...

--

Ted Roche

Ted Roche & Associates, LLC

http://www.tedroche.com

--- StripMime Report -- processed MIME parts ---

multipart/alternative

text/plain (text body -- kept)

text/html

---

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/CACW6n4v-NOfVN5mqXb+qH+mES2M_RTQw-y9iXFT8sQbaRYCL=g@mail.gmail.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Ted Roche
Back to top
Re: VFP - web app interchange format?

Author: Ted Roche

Posted: 2014-02-26 10:15:46   Link

Oops. Hit the send too fast!

Yes, JSON is very popular. And Unicode. I attended a Web Development Meetup

last night where the subject was Ember.js and the presenter gave a very

good overview in two fact-packed hours. There was a lot to absorb.

Just the buzzterms were dazzling. Ember is a spin-off of SproutCore and

uses Handlebars as its templating language. It is MVC although the views

are becoming less prominent with the introduction of HTML5 Web Components.

Next version is targeted at ECMAScript6. Ember is trying to layer on an

inheritance model more similar to Ruby than the Prototype model of

JavaScript.

Essentially, Ember provides a runtime for a full application written in

Javascript running in the browser, a killer solution for mobile devices.

Essentially, it has its own idea of a cursor engine (shades of VFP!) in the

browser with asynchronous and/or delayed updating to an internet server,

using Promises (http://promises-aplus.github.io/promises-spec/) to continue

onward and manage callbacks, while the data goes up to potentially yet

another MVC application written with the proper API and, sure enough,

passing JSON back and forth 'tween the two.

It was all a bit much.

On Wed, Feb 26, 2014 at 10:24 AM, Bill Arnold <

billjarnold@h2officesolutions.com> wrote:

> A mention of IBM's Watson might be helpful because it uses JSON.

>

>

> https://www.ibmdw.net/watson/docs/developing-watson-apis/using-question-answ

> er-api-access-watson-capabilities/

>

>

> Bill

>

>

[excessive quoting removed by server]

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/CACW6n4v_wq+ZpHn9TbXLO59grRMV4t8g7=TMMkGVuLyafMHuaw@mail.gmail.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Ted Roche
Back to top
RE: VFP - web app interchange format?

Author: Bill Arnold

Posted: 2014-02-26 10:38:46   Link

Hi Ted,

> A mention of IBM's Watson might be helpful because it uses JSON.

>

Hi, Bill! Long time no chat...

-------------

I must apologize for not contributing while gaining so much from yourself

and the ProFox community. I am mindful of doing better. I've basically been

hunkered down polishing my product (for all of its dozen or so users, but

soon to be millions and billions :)

I'm particularly excited about the Watson project. Can't say I'm an expert

because I have yet to use it, but I have been devoting time to formulating

the questions I will give my customers along with the interface. I spend so

much time creating reports and charts (VFPx and FoxCharts rocks!) to help

them better manage their business, but it's clear that reports are tools

they use to get answers to questions that Watson may be a better approach

with. An example: "how many more jobs can I handle if I hire 1 additional

employee?", "Explain"

On another front, I've got Win 8.1 on an MS Surface tablet. Very happy with

VFP appearance and performance, but so far it's been of limited real value.

I expected it to be more useful.

Bill

_______________________________________________

Post Messages to: ProFox@leafe.com

Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox

OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech

Searchable Archive: http://leafe.com/archives/search/profox

This message: http://leafe.com/archives/byMID/profox/001e01cf3311$385ca510$a915ef30$@h2officesolutions.com

** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.

©2014 Bill Arnold