How 'bout that? An on-topic question!
I need to upload/download small (10-100 records) batches of data between a
web application (likely Ruby, maybe PHP) and a VFP application. Data is
straight-forward, 3 simple tables of data: customers names and addresses,
order details and some numeric constant-ishes. The idea is that the web app
assigns a batch of customers, the sales guy downloads the data to their
portable VFP apps, load it up, and visit the customer sites, where internet
of any form (cellular, wireless, wired, modem) is not available. Data is
gathered/updated, and later (when internet available) tabular data loaded
back up to the mothership, overwriting existing records.
VFP is a bridging technology; I want the import/export to be generic enough
that other devices (iPad, Android, WinTab) might be used in future versions
with minimal recoding. My thoughts are that the two compatible formats
these days are JSON and XML.
Questions:
1. Anyone doing something like this? What techniques/technologies are you
using?
I see Sweet Potato (yay, Craig!) and Google Code both have JSON
translators. Experiences?
VFP can do CursorToXML() and the reverse. Problems?
2. Problems with translating character data between web app (typically
UTF-8) and FoxPro (ASCII/ANSI Windows code pages)? Customer data will
likely be Euro-American English (accents, tildes, umlauts, etc.) suitable
for US Postal delivery, so not Persian, Cyrilic or Asian character sets.
I'm pretty sure CursorToXML should handle encoding back-and-forth pretty
well.
--
Ted Roche
Ted Roche & Associates, LLC
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CACW6n4sAYkVVb8UN5PNwBHRRqizAnt0xmQpxJn7=m8Ssq_XmrA@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
I found that using ActiveVFP on the server and VFP on the client was pretty slick using the CURSORTO XML and XMLTOCURSOR. Basically VFP web services awesome!
-----Original Message-----
From: ProFox [mailto:profox-bounces@leafe.com] On Behalf Of Ted Roche
Sent: Sunday, February 23, 2014 11:25 AM
To: profox@leafe.com
Subject: VFP - web app interchange format?
How 'bout that? An on-topic question!
I need to upload/download small (10-100 records) batches of data between a web application (likely Ruby, maybe PHP) and a VFP application. Data is straight-forward, 3 simple tables of data: customers names and addresses, order details and some numeric constant-ishes. The idea is that the web app assigns a batch of customers, the sales guy downloads the data to their portable VFP apps, load it up, and visit the customer sites, where internet of any form (cellular, wireless, wired, modem) is not available. Data is gathered/updated, and later (when internet available) tabular data loaded back up to the mothership, overwriting existing records.
VFP is a bridging technology; I want the import/export to be generic enough that other devices (iPad, Android, WinTab) might be used in future versions with minimal recoding. My thoughts are that the two compatible formats these days are JSON and XML.
Questions:
1. Anyone doing something like this? What techniques/technologies are you using?
I see Sweet Potato (yay, Craig!) and Google Code both have JSON translators. Experiences?
VFP can do CursorToXML() and the reverse. Problems?
2. Problems with translating character data between web app (typically
UTF-8) and FoxPro (ASCII/ANSI Windows code pages)? Customer data will likely be Euro-American English (accents, tildes, umlauts, etc.) suitable for US Postal delivery, so not Persian, Cyrilic or Asian character sets.
I'm pretty sure CursorToXML should handle encoding back-and-forth pretty well.
--
Ted Roche
Ted Roche & Associates, LLC
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
[excessive quoting removed by server]
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/41fa58bff3534218bedad3255074069c@BLUPR05MB007.namprd05.prod.outlook.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
I would advocate using JSON over XML. JSON is less wordy (important over
the wire) and is essentially serialised JavaScript. If you are working
with web services on other platforms in the future I would say that
JavaScript will be the lingua franca.
In terms of VFP and JSON, I have used Craig's library in a production
environment. That project involved a Visual FoxPro EXE running as a
Windows service, which serialised DBF data and sent it to a web service.
It worked 95% of the time but (IIRC) I kept hitting edge cases that
would cause it to fail. As a result I ended up just rolling my own JSON
and calling the 'sanitisation' routines in Craig's library to encode it
properly for the wire. Ultimately I redid the whole thing using C# and
JSON.Net. If I had to use VFP again for a similar thing I would probably
look at the JSON functionality in West Wind Software's West Wind Web
Connection. As great as Craig's stuff is he seems to be otherwise
engaged these last few years, and I imagine you would get better help
and support from a paid solution.
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/1393187582.8765.86819873.214C40D7@webmail.messagingengine.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Tom and Alan:
Thanks for the replies. Good answers.
The web side with definitely be a Linux machine, likely Postgres, and
probably Ruby on Rails. An app like that speaks JSON and XML just fine, and
can even do the encoding/decoding if it makes more sense to do that
server-side. So, if we pick XML for the VFP interface, it's fairly simple
to support a second JSON interface later on. The VFP client will be the
first pass, first-years budget, but the client would really like to get
into smaller smart devices (tablets, phones) if that makes sense (cost
reliability, ease of development on native platform, etc.)
So, the VFP app is really a single-function app: collect specific order
data. (and the auxilliary functions of up- and down-load.) It collects some
specific metrics, runs a custom pricing module, spits out an estimate and
closes the sale.
I'm tempted by the simplicity of CursorToXML() and XMLToCursor() since
their built-in, know about encoding, and can produce valid XML. The VFP can
read the files in, and generate them out with no 3rd party dependency. The
web app would provide a secure up-and down-load.
It will be wordier than JSON since attributes or properties will be
repeated for each row, but the quantity of data is limited to to a day's
work, so that's likely never more than 40 or 50 rows, so I don't think the
bandwidth consumed should be a big concern.
On Sun, Feb 23, 2014 at 3:33 PM, Alan Bourke <alanpbourke@fastmail.fm>wrote:
> I would advocate using JSON over XML.
--
Ted Roche
Ted Roche & Associates, LLC
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CACW6n4vamh3Nk3d8Pzx1TX-Q08uEOHcjOrYNRedicBYsqU47Ng@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
On 2/23/14, 8:25 AM, Ted Roche wrote:
> How 'bout that? An on-topic question!
>
> I need to upload/download small (10-100 records) batches of data between a
> web application (likely Ruby, maybe PHP) and a VFP application.
On the one hand, JSON is great for interchange because of its compaction
ability (if you do away with readability), its readability (if you do
away with compactness), its general acceptance, and its flexibility. On
the other hand, perhaps CSV should be considered as well, for ease
importing the data into VFP. I don't know, there's probably some VFP
json library available now, but VFP's import/export from/to CSV I
remember being pretty excellent, easy, and fast.
> Data is
> straight-forward, 3 simple tables of data: customers names and addresses,
> order details and some numeric constant-ishes. The idea is that the web app
> assigns a batch of customers, the sales guy downloads the data to their
> portable VFP apps, load it up, and visit the customer sites, where internet
> of any form (cellular, wireless, wired, modem) is not available. Data is
> gathered/updated, and later (when internet available) tabular data loaded
> back up to the mothership, overwriting existing records.
Seems like you are also going to want some sort of merging smartness so
that one salesperson can't overwrite another's history just because he
added a phone number (extreme example).
> VFP is a bridging technology; I want the import/export to be generic enough
> that other devices (iPad, Android, WinTab) might be used in future versions
> with minimal recoding. My thoughts are that the two compatible formats
> these days are JSON and XML.
JSON or CSV. I wouldn't even consider XML for this. Client database
should be SQLite (connect with ODBC from VFP), perhaps even doing that
now to avoid headaches later, like with conflicting encodings (SQLite is
UTF-8 by default even on Windows, none of the legacy ansi mess).
> 1. Anyone doing something like this? What techniques/technologies are you
> using?
I have a python desktop app in the field that has a single-user mode.
Client database is SQLite with data merging with the mothership's MySQL
data using a custom web service that transfers SQL statements. Yeah, I
know. I did this in 2007, what can I say. Today, I'd use JSON as the
interchange format. It works very well.
If you can, use UUID primary keys, so that your salesguy can create new
records that have no chance of colliding with records on the mothership.
> VFP can do CursorToXML() and the reverse. Problems?
Yeah, the XML. ;)
> 2. Problems with translating character data between web app (typically
> UTF-8) and FoxPro (ASCII/ANSI Windows code pages)?
You WILL run into encoding/decoding issues if you don't address this up
front. Can VFP tables store in utf-8? If not, choose a database that
can, especially since it'll be a requirement later on (sqlite on devices).
> Customer data will
> likely be Euro-American English (accents, tildes, umlauts, etc.) suitable
> for US Postal delivery, so not Persian, Cyrilic or Asian character sets.
> I'm pretty sure CursorToXML should handle encoding back-and-forth pretty
> well.
I'm not sure about this, but you might be correct. I guess you could
find out. :)
Paul
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/530B9B02.6010602@mcnettware.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
A mention of IBM's Watson might be helpful because it uses JSON.
https://www.ibmdw.net/watson/docs/developing-watson-apis/using-question-answ
er-api-access-watson-capabilities/
Bill
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/001d01cf3306$e9368b40$bba3a1c0$@h2officesolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
On Mon, Feb 24, 2014 at 2:18 PM, Paul McNett <paul@mcnettware.com> wrote:
>
> On the one hand,... On the other hand,...
>
>
Yeah, me, too. Just typing in (and simplifying) the problem to form the
question let me look at my biases.
Minimal data, a days batch, shouldn't be a bandwidth issue, so less
concerned with wordiness than ease of debugging. Can always compact later
if needed.
CSV is okay if used right (commas, quotes, etc.) but an amazing number of
products don't understand the simple rules.
> Seems like you are also going to want some sort of merging smartness so
> that one salesperson can't overwrite another's history just because he
> added a phone number (extreme example).
This is a really simple case with pessimistic locking business rules: the
mothership "checks out" a record to the field guy, so no updates while it
is checked out. SImple "Last Save Wins" rules for data merging.
> JSON or CSV. I wouldn't even consider XML for this. Client database should
> be SQLite (connect with ODBC from VFP), perhaps even doing that now to
> avoid headaches later, like with conflicting encodings (SQLite is UTF-8 by
> default even on Windows, none of the legacy ansi mess).
XML is just JSON with too many pointy-brackets :) CursorToXML and its
obverse are built-in and testing indicates support for two-way translation
of UTF8 to Windows ANSI (Emilè Cöté appears correct on both platforms) so
no futzing with the horrors of STRTRAN() or STRCONV() to try to get simple
accented characters.
> Client database is SQLite with data merging with the mothership's MySQL
> data using a custom web service that transfers SQL statements. Yeah, I
> know. I did this in 2007, what can I say.
SQL Server has a fancy-schmantzy replication scheme called 'log shipping'
which is essentially this, wrapped in a proprietary binary format. It's not
an awful solution. I have something similar for a backchannel update on a
publish-and-subscribe mechanism myself.
> Today, I'd use JSON as the interchange format. It works very well.
>
The issue I'm seeing is the couple of JSON libs I've found for VFP seem to
hand-code "foreign" character translations (they're all foreign, folks!) I
don't want the whole house of cards to fall apart the first time we run
into an Å.
> If you can, use UUID primary keys, so that your salesguy can create new
> records that have no chance of colliding with records on the mothership.
Good idea.
ATM, the barebones design is for the VFP app to generate export files to
disk, and then the operator manually upload later when docked. Since they
could upload last nights Netflix stream instead, we're looking at
quarantining and doing some sanity checks and possibly individual reviews
before actual import to production tables. Volume is low enough that this
is feasible, and we are replacing a huge about of manual keypunching from
pencil-filled forms, so this is still an order-of-magnitude time-saver. As
the app matures, I'm sure we'll automate this.
> VFP can do CursorToXML() and the reverse. Problems?
>
> Yeah, the XML. ;)
Yeah, now I have two problems :) But they're known problems, rather than
unknown unknowns :)
> I'm not sure about this, but you might be correct. I guess you could find
> out. :)
>
Yeah, all will need to be sanity checked. After all, in theory, theory and
practice are the same :)
--
Ted Roche
Ted Roche & Associates, LLC
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CACW6n4sNy9O_7E9T2Zzci9djhNTzUwS2HP4HOD6hHc2ZK5yb3A@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
On Wed, Feb 26, 2014 at 10:24 AM, Bill Arnold <
billjarnold@h2officesolutions.com> wrote:
> A mention of IBM's Watson might be helpful because it uses JSON.
>
Hi, Bill! Long time no chat...
--
Ted Roche
Ted Roche & Associates, LLC
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CACW6n4v-NOfVN5mqXb+qH+mES2M_RTQw-y9iXFT8sQbaRYCL=g@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Oops. Hit the send too fast!
Yes, JSON is very popular. And Unicode. I attended a Web Development Meetup
last night where the subject was Ember.js and the presenter gave a very
good overview in two fact-packed hours. There was a lot to absorb.
Just the buzzterms were dazzling. Ember is a spin-off of SproutCore and
uses Handlebars as its templating language. It is MVC although the views
are becoming less prominent with the introduction of HTML5 Web Components.
Next version is targeted at ECMAScript6. Ember is trying to layer on an
inheritance model more similar to Ruby than the Prototype model of
JavaScript.
Essentially, Ember provides a runtime for a full application written in
Javascript running in the browser, a killer solution for mobile devices.
Essentially, it has its own idea of a cursor engine (shades of VFP!) in the
browser with asynchronous and/or delayed updating to an internet server,
using Promises (http://promises-aplus.github.io/promises-spec/) to continue
onward and manage callbacks, while the data goes up to potentially yet
another MVC application written with the proper API and, sure enough,
passing JSON back and forth 'tween the two.
It was all a bit much.
On Wed, Feb 26, 2014 at 10:24 AM, Bill Arnold <
billjarnold@h2officesolutions.com> wrote:
> A mention of IBM's Watson might be helpful because it uses JSON.
>
>
> https://www.ibmdw.net/watson/docs/developing-watson-apis/using-question-answ
> er-api-access-watson-capabilities/
>
>
> Bill
>
>
[excessive quoting removed by server]
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/CACW6n4v_wq+ZpHn9TbXLO59grRMV4t8g7=TMMkGVuLyafMHuaw@mail.gmail.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Hi Ted,
> A mention of IBM's Watson might be helpful because it uses JSON.
>
Hi, Bill! Long time no chat...
-------------
I must apologize for not contributing while gaining so much from yourself
and the ProFox community. I am mindful of doing better. I've basically been
hunkered down polishing my product (for all of its dozen or so users, but
soon to be millions and billions :)
I'm particularly excited about the Watson project. Can't say I'm an expert
because I have yet to use it, but I have been devoting time to formulating
the questions I will give my customers along with the interface. I spend so
much time creating reports and charts (VFPx and FoxCharts rocks!) to help
them better manage their business, but it's clear that reports are tools
they use to get answers to questions that Watson may be a better approach
with. An example: "how many more jobs can I handle if I hire 1 additional
employee?", "Explain"
On another front, I've got Win 8.1 on an MS Surface tablet. Very happy with
VFP appearance and performance, but so far it's been of limited real value.
I expected it to be more useful.
Bill
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/001e01cf3311$385ca510$a915ef30$@h2officesolutions.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.