I know there's several rsync users on this forum. Do any of you
have any experience or thoughts on the following usage scenario:
~1000 user specific folders containing a random mix of files
- want to sync files with specific extensions ignoring other
files
- want to move (transfer then delete) files with a specific
extension
Would rsync be appropriate for this scenario or would a server
side script be a better option?
Thanks,
Malcolm
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/1277222971.21047.1381355999@webmail.messagingengine.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
On Jun 22, 2010, at 12:09 PM, Malcolm Greene wrote:
> I know there's several rsync users on this forum. Do any of you
> have any experience or thoughts on the following usage scenario:
>
> ~1000 user specific folders containing a random mix of files
> - want to sync files with specific extensions ignoring other
> files
You could use either --include=PATTERN or --include-from=FILE to limit the files that rsync will consider. If you have more than 1 or 2 extensions, puth them in a file and use the second version.
> - want to move (transfer then delete) files with a specific
> extension
Use the --remove-source-files option.
> Would rsync be appropriate for this scenario or would a server
> side script be a better option?
Use rsync. The time spent learning the ins and outs of the tool (the man page for rsync is priceless) will be much, much less than what you would spend writing something to replace it.
-- Ed Leafe
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/E37D8607-084B-44DA-8B47-545119937056@leafe.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
I use rsync and I have several posts here. For use from on a XP computer to update one folder (dropbox) from some other
folders on the same computer/local network it works great; better than anything I have tried.
To copy from Windows to an offsite Linux server (Webfaction) it works great also, however, I still have not figured a
way to avoid having to type in the password manually. I am sure it is something simple like where to put & name the
private key on the windows computer, but I have several hours of research invested with no result. Webfaction assured me
I have the public key in the correct file in the correct folder on their server.
plink works great to copy files from one folder to another on the Linux server from a batch file running on windows. So
I update a single location on the Linux server from windows with rsync (very slow sFTP speed). then I use plink to copy
the files to the correct folders on the Linux server (very quick)
----- Original Message -----
From: "Ed Leafe" <ed@leafe.com>
To: "ProFox Email List" <profox@leafe.com>
Sent: Tuesday, June 22, 2010 9:26 AM
Subject: Re: [NF] Rsync usage scenario?
On Jun 22, 2010, at 12:09 PM, Malcolm Greene wrote:
> I know there's several rsync users on this forum. Do any of you
> have any experience or thoughts on the following usage scenario:
>
> ~1000 user specific folders containing a random mix of files
> - want to sync files with specific extensions ignoring other
> files
You could use either --include=PATTERN or --include-from=FILE to limit the files that rsync will consider. If you have
more than 1 or 2 extensions, puth them in a file and use the second version.
> - want to move (transfer then delete) files with a specific
> extension
Use the --remove-source-files option.
> Would rsync be appropriate for this scenario or would a server
> side script be a better option?
Use rsync. The time spent learning the ins and outs of the tool (the man page for rsync is priceless) will be much, much
less than what you would spend writing something to replace it.
-- Ed Leafe
[excessive quoting removed by server]
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/149501cb122a$99f5a490$7a00a8c0@w2k3s02
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Ed and Ken,
Thanks for your advice! It sounds like I have some reading to do ("the
man page for rsync is priceless").
Malcolm
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/1277231575.12374.1381379761@webmail.messagingengine.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
> "the man page for rsync is priceless"
Here's a link for anyone following the recent rsync threads.
http://www.samba.org/ftp/rsync/rsync.html
Malcolm
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/1277234135.19439.1381386231@webmail.messagingengine.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
On 6/22/10 9:09 AM, Malcolm Greene wrote:
> I know there's several rsync users on this forum. Do any of you
> have any experience or thoughts on the following usage scenario:
>
> ~1000 user specific folders containing a random mix of files
> - want to sync files with specific extensions ignoring other
> files
> - want to move (transfer then delete) files with a specific
> extension
>
> Would rsync be appropriate for this scenario or would a server
> side script be a better option?
I believe strait rsync can handle all that. Although, I'm a fan of wrapping things,
especially things with lots of flags to set, in a script.
Paul
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/4C214FA5.3040301@ulmcnett.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
On 6/22/10 9:47 AM, kamcginnis@gmail.com wrote:
> To copy from Windows to an offsite Linux server (Webfaction) it works great also, however, I still have not figured a
> way to avoid having to type in the password manually. I am sure it is something simple like where to put& name the
> private key on the windows computer, but I have several hours of research invested with no result. Webfaction assured me
> I have the public key in the correct file in the correct folder on their server.
Are you using Cygwin on the Windows end? If not, you should be.
Paul
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/4C214FFB.7000602@ulmcnett.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Hi Paul,
Yes, I am using cygwin.
----- Original Message -----
From: "Paul McNett" <p@ulmcnett.com>
To: <profox@leafe.com>
Sent: Tuesday, June 22, 2010 5:06 PM
Subject: Re: [NF] Rsync usage scenario?
On 6/22/10 9:47 AM, kamcginnis@gmail.com wrote:
> To copy from Windows to an offsite Linux server (Webfaction) it works great also, however, I still have not figured a
> way to avoid having to type in the password manually. I am sure it is something simple like where to put& name the
> private key on the windows computer, but I have several hours of research invested with no result. Webfaction assured
me
> I have the public key in the correct file in the correct folder on their server.
Are you using Cygwin on the Windows end? If not, you should be.
Paul
[excessive quoting removed by server]
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/177f01cb128c$74efb8e0$7a00a8c0@w2k3s02
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
On 6/22/10 9:27 PM, kamcginnis@gmail.com wrote:
> Yes, I am using cygwin.
Ok, the key difference on Windows using Cygwin versus *nix machines is that there's
no /home directory. /home is underneath whereever cygwin is installed, usually
c:\cygwin\home. Here's what I did on my Vista machine, in Windows Terminal, to enable
passwordless logins to my server:
ssh-keygen -t dsa
<press enter through every prompt>
<make a note of the directory where it tells you the key is installed,
for me it was /home/Administrator/.ssh/id_dsa.pub>
c:
cd c:\cygwin\Administrator\.ssh
scp id_dsa.pub root@paulmcnett.com:
<enter password>
ssh root@paulmcnett.com
<enter password>
cat id_dsa.pub >> .ssh/authorized_keys2
exit
ssh root@paulmcnett.com
<no password prompt>
HTH
If your client machine is ever compromised, first thing to do is go to the server and
remove the entry in .ssh/authorized_keys2 corresponding to the client, so the thief
can't log on to your server for free.
Paul
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/4C221C6B.6090302@ulmcnett.com
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.
Thank you Paul. I have it working now. It was there in front of me all the time. I just needed someone to tell me where
the keys were put automatically by ssh-keygen
----- Original Message -----
From: "Paul McNett" <p@ulmcnett.com>
To: <profox@leafe.com>
Sent: Wednesday, June 23, 2010 7:38 AM
Subject: Re: [NF] Rsync usage scenario?
On 6/22/10 9:27 PM, kamcginnis@gmail.com wrote:
> Yes, I am using cygwin.
Ok, the key difference on Windows using Cygwin versus *nix machines is that there's
no /home directory. /home is underneath whereever cygwin is installed, usually
c:\cygwin\home. Here's what I did on my Vista machine, in Windows Terminal, to enable
passwordless logins to my server:
ssh-keygen -t dsa
<press enter through every prompt>
<make a note of the directory where it tells you the key is installed,
for me it was /home/Administrator/.ssh/id_dsa.pub>
c:
cd c:\cygwin\Administrator\.ssh
scp id_dsa.pub root@paulmcnett.com:
<enter password>
ssh root@paulmcnett.com
<enter password>
cat id_dsa.pub >> .ssh/authorized_keys2
exit
ssh root@paulmcnett.com
<no password prompt>
HTH
If your client machine is ever compromised, first thing to do is go to the server and
remove the entry in .ssh/authorized_keys2 corresponding to the client, so the thief
can't log on to your server for free.
Paul
[excessive quoting removed by server]
_______________________________________________
Post Messages to: ProFox@leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/profox
OT-free version of this list: http://leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/195a01cb12e8$45ac8d40$7a00a8c0@w2k3s02
** All postings, unless explicitly stated otherwise, are the opinions of the author, and do not constitute legal or medical advice. This statement is added to the messages for those lawyers who are too stupid to see the obvious.