Difference between revisions of "File transfer between hosts"

From Storrs HPC Wiki
Jump to: navigation, search
(FileZilla)
 
(14 intermediate revisions by 7 users not shown)
Line 1: Line 1:
{{DISPLAYTITLE:Upload and download files from and to HORNET cluster }}
+
{{DISPLAYTITLE:Upload and download files from and to the HPC Cluster }}
  
This page provides instructions on how to copy files to and from the Hornet cluster
+
This page provides instructions on how to copy files to and from the HPC Cluster using Secure Copy (SCP), which transfers files between computers using the Secure Shell (SSH) protocol. In order to use any of these methods to transfer files, you must either be on the Storrs campus or using the UConn VPN. See [[HPC_Getting_Started#Off-campus_Access|this page]] for VPN instructions.
 +
== FileZilla ==
  
==SCP==
+
FileZilla is a graphical file transfer utility for all operating systems.
'''Secure Copy''' or '''SCP''' is a means of securely transferring computer files between a local host and a remote host or between two remote hosts.  It is based on the Secure Shell (SSH) protocol.
+
[https://sourceforge.net/projects/filezilla/ Download] and run Filezilla Client.
  
===How to SCP (Secure Copy) with FileZilla on Windows ===
+
# In the top menu, click on File > Site Manager.
You can easily transfer files from a server that has SSH (Secure Shell) running but not FTP (File Transfer Protocol) using Filezilla on Windows. When using Filezilla it will be called SFTP (Secure File Transfer Protocol) and will allow you to copy files from a server to your local Windows computer with ease. Follow the simple steps below to securely copy files from a server that is running SSH.
+
#: [[File:FileZilla_Site_Manager.png]]
 +
# Click on "New Site" and call it, say, "hpc". Fill in the values for "Host", "Protocol" and "Login" (with your NetID) as shown below. Then click "Connect".
 +
#: [[File:Filezilla_Add_Site.png]]
 +
# When Filezilla connects, you will see the remote file list.
 +
#: [[File:FileZilla_Site_Connected.png | 800px]]
 +
# To transfer files you can drag and drop between the left (local) and right (remote) computers.
 +
# The next time you start FileZilla, you can quickly login with the Site Manager button.
 +
#: [[File:Filezilla_How_To_Connect.png]]
  
'''1''' Open Filezilla: First you will need to launch Filezilla. If you do not have Filezilla you can obtain it for free from [http://sourceforge.net/projects/filezilla/files/FileZilla_Client/3.5.3/FileZilla_3.5.3_win32-setup.exe/download?accel_key=57%3A1347940439%3Ahttp%253A//filezilla-project.org/download.php%253Ftype%253Dclient%3A66265939%24b1747b4ba64c3304d405e7136b322b161c5fe6f1&click_id=7a2d435a-0144-11e2-adb4-0200ac1d1d8b&source=accel this link]. Then please install it on your system.
+
== SCP (Mac OS or GNU/Linux) ==
 +
To transfer large files between 2 computers, it is useful to directly transfer between those computers instead of copying to your local computer.
 +
One can use <code>scp</code> on the cluster to copy files.
 +
<code>scp</code> is a command line utility bundled with UNIX operating systems.
 +
For example, to upload file(s):
 +
scp <local_file> <NetID>@login.storrs.hpc.uconn.edu:/path/to/remote/destination/directory
 +
To download a directory:
 +
scp -r <NetID>@login.storrs.hpc.uconn.edu:/path/to/remote/source/directory . # The dot means current directory; -r means recursively copy everything in the directory
  
'''2''' Open Site Manager: Click on File in the top navigation menu and then select “Site Manager”:
+
== WinSCP (Windows only) ==
  
[[File:site_manager.png | 800px]]
+
[http://winscp.net/ WinSCP] is already installed on SkyBox machines.
 +
Use the same connection settings as FileZilla.
  
This will open Filezilla’s Site Manager, as shown below, where you can add, delete, and modify sites that you visit often.Now you can follow the operations in the figure below.
+
[[Category:Core]]
 
 
[[File:operation_on_site_manager.png]]
 
 
 
Now Filezilla will give you a response as below, it means you connect successfully. Then, you click "ok" button on the pop window.
 
 
 
[[File:response.png | 800px]]
 
 
 
Finally, Filezilla shows the information below. You can now upload or download files.
 
 
 
[[File:Final_result.png | 800px]]
 
 
 
'''3''' upload or download file
 
Assume you want to upload file "1.png" under file folder "C:\Users\boh10002\Desktop\" in your computer to the file folder "/home/boh10002" in the server, as shown below. Then you choose the file want to upload and right click your Mouse button. A window pop up, you choose the upload option and file begins uploading. You can also change the folder in your computer or server as shown in below.
 
 
 
[[File:upload_operation.png | 800px]]
 
 
 
It is similar for download operation.
 
 
 
=== Alternative SCP client on Windows ===
 
An alternative SCP client WinSCP could be downloaded from http://ninite.com/ or http://winscp.net/eng/download.php.
 
 
 
<iframe width="560" height="315" src="http://www.youtube.com/embed/AJdG5W7RZcc" frameborder="0" allowfullscreen></iframe>
 
 
 
===SCP under linux platform===
 
The '''SCP program''' is a software tool implementing the SCP protocol as a service daemon or client.  It is a program to perform secure copying.  The SCP server program is typically the same program as the SCP client.
 
 
 
Perhaps the most widely used SCP program is the command line <tt>scp</tt> program, which is provided in most SSH implementations. The <tt>scp</tt> program is the secure analog of the <tt>rcp</tt> command.  The <tt>scp</tt> program must be part of all SSH servers that want to provide SCP service, as <tt>scp</tt> functions as SCP server too.
 
 
 
Typically, a syntax of <tt>scp</tt> program is like the syntax of <tt>cp</tt>:
 
 
 
Copying file to Hornet cluster:
 
<tt>scp ''SourceFile'' ''user''@''hornet.engr.uconn.edu'':''directory''/''TargetFile''</tt>
 
 
 
Copying file from Hornet cluster:
 
<tt>scp ''user''@''hornet.engr.uconn.edu'':''directory''/''SourceFile'' ''TargetFile''</tt>
 
<tt>scp -r ''user''@''hornet.engr.uconn.edu'':''directory''/''SourceFolder'' ''TargetFolder''</tt>
 
 
 
Note that if the remote host uses a port other than the default of 22, you can specify it in the command. For example, copying a file from host:
 
<tt>scp -P 2222 ''user''@''host'':''directory''/''SourceFile'' ''TargetFile''</tt>
 

Latest revision as of 11:07, 18 November 2016


This page provides instructions on how to copy files to and from the HPC Cluster using Secure Copy (SCP), which transfers files between computers using the Secure Shell (SSH) protocol. In order to use any of these methods to transfer files, you must either be on the Storrs campus or using the UConn VPN. See this page for VPN instructions.

FileZilla

FileZilla is a graphical file transfer utility for all operating systems. Download and run Filezilla Client.

  1. In the top menu, click on File > Site Manager.
    FileZilla Site Manager.png
  2. Click on "New Site" and call it, say, "hpc". Fill in the values for "Host", "Protocol" and "Login" (with your NetID) as shown below. Then click "Connect".
    Filezilla Add Site.png
  3. When Filezilla connects, you will see the remote file list.
    FileZilla Site Connected.png
  4. To transfer files you can drag and drop between the left (local) and right (remote) computers.
  5. The next time you start FileZilla, you can quickly login with the Site Manager button.
    Filezilla How To Connect.png

SCP (Mac OS or GNU/Linux)

To transfer large files between 2 computers, it is useful to directly transfer between those computers instead of copying to your local computer. One can use scp on the cluster to copy files. scp is a command line utility bundled with UNIX operating systems. For example, to upload file(s):

scp <local_file> <NetID>@login.storrs.hpc.uconn.edu:/path/to/remote/destination/directory

To download a directory:

scp -r <NetID>@login.storrs.hpc.uconn.edu:/path/to/remote/source/directory . # The dot means current directory; -r means recursively copy everything in the directory

WinSCP (Windows only)

WinSCP is already installed on SkyBox machines. Use the same connection settings as FileZilla.