File transfer via SMB

From Storrs HPC Wiki
Jump to: navigation, search

What is SMB

SMB is a file transfer protocol supported by nearly every type of file storage device or operating system. You can use the SMB protocol on the HPC cluster via the smbclient command. By following the instructions below, you can transfer data from the HPC cluster to any device on campus that supports the SMB protocol, including the UITS Research Data service, a departmental NAS (Synology, etc.), an individual Windows desktop, and more.

Connecting to an SMB server

To access the SMB server, you need to use the following command:

$ smbclient -W DOMAIN -U NetID //server.uconn.edu/folder/path 
  • Replace DOMAIN with the appropriate domain for your department, usually "grove.ad.uconn.edu" (UITS) or "ad.engr.uconn.edu" (School of Engineering).
  • Replace NetID with your university NetID.
  • Replace server.uconn.edu with the network name of your file server.
  • Replace folder/path with the set of directories that correspond to your network share.

Example: UITS P and Q Drives

P Drive

UITS describes the P: drive as:

Personal (P:) Drives are available to all students, staff, and faculty who have a NetID to save files for access on any university computer. These drives allow for 50 Gigabytes(GB) maximum of university related files and is private for your individual use only.

These drives are accessible through the HPC cluster using smbclient.

smbclient -W grove.ad.uconn.edu -U NetID //guild.grove.ad.uconn.edu/Home
smb: \> ls

Replace NetID with your actual university NetID.

Q Drive

UITS describes the Q: drive as:

Specific groups and departments may have a Shared (Q:) Drive to distribute files between users (faculty, staff, and student employees may be granted access).

These drives are accessible through the HPC cluster using smbclient.

smbclient -W grove.ad.uconn.edu -U NetID //guild.grove.ad.uconn.edu/EFS
smb: \> ls

Replace NetID with your actual university NetID. You can then change directories to your departmental file share.

Example: UITS HNAS for School of Engineering

If you'd like to use smbclient interactively:

smbclient -W ad.engr.uconn.edu -U <user_name> //hnas.engr.uconn.edu/EngrGroupMixed/
Password: <Enter password here>
Domain=[ENGR_STUDENT] OS=[Hitachi NAS Platform 12.2.3753.08] Server=[Hitachi NAS Platform 12.2.3753.08]
smb: \> cd <share_name>

Where <user_name> is your School of Engineering user name, and <share_name> is the name of your UITS HNAS share.

Alternatively, if you're writing a script and want to automate your data transfers, here is an example:

smbclient -W ad.engr.uconn.edu -U <user_name> -D <share_name> -c "put <file_name>" //hnas.engr.uconn.edu/EngrGroupMixed "<password>"

Where <user_name> and <password> are your School of Engineering credentials, <share_name> is the name of your UITS HNAS share, and <file_name> is the name of the file to transfer.

Copying files

To copy the file from the local file systemt to the SMB server:

smb: \> put local_file remote_file

To copy the file from the SMB serverto the local file systemt:

smb: \> get remote_file local_file

Example: Copy a file from HNAS to HPC

$ smbclient -W ENGR_STUDENT -U EngrID //hnas.engr.uconn.edu/EngrGroupMixed
smb: \> cd myshare\
smb: \myshare\> ls
  .                                   D        0  Tue Oct  6 14:03:05 2015
  ..                                  D        0  Tue Oct  6 14:03:05 2015
  parallel_studio_xe_2015_update3.tgz      A 4156836825  Mon Apr 20 02:15:32 2015
  .DS_Store                          AH    12292  Fri Oct  9 15:04:33 2015
  iso                                 D        0  Tue Sep 29 10:21:44 2015
  Passwords.kdbx                      A     8462  Tue Oct  6 14:03:05 2015
  hpc                                 D        0  Tue Oct  6 10:55:40 2015
  .TemporaryItems                    DH        0  Wed Sep 23 13:44:36 2015 

                65535 blocks of size 1073741824. 61536 blocks available
smb: \myshare\> !ls
smb: \myshare\> !pwd
/shared/admin/RedHat
smb: \myshare\> get parallel_studio_xe_2015_update3.tgz
getting file \myshare\parallel_studio_xe_2015_update3.tgz of size 4156836825 as parallel_studio_xe_2015_update3.tgz (59640.2 KiloBytes/sec) (average 59640.2 KiloBytes/sec)
smb: \myshare\> !ls
parallel_studio_xe_2015_update3.tgz
smb: \myshare\> exit

Commands for smbclient

?/help [command]
If command is specified, the ? command will display a brief informative message about the specified command. If no command is specified, a list of available commands will be displayed.
! [shell command]
If shell command is specified, the ! command will execute a shell locally and run the specified shell command. If no command is specified, a local shell will be run.
cd <directory name>
If "directory name" is specified, the current working directory on the server will be changed to the directory specified. This operation will fail if for any reason the specified directory is inaccessible.
If no "directory name" is specified, the current working directory on the server will be reported.
del/rm <mask>
The client will request that the server attempt to delete all files matching mask from the current working directory on the server.
dir/ls <mask>
A list of the files matching mask in the current working directory on the server will be retrieved from the server and displayed.
du <filename>
Does a directory listing and then prints out the current disk usage and free space on a share.
exit/quit
Terminate the connection with the server and exit from the program.
get <remote file name> [local file name]
Copy the file called remote file name from the server to the machine running the client. If specified, name the local copy local file name. Note that all transfers in smbclient are binary. See also the lowercase command.
history
Displays the command history.
lcd [directory name]
If directory name is specified, the current working directory on the local machine will be changed to the directory specified. This operation will fail if for any reason the specified directory is inaccessible.
If no <cdoe>directory name</code> is specified, the name of the current working directory on the local machine will be reported.
lowercase
Toggle lowercasing of filenames for the get and mget commands.
When lowercasing is toggled ON, local filenames are converted to lowercase when using the get and mget commands. This is often useful when copying (say) MSDOS files from a server, because lowercase filenames are the norm on UNIX systems.
mask <mask>
This command allows the user to set up a mask which will be used during recursive operation of the mget and mput commands.
The masks specified to the mget and mput commands act as filters for directories rather than files when recursion is toggled ON.
The mask specified with the mask command is necessary to filter files within those directories. For example, if the mask specified in an mget command is "source*" and the mask specified with the mask command is "*.c" and recursion is toggled ON, the mget command will retrieve all files matching "*.c" in all directories below and including all directories matching "source*" in the current working directory.
Note that the value for mask defaults to blank (equivalent to "*") and remains so until the mask command is used to change it. It retains the most recently specified value indefinitely. To avoid unexpected results it would be wise to change the value of mask back to "*" after using the mget or mput commands.
md/mkdir <directory name>
Create a new directory on the server (user access privileges permitting) with the specified name.
mget <mask>
Copy all files matching mask from the server to the machine running the client.
Note that mask is interpreted differently during recursive operation and non-recursive operation - refer to the recurse and mask commands for more information. Note that all transfers in smbclient are binary. See also the lowercase command.
mput <mask>
Copy all files matching mask in the current working directory on the local machine to the current working directory on the server.
Note that mask is interpreted differently during recursive operation and non-recursive operation - refer to the recurse and mask commands for more information. Note that all transfers in smbclient are binary.
prompt ON/OFF
Toggle prompting for filenames during operation of the mget and mput commands.
When toggled ON, the user will be prompted to confirm the transfer of each file during these commands. When toggled OFF, all specified files will be transferred without prompting.
put <local file name> [remote file name]
Copy the file called local file name from the machine running the client to the server. If specified, name the remote copy remote file name. Note that all transfers in smbclient are binary. See also the lowercase command.
rd/rmdir <directory name>
Remove the specified directory (user access privileges permitting) from the server.

Reference

More command information is available at Smbclient.