How-To: Use File Exchange¶
Obtaining File Boxes
At the moment, file boxes are only available to members of core facilities (e.g., genomics, bioinformatics, or metabolomics) for exchanging files for their collaboration partners. Currently, HPC users cannot use the file box mechanism on their own.
BIH HPC IT provides a file exchange server to be used by the BIH core facilities and their users.
The server is located in the BIH DMZ in Buch.
Users authenticate using their Charite/BIH (
user@CHARITE) or MDC accounts (
File exchange is organized using "file boxes", directories created on the server to which selected users are granted access.
Access control list maintenance is done with audit-trails ("Revisionssicherheit") and the file access itself is also logged to comply with data protection standards.
Access from Charite Network
Access from the Charite network (IP ranges
10.x.x.x) must connect through the Charite proxy (
Depending on the client software that you are using, you might have to configure the proxy.
File Box Management¶
File boxes are created by the core facilities (e.g., the genomics facilities at Charite and MDC). The facility members also organize the access control. Please talk to your core facility contact on file exchange.
External users must obtain a Charite or MDC account first. Account creation is handled by the core facilities that the external user is a customer of.
Generally, you will be given a URL to your file box similar to
The files are served over an encrypted connection using WebDAV (which uses HTTPS).
The following describes how to access the files in the box from different platforms.
We describe how to access the files on the command line using the
The program is preinstalled on the BIH (and the MDC cluster) and you should be able to just install it with
yum install lftp on CentOS/Red Hat or
apt-get install lftp on Ubuntu/Debian.
lftp, you have to add some configuration first:
# cat >>~/.lftprc <<"EOF" set ssl:verify-certificate no set ftp:ssl-force yes EOF
In case that you want to access the files using a graphical user interface, search Google for "WebDAV" and your operating system or desktop environment. File browsers such as Nautilus and Thunar have built-in WebDAV support.
First, log into the machine that has
The login nodes of the BIH cluster do not have it installed but all compute and file transfer nodes have it.
Go to the data download location.
host:~$ mkdir -p ~/scratch/download_dir host:~$ cd ~/scratch/download_dir
You can open the connection using
open -u <user>@<DOMAIN> https://file-exchange.bihealth.org/<file-box-id>/ (NB: there is a trailing slash) where
<user>is your user name, e.g.,
<file-box-id>the file box ID from the URL provided to you.
When prompted, use your normal Charite/MDC password to login.
host:download_dir$ lftp lftp :~> open -u holtgrem@CHARITE https://file-exchange.bihealth.org/c62910b3-c1ba-49a5-81a6-a68f1f15aef6 Password: cd ok, cwd=/c62910b3-c1ba-49a5-81a6-a68f1f15aef6 lftp holtgrem@CHARITE@file-exchange.bihealth.org:/c62910b3-c1ba-49a5-81a6-a68f1f15aef6>
You can find a full reference of
lftp on the lftp man page.
You could also use
help COMMAND on the lftp prompt.
For example, to look at the files of the server for a bit...
lftp holtgrem@CHARITE@file-exchange.bihealth.org:/c62910b3-c1ba-49a5-81a6-a68f1f15aef6> ls drwxr-xr-x -- / drwxr-xr-x -- dir -rw-r--r-- -- file1 lftp holtgrem@CHARITE@file-exchange.bihealth.org:/c62910b3-c1ba-49a5-81a6-a68f1f15aef6> find ./ ./dir/ ./dir/file2 ./file1
To download all data use
mirror, e.g. with
-P 4 to use four download threads.
lftp holtgrem@CHARITE@file-exchange.bihealth.org:/c62910b3-c1ba-49a5-81a6-a68f1f15aef6> mirror . Total: 2 directories, 3 files, 0 symlinks New: 3 files, 0 symlinks lftp holtgrem@CHARITE@file-exchange.bihealth.org:/c62910b3-c1ba-49a5-81a6-a68f1f15aef6> exit host:download_dir$ tree . ├── dir │ └── file2 ├── file1 └── file.txt 1 directory, 3 files
A common error to see is
mirror: Fatal error: gnutls_record_recv: The TLS connection was non-properly terminated..
You can just ignore this.
To upload data, you can use
mirror -R . which is essentially the "reverse" of the mirror command.
lftp holtgrem@CHARITE@file-exchange.bihealth.org:/c62910b3-c1ba-49a5-81a6-a68f1f15aef6> mirror -R mirror: Fatal error: gnutls_record_recv: The TLS connection was non-properly terminated. mirror: Fatal error: gnutls_record_recv: The TLS connection was non-properly terminated. mirror: Fatal error: gnutls_record_recv: The TLS connection was non-properly terminated. Total: 2 directories, 3 files, 0 symlinks Modified: 3 files, 0 symlinks 4 errors detected
We recommend to use WinSCP for file transfer.
- Pre-packaged WinSCP on Charite Workstations. Charite IT has packaged WinSCP and you can install it using Matrix24 Empirum on Windows 10 using these instructions in the Charite intranet.
- Installing WinSCP yourself. You can obtain it from the WinSCP Download Page. A "portable" version is available that comes as a ZIP archive that you just have to extract without an installer.
After starting WinSCP, you will see a window titled
Just paste the URL (e.g.,
https://file-exchange.bihealth.org/c62910b3-c1ba-49a5-81a6-a68f1f15aef6/) of the file box into the
Host name entry field.
In this case, the fields
File protocol etc. will be filled automatically.
Next, enter your user name as
user@MDC-BERLIN (the capitalization of the part behind the
@ is important).
The window should now look similar to the one below.
Proxy Configuration on Charite Network
If you are on the Charite network then you have to configure the proxy. Otherwise, you have to skip this step.
Advanced and a window titled
Advanced Site Settings will pop up.
Connection / Proxy in the left side.
HTTP for the
proxy.charite.de as the
Proxy host name and set the
Port number to
The window should nwo look as below.
OK to apply the proxy settings.
You can now transfer files between the file exchange server and your local computer using drag and drop between WinSCP and your local Windows File Explorer.
Alternatively, you can use the two-panel view of WinSCP to transfer files as described here.
Proxy Configuration on Charite Network
If you are on the Charite network then you must have configured the proxy appropriately. Otherwise, you have to skip this step.
You can find them in your
System Preference in the
Network section, in the
Advanced tab of your network (e.g.,
If you want to use a graphical interface then we recommend the usage of Cyberduck.
After starting the program, click
Open Connection on the top left, then select
WebDAV (HTTPS) and fill out the form as in the following way.
Paste the file box URL into the server field and use your login name (
user@MDC-BERLIN) with your usual password.
If you need to perform access through a graphical user interface on your Mac, please contact firstname.lastname@example.org for support.
The file exchange server has the
fail2ban software installed and configured (Charite, MDC, and BIH IPs are excluded from this).
If you are entering your user/password incorrectly for more than 5 times in 10 minutes then your machine will be banned for one hour. This means someone else that has the same IP address from the side of the file exchange server can get you blocked. This can happen if you are in the same home or university network with NAT or if you are behind a proxy. In this case you get a "connection refused" error. In this case, try again in one hour.