Using Remote (FTP) Storage for exceRpt

We have recently implemented a new feature that allows users to deposit their result files onto our FTP server, as opposed to our cluster.
  • Files are still accessible via Genboree Workbench, but they are also accessible via FTP client
  • Downloading via FTP client is more reliable for large files - you can even resume your downloads!

Below, we'll go step-by-step through the process of setting up your remote storage area on Genboree and then downloading your exceRpt result files.

Step 1: Creating Your Remote Storage Area

The first step in the process is creating a remote storage area in your Database of choice.
If you're unfamiliar with Genboree, you should first learn about Groups and then learn about creating Databases.
Note that your account comes with a default group named after your user login.

After you have created a Database, you should drag it from the Data Selector panel to the Output Targets panel.
Then, you can select the Create Remote Storage Area tool in the tool menu at the top of the screen:

When you click the Create Remote Storage Area button, a window like the following will appear:

There are two different settings you can change:

  1. Name of Remote Storage Area
  2. Remote Storage Type

You can name the remote storage area anything you want, so long as the name is unique among folders in the Files area of your Database.
This is because the remote storage area is represented as a folder in the Files area.

Under "Remote Storage Type", you can select the particular type of remote storage area that you want to create.
Currently, we only support the Genboree FTP server - thus, you can go ahead and keep the default option of "Genboree Virtual FTP".

After you click "Submit", you will receive notification that your remote storage area was successfully created (or an informative error message if something went wrong).
Your remote storage area will now be available in the Files area under the Database you chose:

You can use this tool to create as many remote storage areas as you like, so long as they all have different names.

Step 2: Submitting Your Samples to exceRpt

Now that you've created your remote storage area, the next step is to submit your files for processing through exceRpt.
This tutorial will provide guidance on submitting your samples.
In particular, you will need to select your newly created remote stage area in the Remote Storage Area menu under Advanced Options.
The default option for this menu is None Selected - you should choose the remote storage area where you want to store your exceRpt result files.

After you click "Submit", your files will be processed through exceRpt.
If any issue arises during processing, you will receive an email notifying you about the issue.
If you have additional questions about your submission, you can always email exRNA Team for help.

Step 3: Accessing Your Results

After we've finished processing your samples, you'll be able to access your results by both the Genboree Workbench and your FTP client.

Accessing Your Results via the Genboree Workbench

If you want to use the Genboree Workbench, accessing your samples is just like any other exceRpt job, with one small difference.
With normal exceRpt submissions, the base directory for your exceRpt pipeline runs can be found in the Files area.
However, if you use a remote storage area, the base directory for your exceRpt pipeline runs will be located in that remote storage folder.
You can see an example below:

Accessing Your Results via Your FTP Client

Before you can access your results, you will need an account on our FTP server.
You should email exRNA Team and ask her to create a personal FTP account for you so that you can log onto our FTP server.
Please note that creating a Genboree account does not create your FTP account as well.
You will always need to contact the exRNA Team in order to create your FTP account.

Please include the following information in your email to the exRNA Team:
  1. Your Genboree username. Ex: william_thistle
  2. Your Group name. Ex: william_thistle2_group
  3. The name of the database which contains the remote storage area.
  4. The name of the remote storage area you created following the above instructions so we know which folder to give FTP access. Ex: virtualFTP-Genboree
  5. The PI name. This is important if you will be doing data submissions. Ex: Dr. Milosavljevic

Note: If you would like multiple people to have download permissions for this FTP folder you can include the usernames of the other people here

After we have confirmed that your FTP account is created, you can log into our FTP server at ftps://ftps.genboree.org with your Genboree username and password.
When you log in, you should see a directory named genboree:genboree.org. You should then follow a series of nested directories that include your
Group name, Database name, and remote storage area name. This final directory is where you will be able to find your exceRpt result files.
If I wanted to navigate to the result files given in the example above, I would go to the following path:
  • /genboree:genboree.org/william_thistle2_group/Your_Database/virtualFTP-Genboree/exceRptPipeline_v3.3.0/
    • Note that "Your Database" has been escaped so it's now "Your_Database".

Please note that you must send an email to us for each new Database you want to use for remote storage.
For example, say you create a remote storage area in Database A and then ask us to expose that area to your FTP username.
We will do so, and then you will be able to see that area via your FTP client.
If you create more remote storage areas in the same Database (Database A), then you will also be able to see those areas without emailing us again.
However, if you create a new Database on Genboree (Database B) and then create a remote storage area in that Database,
you will need to email us again with the name of the new Database so we can expose it to your FTP username.

Also available in: HTML TXT