Processing Your Files

After you upload your three files (manifest file, metadata archive, data archive) to our FTP server, we will begin processing your files automatically.
  • Processing your files can take anywhere from a few hours to a few days (depending on the size of your submission).
  • You will receive a variety of emails while we're processing your files, and an "ERCC Final Processing" email will indicate that your processing is complete.

Troubleshooting a Failed Submission

  • If your files continue to sit in your inbox after a few hours, make sure that you correctly followed the required format for your files:
    • Each file must have the same prefix
    • Your data file must end in _data and must be a .tar.gz or .zip file
    • Your metadata file must end in _metadata and must be a .tar.gz or .zip file
    • Your manifest file must be a .manifest.json file
    • EXAMPLE: test.manifest.json,,
  • It is likely that your initial submission will fail for some reason (invalid metadata, some issue with your manifest file, etc.). This is totally normal!
  • There are a couple of steps you can take if you receive a failure e-mail:
    • Read the error message at the bottom of the e-mail and see if it is informative.
      • Often times, if there is an error in one or more of your metadata files, the error e-mail will tell you exactly why the pipeline failed.
    • If the error message isn't helpful or you're still perplexed, feel free to send an e-mail to William to get some help.
  • We check each part of your submission (manifest file / metadata archive / data archive) in order.
    If any of your submitted files are unchecked or pass inspection, they will be moved back to your lab's submission inbox.
    • For example, if there are errors in your manifest file, we will automatically move your metadata archive and data archive back to your submission inbox.
  • This makes the submission process easier, since you don't have to keep uploading your files or moving them around on the FTP server.
  • You'll be able to see a list of unchecked / working files in the WORKING FILES section in the email you receive.
  • If one of your files fails processing and you still want to download it (maybe to make edits), you can find it on the FTP server in your lab's "working" subdirectory.
    • The full path of the file will be given in the BROKEN FILES section in the email you receive.
  • Finally, even if your submission is generally processed successfully, it is possible that some of your individual samples may fail processing.
    This could be an issue with the FASTQ files themselves, or it could be an issue with exceRpt's handling of the FASTQ files.
    At any rate, if any of your samples fail processing, you will receive a failure email related to that sample, and that failure email will likely be informative
    on why the sample failed processing. If you have any questions, feel free to contact William to inquire.

Locating Your Finished Submission on the exRNA Atlas

  • After your files have been successfully processed, it will take some time for the associated results to appear on the Atlas.
  • We deploy updates to the Atlas in phases, and there may be other fixes that need to take place before your results will appear.
  • By default, your results will be made available in the consortium-only Atlas.
  • After the standard embargo period of 1 year has expired, those results will be made available on the public Atlas.
    • You can read more about the embargo period and related topics on the Data Access Policy page.
    • If you would like to move your results to the public Atlas sooner, you can email William.

Also available in: HTML TXT