Prepare Your qPCR Manifest File

After you have finished preparing your data archive and metadata archive, you have to complete the third and final part of your submission: the manifest file.
The manifest file is the "glue" that links together all of your metadata and data. It also provides some important, additional information required to process your submission.

Your manifest file name will have the same prefix as your other files (data archive, metadata file) and will end in "_qPCR.manifest.json".
For example, if my data archive was named "samples_qPCR_data.zip", then my manifest file would be named "samples_qPCR.manifest.json".
As you work on your manifest file, make sure that you save regularly so you don't lose your progress!

Step 1. Download Template Manifest File

First, you will want to download a template of the manifest file.
You can find that template here.
You will complete your manifest file by filling in values between the quotation marks for each property.

Below, you can see what the template looks like:

 1 {
 2   "studyName": "",
 3   "userLogin": "",
 4   "md5CheckSum": "",
 5   "runMetadataFileName": "",
 6   "submissionMetadataFileName": "",
 7   "studyMetadataFileName": "",
 8   "experimentMetadataFileName": "",
 9   "biosampleMetadataFileName": "",
10   "donorMetadataFileName": "",
11   "qPCRTargetsMetadataFileName": "",
12   "settings":
13   {
14     "analysisName": "" 
15   }
16 }

Step 2. Open Your Manifest File

Next, you will need to open your manifest file in your favorite text editor.
You can find some recommendations below:

  • In Windows: Notepad++ or Wordpad (with "word wrap" turned off)
  • In Linux/Unix: gedit
  • In Mac OSX: "TextEdit" program
  • Command Line: You can also always use the terminal to edit files (vim, nano, etc.).

Step 3. Compute the MD5 Checksum of your Data Archive

  • NOTE: You only need to compute the MD5 checksum of your data archive if you are submitting a data archive (it's an optional file!).
  • You already know most of the information for your manifest file, but you'll need to compute the MD5 checksum of your data archive before you proceed.
  • Every file has an MD5 checksum associated with it. This checksum is based on the exact contents of the file, so two different files will basically never have the same MD5 checksum.
  • By computing the MD5 checksum of your version of the data archive and then providing that checksum to us, you give us a way of checking that the file transfer completed successfully.
  • When processing your files, we compute our own MD5 checksum of your data archive and compare it to the checksum that you gave us.
    If the checksums don't match, that means that the entire file did not transfer properly to us (or that you supplied the wrong checksum).
  • To compute the MD5 checksum on Linux/Unix/Mac for a given file, open up a terminal and type "md5sum [fileName]",
    where [fileName] is a path to your file. The md5sum will be displayed in the terminal, and you can just copy / paste it into the appropriate field.
  • cd /home/myHomeDir/myDataDir
md5sum samples_qPCR_data.tar.gz
  • If you're using Windows or are uncomfortable with using the terminal, there are a number of different stand-alone programs that will help you
    compute the MD5 checksum for a given file. You can see some examples here and here.
  • IMPORTANT NOTE: If you edit any files in your data archive, you will have to recompute your MD5 checksum
    before submitting your files for processing (because the contents of the data archive have changed).

Step 4. Fill Out the Top Section of Your Manifest

The top section of your manifest contains information that applies to all samples in your submission.
Below, we'll go through each property and tell you how to fill them all out.

  • studyName: This is the name of your study. Name your study something which captures the overall "feel" of the submission.
    • EXAMPLE: Since I want to compare CSF versus serum samples for Parkinson's patients, I wrote "CSF vs. Serum Parkinson's June 2017".
  • userLogin: This is your Genboree user login.
    • EXAMPLE: I wrote "william_thistle" because that's the name I use to log in to Genboree.
  • md5CheckSum: This is the MD5 checksum of the data archive (not the metadata archive and not the manifest file). We give directions above on how to compute the MD5 checksum.
    • EXAMPLE: I wrotee "b9355772f35516837a06666f7c56afdd" because I got that value when I computed the MD5 checksum of my data archive.
    • REMINDER: The MD5 checksum is only required if you submit a data archive (it's optional!).
  • runMetadataFileName: This is the file name of your Runs metadata file.
    • EXAMPLE: I wrote "testRun.metadata.tsv" because that's the name of my Runs metadata file.
  • submissionMetadataFileName: This is the file name of your Submissions metadata file.
    • EXAMPLE: I wrote "testSubmissions.metadata.tsv" because that's the name of my Submissions metadata file.
  • studyMetadataFileName: This is the file name of your Studies metadata file.
    • EXAMPLE: I wrote "testStudies.metadata.tsv" because that's the name of my Studies metadata file.
  • experimentMetadataFileName: This is the file name of your Experiments metadata file.
    • EXAMPLE: I wrote "testExperiments.metadata.tsv" because that's the name of my Experiments metadata file.
  • donorMetadataFileName: This is the file name of your Donors metadata file.
    • EXAMPLE: I wrote "testDonors.metadata.tsv" because that's the name of my Donors metadata file.
  • biosampleMetadataFileName: This is the file name of your Biosamples metadata file.
    • EXAMPLE: I wrote "testBiosamples.metadata.tsv" because that's the name of my Biosamples metadata file.
  • qPCRTargetsMetadataFileName: This is the file name of your qPCR Targets metadata file.
    • EXAMPLE: I wrote "testqPCRTargets.metadata.tsv" because that's the name of my qPCR Targets metadata file.

So far, our template should look something like this:

 1 {
 2   "studyName": "CSF vs. Serum Parkinson's June 2017",
 3   "userLogin": "william_thistle",
 4   "md5CheckSum": "b9355772f35516837a06666f7c56afdd",
 5   "runMetadataFileName": "testRun.metadata.tsv",
 6   "submissionMetadataFileName": "testSubmissions.metadata.tsv",
 7   "studyMetadataFileName": "testStudies.metadata.tsv",
 8   "experimentMetadataFileName": "testExperiments.metadata.tsv",
 9   "biosampleMetadataFileName": "testBiosamples.metadata.tsv",
10   "donorMetadataFileName": "testDonors.metadata.tsv",
11   "qPCRTargetsMetadataFileName": "testqPCRTargets.metadata.tsv",
12   "settings":
13   { 
14     "analysisName": "" 
15   }
16 }

Step 5. Fill Out the Settings Section of Your Manifest

The "settings" section at the bottom of the manifest file provides some ability to customize how your submission is processed.
Below, we'll go over the different options and describe briefly what they do.

Setting Name Description and Possible Values
analysisName analysis name - used for naming job-specific folder on Genboree and for naming certain files in your results. Default uses timestamp to indicate when the job was submitted (this is a good idea!).
genomeVersion genome version of your output database / your data. Default is hg19. Other supported genomes are hg38 and mm10.

IMPORTANT NOTES

You need to specify an analysisName in your manifest file, as this setting provides valuable information for organizing your submission.
We recommend that you structure your analysisName in the following way:
  • First, put your PI ID followed by -. This is the first letter of your PI's first name, followed by the first four letters of your PI's last name, followed by a 1.
    For example, my PI ID is AMILO1, since my PI is Aleksandar MILOsavljevic.
  • Second, put some kind of label for your submission followed by -.
    For example, I might put "Serum_vs_Plasma_Controls" if I was comparing healthy controls in serum and plasma.
  • Third, put the date of your submission in the format YYYY-MM-DD.
    For example, I would put 2017-06-01 if I was submitting my files on June 1, 2017.
  • Our final analysisName would look like the following: AMILO1-Serum_vs_Plasma_Controls-2017-06-01.

Make sure that you specify "genomeVersion": "mm10" or "genomeVersion": "hg38" if your samples use one of these alternative reference genomes (hg19 is the default).

Now, our (completed) manifest file looks like the following:

 1 {
 2   "studyName": "CSF vs. Serum Parkinson's June 2017",
 3   "userLogin": "william_thistle",
 4   "md5CheckSum": "b9355772f35516837a06666f7c56afdd",
 5   "runMetadataFileName": "testRun.metadata.tsv",
 6   "submissionMetadataFileName": "testSubmissions.metadata.tsv",
 7   "studyMetadataFileName": "testStudies.metadata.tsv",
 8   "experimentMetadataFileName": "testExperiments.metadata.tsv",
 9   "biosampleMetadataFileName": "testBiosamples.metadata.tsv",
10   "donorMetadataFileName": "testDonors.metadata.tsv",
11   "qPCRTargetsMetadataFileName": "testqPCRTargets.metadata.tsv",
12   "settings":
13   { 
14     "analysisName": "AMILO1-Serum_vs_Plasma_Controls-2017-06-01" 
15   }
16 }

If you remove or add a setting, make sure that your terms are still separated sensibly by commas.
For example, if I added another property like genomeVersion after analysisName, I would put a comma after analysisName (but no comma after genomeVersion).

You can download this example manifest file here.

Step 7. Validate and Save Your Manifest File

After you've finished working on your manifest file, you should make sure that the file is formatted correctly by using a JSON validator like JSONLint.
Simply copy-paste your manifest content into the text box and then click "Validate" to see if there are any errors in your manifest file.
If there are any errors, use the error messages provided by the JSON validator to fix your manifest file.
You're now done with creating your manifest file! Save it a final time and you're ready to upload your submission for processing.

Summary

  1. Download template manifest file
  2. Open your manifest file
  3. Compute the MD5 checksum of your data archive (not your manifest file, not your metadata archive) if necessary
  4. Fill out the top section of your manifest
  5. Fill out the settings section of your manifest
  6. Validate and save your manifest file

Also available in: HTML TXT