Name tags for following complex histories

Author(s) orcid logoHelena Rasche avatar Helena Rasche
Reviewers Helena Rasche avatar Saskia Hiltemann avatar Bérénice Batut avatar Gildas Le Corguillé avatar Björn Grüning avatar
Overview
Creative Commons License: CC-BY Questions:
  • What are name tags or hash tags?

  • How can I use them to keep track of my data?

Objectives:
  • Learn how to set name tags

  • Learn how they are propagated

Time estimation: 20 minutes
Supporting Materials:
Published: Apr 8, 2019
Last modification: Nov 9, 2023
License: Tutorial Content is licensed under Creative Commons Attribution 4.0 International License. The GTN Framework is licensed under MIT
purl PURL: https://gxy.io/GTN:T00154
rating Rating: 4.2 (0 recent ratings, 9 all time)
version Revision: 8

Long ad hoc data analysis sessions in Galaxy can leave you with a multitude of datasets named things like “1724: Summarize taxa on data 1628 and data 1629” or “1853: Pick OTUs Combined: OTUs”. While these names are useful in smaller analysis sessions with fewer datasets, as the size of your history grows, so does the amount of time it takes to figure out where your datasets came from.

You have two options to deal with this:

  • Re-name datasets constantly with more useful titles
  • Use name tags once, on the inputs to your analysis

Name tags “propagate”, they will follow your datasets throughout their analysis. To more clearly see this, look at the simple workflow below. If the input datasets are tagged TB8_R1 and TB8_R2, these tags will propagate to the outputs of any tools run on these datasets. If a tool has multiple inputs with tags, the outputs will have all of the tags like in the bowtie2 tool.

A workflow with some tags overlaid.

It can turn your histories from a complicated mess into an easier to follow analysis, as each step is clearly labelled with the original input sample corresponding to each analysis:

Bad Good
History with untagged and poorly named datasets. History with tagged datasets showing how much easier it is.
How would you identify which input sample corresponds with which featureCounts output? With name tags applied before the start of analysis, each output is also tagged with the appropriate sample name
Agenda

In this tutorial, we will cover:

  1. Setting name tags manually
  2. Setting name tags with the Rule Builder
  3. Name tags in Action

Setting name tags manually

Hands-on: Set name tags manually
  1. Create a new history

    To create a new history simply click the new-history icon at the top of the history panel:

    UI for creating new history

  2. Using the Upload galaxy-upload tool select Paste/Fetch data and paste some data like “Hello, World”

  3. When this is uploaded, click on the title of the dataset in your history panel to expand it

  4. Add some name tags (hashtags) to your dataset:

    • #female
    • #male
    • #dmel6
    • #ecoli

    Datasets can be tagged. This simplifies the tracking of datasets across the Galaxy interface. Tags can contain any combination of letters or numbers but cannot contain spaces.

    To tag a dataset:

    1. Click on the dataset to expand it
    2. Click on Add Tags galaxy-tags
    3. Add tag text. Tags starting with # will be automatically propagated to the outputs of tools using this dataset (see below).
    4. Press Enter
    5. Check that the tag appears below the dataset name

    Tags beginning with # are special!

    They are called Name tags. The unique feature of these tags is that they propagate: if a dataset is labelled with a name tag, all derivatives (children) of this dataset will automatically inherit this tag (see below). The figure below explains why this is so useful. Consider the following analysis (numbers in parenthesis correspond to dataset numbers in the figure below):

    1. a set of forward and reverse reads (datasets 1 and 2) is mapped against a reference using Bowtie2 generating dataset 3;
    2. dataset 3 is used to calculate read coverage using BedTools Genome Coverage separately for + and - strands. This generates two datasets (4 and 5 for plus and minus, respectively);
    3. datasets 4 and 5 are used as inputs to Macs2 broadCall datasets generating datasets 6 and 8;
    4. datasets 6 and 8 are intersected with coordinates of genes (dataset 9) using BedTools Intersect generating datasets 10 and 11.

    A history without name tags versus history with name tags

    Now consider that this analysis is done without name tags. This is shown on the left side of the figure. It is hard to trace which datasets contain “plus” data versus “minus” data. For example, does dataset 10 contain “plus” data or “minus” data? Probably “minus” but are you sure? In the case of a small history like the one shown here, it is possible to trace this manually but as the size of a history grows it will become very challenging.

    The right side of the figure shows exactly the same analysis, but using name tags. When the analysis was conducted datasets 4 and 5 were tagged with #plus and #minus, respectively. When they were used as inputs to Macs2 resulting datasets 6 and 8 automatically inherited them and so on… As a result it is straightforward to trace both branches (plus and minus) of this analysis.

    More information is in a dedicated #nametag tutorial.

These are example tags, but it is recommended that you use name tags to reflect the identity of the input sample, in some way that’s important to you. If you’re doing an analysis comparing wildtype and treated samples, you might tag things based on the condition. If you’re processing NGS data you might want to tag based on the read set where each sample originated.

If you’re using a Galaxy server newer than 19.01, you will notice that the tags each have different colours:

Name tags have different colours based on their contents.

The colours are generated from a checksum of the text of each tag. Every time the same tag is used, the same colour will be generated. Different words will usually result in different colours.

This allows tags with very similar lengths and word shapes (e.g. peaks and genes) which can be mistake at small font size if you aren’t careful. While current evidence (Jacobs 2017) doesn’t support the “word shape” model of word recognition, it has still been seen to be beneficial to provide more contrast between words when mistakes can be costly (Degani 1999). In the context of Galaxy, mistakes are usually not incredibly costly, but can be annoying. This gives an easy additional signal on which you can mentally filter your datasets.

You can read some of the discussion that went into implementing the tags in the pull request to Galaxy.

Setting name tags with the Rule Builder

In this section we will demonstrate how you can add name tags to datasets automatically using the rule-based uploader. This is especially useful when you have a large number of datasets you want to tag all at once.

For more information about the rule-based uploader, please see this dedicated tutorial

Hands-on: Set name tags during upload
  1. Open the Upload galaxy-upload interface
  2. Click on Rule-based at the top

    Rule-based upload.

    As you can see in this dialog, data can be selected from a history dataset or pasted in directly

  3. Set Upload data as: to Datasets
  4. Paste the following links into the text box

    https://zenodo.org/record/1185122/files/GSM461176_untreat_single.counts
    https://zenodo.org/record/1185122/files/GSM461177_untreat_paired.counts
    https://zenodo.org/record/1185122/files/GSM461178_untreat_paired.counts
    https://zenodo.org/record/1185122/files/GSM461179_treat_single.counts
    https://zenodo.org/record/1185122/files/GSM461180_treat_paired.counts
    https://zenodo.org/record/1185122/files/GSM461181_treat_paired.counts
    https://zenodo.org/record/1185122/files/GSM461182_untreat_single.counts
    
  5. Click Build

  6. We will add a regex that creates 3 new columns with accession, treatment and library type:
    • Click on the Columnn button and then Using a Regular Expression
    • Select Create columns matching expression groups

      • Paste .*(GSM.*)_(.*)_(.*).counts in “Regular Expression”
      • Set “Number of Groups” to 3
      • Click on Apply

      We should have now a table with 4 columns: link, sample name, treatment, sequencing type

    • Click on Rules and then Add / Modify Column Definitions
      • Click on Add Definition and select:
        • “URL”: Column A
        • “Name”: Column B
        • “Name Tag”: Column B
        • “General Purpose Tags(s)”: Columns C and D
    • Click Apply
    • Click Upload

This will add 7 new datasets to your history, each one tagged and named appropriately.

Automatically tagged uploads.

The datasets now have two types of tags: name tags and general purpose tags. General purpose tags, unlike name tags, will not propagate. They’re mainly useful for searching through your datasets (Try entering tag=paired in your History search box).

Question
  1. How many tags does the dataset named GSM461177 have? Which ones?
  2. How many of those tags are propagating name tags?
  1. 3 tags. You can find all tags of a dataset by first clicking on it to expand it, and then clicking on the tags icon galaxy-tags. The tags are #GSM461177, paired, and untreat
  2. 1 propagating name tag. Only hashtags (tags starting with a # symbol) will propagate to downstream results derived from it. So in this example only #GSM461177 is a propagating name tag.

Name tags in Action

Now that we have our name tags in place, let’s see them in action!

Hands-on: Some analysis steps
  1. Select first tool lines from a dataset (head)
    • param-files “File to select”: Select everything starting with GSM
  2. Click Run Tool

We can already see the benefits of name tags. Even though our outputs are named things like “Select first on data 8”, they’re also visibly tagged with the name of the original input.

Tagged datasets are clearly traceable.

Next we’ll join two of these datasets together, to see the tagging behaviour on summaries of multiple datasets. In any box which allows selecting single files, you can also search on name tags.

Filtering datasets is possible on tag names.

Hands-on: Multiple inputs
  1. Join two datasets tool side by side on a specified field
    • param-file “Join”: Click the select box and type “461181” to filter the list and use the “Select first on data …” file
    • param-file “With”: Click the select box and type “461182” to filter the list and use the “Select first on data …” file
  2. Click Run Tool

The output will include all of the name tags from all of the inputs:

The output dataset should be tagged with both of the inputs.

If you compare the tags of the last output joining the two tables, and the original inputs, you will notice that the “General Purpose” tags were lost along the way. Only name tags are propagated with each analysis step.