Index: User Guide

Loading and Running Workflows

U-Compare should soon finish loading. The rest of this document provides a walk-through of several tasks that should give readers a reasonably good idea of how to use U-Compare. The task covered in this first section is the loading and running of a predefined protein Named Entity Recognizer (NER) comparison workflow. This will introduce the U-Compare Workflow Manager and Session Manager windows and so provide a general overview of U-Compare.

U-Compare Workflow Manager

Once U-Compare has finished loading the ``U-Compare: Workflow Manager'' window will appear. This window is the main U-Compare window and is used to construct, edit and run workflows. As can be seen below, it is divided into two main sections, the ``Workflow Viewer'' on the left, and the ``Component Library'' on the right.

Workflows are constructed and edited in the viewer, and their components are sourced from the library. We will make use of this window more in the next section, but for now notice the progress bar, start and stop buttons on the left below the Workflow Viewer. These buttons are used to stop and start processing of the workflow currently displayed by the viewer, and the progress bar will display the progress of any processing tasks.

Loading a Workflow

We want to look at the results produced by running a workflow, but before doing that need a workflow to run. For now we will just load a predefined workflow, before coming back in the next section to look at how to construct one. U-Compare comes with several predefined workflows designed to give users an idea of what is possible. The workflow we will have a look at now is the ``U-Compare NER Comparison'' workflow which runs several named entity recognizers in parallel and produces statistics allowing their proformance to be compared.

To load this workflow first select the ``Workflow'' menu in the top left hand corner, which should give the above list of options. The ``New workflow'' option, will discard the current workflow and open a new blank workflow in the workflow viewer. The ``Save workflow'' option saves the current workflow allowing it to be used again later. The ``Export Workflow/Components'' and ``Import Workflow/Components'' options allow workflows to be exported to/imported from the file system.

The remaining three menu items are a list of all currently saved workflows, initially just the predefined ones. To load the ``U-Compare NER Comparison'' workflow just click on its name in this list. Once the workflow loads it will appear in the workflow viewer and should look something like this:

Scrolling the left panel you should see the two top level components in this workflow, ``Aimed Collection Reader'' and ``Protein NERs Comparison''. ``Aimed Collection Reader'' reads in files from the AIMED corpus and annotates them with gold standard Sentence and Protein Annotations. The ``Protein NERs Comparison'' component is an aggregate component constructed from three named entity recognizer components that it is meant to compare. When the workflow is processed, all three of the NER components will be run against the document(CAS) produced by the collection reader and then their outputs will be compared against each other and against the GOLD standard data produced by the collection reader.

Running the Workflow

Now that we have our sample workflow open, lets run it. To do this just click the play button (the Triangle in a circle) below the viewer at the bottom left of the Workflow Manager window. Once you click play, U-Compare will begin initializing each of the components in the current workflow, and once complete, will start processing the workflow.

Workflow initialization can take quite a while if the workflow includes components that haven't been used before (these will automatically be downloaded from the server). Workflow processing time, after initialization is complete, will depend on several factors including the components used, the number of files read by the collection reader and the speed of the users machine and internet connection. For the current workflow however, both initialization and processing should be completed in at most a couple of minutes.

Session Results Window

Once U-Compare has finished processing the workflow, the ``Session Results'' window will be displayed. This window displays information on the runs of recently processed workflows and includes statistics such as component run times, numbers and positions of generated annotations and detailed component comparison statistics. For the NER workflow it should appear as below.

Statistics are presented under several tabs, the number of which will vary according to which components are including in the workflow. In the case of the NER comparison workflow three tabs are displayed; ``Performance Statistics'', ``Annotation Statistics'' and ``Protein Comparison''. The first two of these are displayed for all workflow, but the ``Protein Comparison'' tab is displayed due to the inclusion of a comparison component in the workflow. We will take a quick look at the two statistics tabs and return to the comparison tab in a later section.

Performance Statistics Tab

Select the ``Performance Statistics'' tab by clicking on the tab title. The tab should look something like that shown below:

This tab provides information on the run time of each component in the workflow. It also provides the option to ``Restore Workflow'' which lets the user reload the workflow that produced this set of results into the workflow viewer. This feature is useful for situation in which you have several session results open and are unsure of just which workflow/configuration it was that produced a particular set of results.

Annotation Statistics Tab

Now select the ``Annotation Statistics'' tab which should look like that shown below:

This tab provides a list of all documents that the workflow's collection reader outputed, gives general information on each and provides a count of the number of each type of annotation that the workflow produced for that document. The collection reader in the current workflow is set to only output a single document which is why this table has only a single line in this example.

Displaying Annotations

In addition to displaying the number of annotations present in each document, this annotation statistics tab also provides a way of viewing what part of the document these annotation cover. To the left of each entry in the table is a ``Show'' button; clicking this button will cause the associated document to be displayed as below:

The display pane shows the original document, with annotated sections underlined in a color corresponding to the type of annotation. The annotation-color key is displayed to the right, with a check box next to each annotation type that controls whether ot not that type of annotation is displayed in the text panel.

To display details on individual annotations click on the corresponding underlined portion of text and additional details will be shown as in the above example. As some document segments may contain a large number of annotations the displayed details may get in the way of each other, if this happens simply click and drag any interfering annotation out of the way.

Further Examples

You should now know how to load and run a workflow, and have some idea of the sorts of results U-Compare provides after processing a workflow. In the next section we will see how to create and configure a new workflow. Later section will then expand this workflow, before finally going on to create and evaluate a U-Compare comparison workflow. At this point you might want to have a look at the ``U-Compare HPSG parse and visualize'' and ``U-Compare POS Combinatorial Comparison'' predefined workflows to get a better idea of what U-Compare is capable of. These workflow can be loaded and run in the same way as the NER comparison workflow in this section.

Index: User Guide

• Launch U-Compare
• Home
• User Guide
• Developer Guide
• Components
• Forum
• Updates
• Publications
• License
• Contact
• Link
• Japanese
• BioNLP 09

by Google