Quality Commenting

Guidelines for quality commenting (annotation) of observation data

The Finnish Species Information Facility (FinBIF) collects observational data from many different sources, such as authorities and databases compiled by research projects and nature enthusiasts. The information is published on the Laji.fi portal as openly as possible. In order to improve the quality of the data, quality comments, i.e. annotations, can be attached to individual observations. Data users can then filter the data based on these comments and quality ratings. Quality control is mainly done on a voluntary basis, so its coverage and speed depend on user activity.

All registered users of Laji.fi can make quality comments. In addition, individuals identified as experts have more rights to make quality labels that affect the findability and identification of observation. This page explains how to make quality comments with the new annotation system released in April 2020. (See also more background information on quality control and finding findings based on their quality.)

General principles of quality commentary

When observations are compiled on the Laji.fi portal, they are initially “unassessed” (unless individual observations have been verified in the original source.) In quality commenting, volunteers can browse the observations and make various entries using the tools provided by Laji.fi.

The aim of the commentary is to improve the quality of the observation data

1. By evaluating and labeling individual observations to different quality levels;
2. By adding and refining species identifications;
3. By passing on information about quality comments to the owner of the observation so that errors can be corrected in the original source;
4. Enabling discussion of findings;
5. By helping all involved to learn about new species and their observation.

Commenters (annotators) are divided to two categories with two different roles:

1. As a basic user – anyone who has logged in to Laji.fi can make comments;
2. As an expert – people defined as experts have more rights than basic users, and their comments carry more weight.

The most important duty of basic users is to add tags to uncertain observations for review. This tag does not directly affect the quality level of the observation, but observations can be searched based on tags.

The most important duty of expert users is to review the observations, as well as to classify them into confidence levels (verified, uncertain, erroneous) and / or to give them a new or refined identification.

Information about new comments (annotations) also passes to the observer, if the observation is recorded by using the Notebook observation system. Comments cannot currently be forwarded to other information systems. The planning on how to transfer them to the Kotka Collection Management System has started.

Guide for Basic Users

All Laji.fi registered users (so-called basic users) can add comments to the observations, which may include

• Tags indicating that the observation should be re-checked for particular reason (coordinates, time/date etc.);
• Proposal for new species identification;
• Free-form text comment.

The quality comments by the basic users do not directly affect the quality classification of the observations, but raise the attention for review by experts. The exception is where the findings and all commentators agree on the species identification. In this case, the observation is classified as “Community verified“.

How do I add comments?

You can browse for observations, as well as their quality ratings and tags, on the observation Browse & Search Annotations tab. Click on an observation to give a comment.

If you are browsing an individual observation record, you can give a comment by using “comment” button.

Consequently opens a modal where the tags and comments can be added:

Basic users can add the following tags to their observations:

• Check Coordinates – The coordinates of the observation appear to be incorrect;
• Check Date/Time – The date or time of the observation seems incorrect;
• Check Location – The location names for the sighting appear to be incorrect;
• Check Observer – The observer or collector appears to be incorrect;
• Check Identification – The species identification appears to be incorrect;
• Check Wildness – The finding appears to be from a non-wild organism, but has not been marked as non-wild. For example, a garden plant or a runaway pet.
• Check – There seems to be something else wrong with the observation.

What to do if …

The observation appears suspicious or incorrect

Add a “check” tag to the observation and a comment describing the reason. This brings it to the list of findings to be reviewed, after which experts/owner can review and verify it.

The observation is clearly misidentified

For example, if a finding is incorrectly identified based on your view after investigating the images attached to it, add the tag “check identification” and, if you want, add a new identification (taxon). This tag brings it to the list of observations to be reviewed, after which experts/owners can verify and/or re-classify it.

You want to confirm the identification

If the observation is correctly identified and you want to confirm this (e.g. because it is a rare species) you can choose “Select current”. If both the observer and the commentator (s) agree on the species, its quality classification will rise to a level guaranteed by the Community.

There are several species in the observation

If it is not clear which species the observation is about (e.g. butterfly and beetle in different images of the same observation), comment on the observation by kindly stating that there should be only one species in the observation.

The quality comment made by the expert is invalid

Add a “check” tag and a comment describing the problem. This returns it to experts for evaluation.

---

One of the principles of the FinBIF is to make data as open as possible. Open data enables new uses of information – what is too imprecise or uncertain for one purpose may be appropriate for another. Therefore, the observations are not completely hidden based on their quality. All data is available and it is the responsibility of the user to decide what is appropriate for his or her intended use. (Read more about quality control and instructions for retrieving observations based on quality ratings.)

---

Guide for Experts

The users defined as experts in Laji.fi have more authority to comment on quality and classify observations.

How to become an expert?

Quality control is mainly done on a voluntary basis. If you are an expert in a species group because of your work or have gained expertise in other ways, and you want to become an expert quality controller, please contact helpdesk@laji.fi and and tell us a little about your skills and background. We also hope that each expert will write a short description of themselves in their own public profile.

For the time being, the admins of FinBIF grants expert access rights. Later, the process will be made more transparent.

Expertise is not limited to each taxa, but the person designated as an expert has the power and responsibility to choose when to make comments as an expert and when as a basic user. For example, a bird expert may know Finnish birds well, but African birds poorly. In this case, he may decide to comment on African observations as a basic user. An important part of expertise is knowing the limits of one’s own competence and refraining from situations where one is not sure enough.

How to make expert annotations

Expert annotators can go through the observations in the same way as basic users (see above), but they have an “expert form” at their disposal that allows them to take more varied action.

When you open an observation for comment, select “switch to expert annotation” to see more features:

This allows you to add expert tags to observations in addition to the basic tags:

• Spam – observation is clearly spam, joke, etc.
• Verified (quality tag) – the observation can be considered verified, taking into account the prevalence of the information (images, recordings, description text, etc.) and species associated with it.
• Uncertain (quality tag) – the observation seems uncertain given the information provided and the prevalence of the species.
• Erroneus (quality tag) – the observation has been checked and found to be clearly incorrect.
• Coarse – This tag coarsens the observation. Can be used if the observer requests coarsening, or for coarsening a foreign sensitive observation. (Finnish observations are coarsened automatically, don’t use this tag for them!)
• Cannot identify species – this tag removes the observation from the checklist, even if it is not classified or identified.
• Non-wild – the observation concerns, for example, a garden plant or a pet.

If you provide a quality rating tag (verified, uncertain, or erroneus), you must also enter the taxon to which your rating applies. In this way, it is unambiguous which taxon you confirmed, even if the observation changes later. (Learn more about how quality tags affect an observation’s quality rating.)

• For example, if a sighting is reported as European green woodpecker, but there is a grey-headed woodpecker in the observation photo, post a comment
  • Identification: grey-headed woodpecker
  • Tag: verified

You can also delete tags provided in previous comments. If you’re reviewing an observation, it’s a good idea to delete previous tags at the same time.

Principles and practical instructions

Pay attention to etiquette and good manners, and assume that the observer is sincere. Errors are often due to a typo made by an observer or someone else, deficient skills, or because the observations have been identified using outdated literature or species concepts.

Observations do not necessarily need image evidence to verify them. A verbal description of the signs is also often sufficient. Use your expertise to make a case-by-case assessment.

Observations should be identified and verified at a higher taxonomic level (such as genus or tribe) rather than marked as erroneus at a lower level (such as species) unless there is good reason to do otherwise.

If there is reason to suspect an observation and it is not possible to identify the species based on the images, descriptions, etc. attached to it, but yes to some higher level, record the most accurate taxon for the observation that you can verify.

• For example, if an observation with an image is defined as a rare reindeer dandelion without justification, mark it as a verified dandelion (i.e., at the genus level) rather than an uncertain reindeer dandelion.

Note that comments apply to the entire observation (not just the species identification), so make sure the location, time, and other information make sense before marking the observation as verified. If there is any ambiguity in these, write a comment asking for clarification. But since evidence of the correctness of place, time, etc. is difficult to give, it cannot be required in all cases either, but the observer must be trusted.

For example

• A cuckoo in June in Finland can be marked as verified without an image, but an observation with an image made in January can be marked as uncertain (typo in the date?).
• An observation of a rare butterfly can be marked as verified, although the location of the observation cannot be proved on the basis of the image.
• The observation of one white stork can be verified on the basis of a verbal description, but the observation of 100 white storks can not.

Please note that most of Laji.fi’s observations do not come from the Notebook Observation System, but from research or public authorities sources, for example. The administrators of these data usually do not read the comments. The name of the collection is displayed on the annotation page in connection with the observation information.

• Recent observations from the iNaturalist service should not be commented on immediately at Laji.fi. In iNaturalist, the user community refines the observation step by step, which is why it usually changes during the first week. And when the identification changes, comments about it are no longer valid (i.e., the comment job is wasted.)

Note that the comment field does not affect the classification or searchability of the observations in any way – they are suitable for reasoning and sometimes communication with the observer. For example, if you tag an observation with “verified” and write “maybe” as a comment, the quality will be “verified.”

If there is an official process for processing a species group observation, use it. For example, in birds, the observations of certain species are reviewed by the Rarity Committee or the Regional Rarity Committees. Only the observations of these species that have been accepted by the committees can be marked as verified.

Examples

Quality comments can be used very flexibly, and expert users are free to choose the method that suits each situation. Here are indicative examples of how slightly different observations can be classified.

Observation 1

Observation: European green woodpecker , Finland, 15.7.2019. No description. With image.

Expert annotation

• Tag: verified
• Identification: grey-headed woodpecker
• Comment: This is grey-headed woodpecker, European green woodpecker is very rare in Finland.

Note: Because the observation has a good photograph, it can be reliably identified at the species level, so it can be marked as verified.

Observation 2

Observation: European green woodpecker , Finland, 15.7.2019. No description. No image.

Expert annotation

• Tag: uncertain
• Identification: European green woodpecker
• Comment: Probably grey-headed woodpecker, European green woodpecker is very rare in Finland.

Note: As there is no documentation of the observation, it should not be verified contrary to the observer’s opinion – there is a (small) possibility that the observation is from a real European green woodpecker, or from some other species.

Observation 3

Observation: European green woodpecker , Finland, 15.7.2019. Description: “woodpecker with a green back and gray otherwise”. No image.

Expert annotation

• Tag: verified
• Identification: Picus (genus covering both European green woodpecker and grey-headed woodpecker)
• Comment: Probably grey-headed woodpecker, European green woodpecker is very rare in Finland.

Note: The description given corresponds to both European green woodpecker and grey-headed woodpecker, so the observation can be confirmed at the genus level.