Meaning of the traffic lights
- 1=successful (green)
- 2=failed and 3=not replicable (warning triangle)
- 4= partially successful (yellow)
For raw data accessibility we use
- 0=not available at all (warning triangle)
- 1=somewhat available but insufficient to replicate final dataset (yellow)
- 2=raw data can be used to replicate a dataset that is not substantially different from the final dataset used for the publication (yellow red)
- 3=final dataset can be exactly replicated from raw data (green)
Please be aware that the symbols by themselves do not say anything about the quality of the studies, they just document what can be found in the literature. If raw data access was correctly assessed in a replication can only be checked by the community. If we are not sufficiently active there is no way to tell if original studies or replications have weaknesses. Be aware that many replications are not replicable themselves. They get the same symbols as the studies they replicate just to illustrate the issues documented. Due to the danger that the traffic lights be misunderstood as a judgement on quality of a study or its replication a more appropriate symbol is now used, a warning triangle for what before was a red light. Please use the discussion pages if you have suggestions, or if you feel confident about it: just go ahead, this is a wiki.
How the traffic lights are set
The traffic lights are set automatically by a bot every night. For studies with more than one replication, replications that refer to more than one original study, and replications that were also replicated the traffic lights need to be set manually. If all results are the same you can just set the according traffic light. In other cases a traffic light with more than one color and a note explaining this should be set. This should be agreed upon on the discussion page. You can set the traffic lights manually by copying the code of the top of this page. Replace the files with Yellow.png or Yellowred.png if needed. Write a note explaining why more than one light is set after "Raw data" or "Replication result".