Relations in Biomedical Ontologie: Responses to Referees

Some general points

The title has changed in light of the renaming of OBO (now: Open BioMEDICAL Ontologies)
Many changes have been incorporated into this version in additional to those discussed below in light of responses to several biologists who were asked to evaluate the original version. Most importantly, a new table (Table 1) has been added, giving an overview of the Relation Ontology with summary definitions and examples.

Referee 1 

Re: Manuscript by Smith and colleagues; reference number S1004201ap 

Referees are asked to ensure that the following questions are addressed in their reports: 
1. Does the work include all necessary controls? YES - position paper 

This paper proposes 10 class-level relations which are intended to 
assist with the curation of biological ontologies and anatomies. 
As no tangible resources are provided and no evaluation is performed, 
it is best understood as a position paper on the methodology of 
ontology development. 

2. Are the conclusions drawn adequately supported by the data shown? 
YES/NO 

Not applicable 

3. Are sufficient details provided to allow replication and comparison by interested readers with related analyses that may have been performed? 

The paper is not self contained. Some definitions for the class-level 
relations are provided in a structured natural language. However, the 
authors refer to a standard theory of mereology (reference 12) and to  
the work of Galton, Bittner and Cohn which cannot be taken to 
constitute a single theory of parts, regions and connectedness. There 
are undoubtedly a common core of relations and definitions, such as 
those given in the paper for part-of, but the authors do not specify 
the core axioms to be adopted in the proposed spatial theory. 

As the revised text now makes clear, the primitive relations are selected because they enjoy a high degree of intelligibility to human beings. The full set of axioms which would be needed to specify the meanings of such relations for the purposes of computer applications is being supplied in a separate paper. (A draft can be made available on request.) The defined relations are all clearly labeled as such, and the definitions are such as to employ only the primitives listed in the paper (also here clearly designated as such). A structured natural language has been used for the purposes of formulating definitions  because the purpose of this paper is to aid the human authors and curators of ontologies in correct use of the corresponding defined relational expressions. In this way, and in light of this purpose, self-containedness has been, we think, achieved. The references to Galton, etc., and the associated claims have accordingly been removed. 
As a position paper, it would be acceptable to motivate the need for 
relations of adjacency, containment, transformation etc and not to 
completely solve the problem of their exact formalisation.  
The paper does not reflect any discussion that may have taken place 
about the alternative interpretations that might be placed on the 
relations, or indicate how the set of relations was arrived at. Rather, the conclusion as to the correct answer is simply stated - and this is in contrast to Smith and Rosse [7] where some discussion is at least 
admitted. 

Discussion has been added as to how the set of relations was arrived at (both the primitive instance-level relations, and also the <class, class> relations which are the primary focus of biological ontologies).
4. Does the manuscript adhere to the field standards for experimentation, nomenclature, and public availability of data (or any other significant standards)? YES/NO 

The paper adopts two different syntactic conventions:  
"C contained_in C1" and "Cct" (where C is the predicate) 
I can't see that any clarity is gained by this approach, nor does the 
lack of parentheses and the avoidance conventional logical connectives 
aid the reader. It would be understandable if the definitions were 
presented in an appendix, in some standard syntax, if the authors 
prefer to keep the main body of the text free from mathematical 
notation. 

The text now includes further discussion of the primary goal of this paper – which is to help human authors and curators (and thus also the users) of biological ontologies. For this purpose, maximal readability was selected at the price of a certain degree of logical elegance. 
We also point out that some of the problems addressed have in fact been well known for some years in the Description Logic community, but -- because few biologists can interpret the standard DL syntax -- the corresponding lessons were not learned. Our semi-formal definitions have been tested in the intended biologist user-communities and found to be highly serviceable and intelligible. 

5. Does the software represent a significant advance over previously 
published tools (as demonstrated by direct comparison with available related 
software) and is it likely to be of broad utility?  

No software is provided. 

6. If you feel that the paper is of broad interest to others in the field, or of outstanding interest to a broad audience of biologists, please say so, with a brief explanation of why. YES/NO 

The paper contributes to the discussion of the proper principles for 
the construction of ontologies and anatomies. This is an important 
concern, and the central proposal that class-level relations should be introduced and defined is convincing. 

My concerns are that while the paper presents a series of proposals for class-level relations, there is little guidance on their use by the community addressed (the GO and OBO developers and curators) for the stated purpose (ontology curation and the prevention of coding errors).  
The has been no 'requirements analysis' and no 'use cases' are 
presented. The proposals do not account for the practical aspects of 
building large ontologies or knowledge based systems although the 
authors could bring such experience to bear. Therefore, the utility of 
the central proposal is not demonstrated to the extent that it 
could be. 

It is now made clear that the requirements analysis is contained in earlier papers, especially in the paper Smith, et al. On the application of formal principles to life science data: A case study in the Gene Ontology. It has also been made clear that the methodology has been applied successfully to medical ontologies such as SNOMED (and corresponding references have been supplied).
It is now also made clear that these proposals in fact reflect explicit requests from and collaboration with the GO, OBO, FMA and GALEN communities (representatives of all of which are included among the co-authors), and incorporates also suggestions from other biologists involved in ontology building who have experimented in their work with the definitions here supplied. Further discussion has been added also as to the kinds of errors (for example ensuing from conflicting readings of “part-of”) which this framework is designed to avoid. Also further references have been added to the literature on such errors in different kinds of biomedical ontologies.
There is no comparison with the relations used in GALEN or in the FMA where relations such as surface_of and the concept of shared parts play a significant role. There is no mention of the proposal of Schulz and 
Hahn (Proc. KR MED 04) who address the same problem as the present 
paper. 

Fixed

Minor comments. 
Axioms should be in a standard logical syntax, and/or explained in text. 
Is a temporal modal logic adopted, or should the phrase "at t" be taken to mean that t is a temporal argument ?  
Figures 2 and 3 are not suitable for publication - what do the dots 
mean ? 

The use of "C transformation_of C{subscript}1" is inconsistent,  
C1 transformation_of C2 or C transformation_of D would be preferable. 

All fixed

If the examples used were more biological e.g. if it is envisaged that regions such as convex hulls can be described then it might be easier to relate the spatial descriptions to anatomy. 

This problem is now moot, since the corresponding claims regarding reasoning over mereotopological relationships, have been weakened. (To support these claims in the way the referee required in his earlier comments required a separate paper, which is nearing completion.)

Referee 2 

I read with interest the manuscript by Smith and colleagues.  

The first thing to say is that paper addresses a genuine problem with biological databases, namely that the meaning of terms and 
relationships are not well defined. This has two major impacts on my work on building and maintaining a database of protein functional interactions. First it makes the integrations of databases difficult since often similar terms are used that mean different things in different databases, and, as the authors point out in various of the citations, meaning is even inconsistent within single databases. Because of this it is difficult to write programs check consistency or detect other kinds of errors. 

However, the paper does not go far enough in satisfying the claims it 
makes. The first claim is. 

"that the class-level relations used in biological ontologies such as 
the Gene Ontology (GO) can be provided with coherent definitions only 
when appeal is made to primitive relations of these sorts - relations 
which involve reference to instances in reality" 

While they convincingly demonstrate the sensibility and clarity of 
defining class relations in this manner, I would have liked to see 
them then refer back to GO and show how these relations repair some of 
the flaws they have noted in some of their previous work. I was also 
unclear by what criteria this initial set of relationships was chosen. 
Are these necessary underpinnings of any biological ontology? Are they 
the subset which, if defined well, give us maximal leverage in 
building solid knowledge bases for knowledge of gene products and 
their functions? 

Further clarification, and examples of types of problems already resolved by this methodology, have been supplied. 

The second: 

"that the provision of such definitions can be exploited to support 
the authoring and maintenance of biological ontologies in the future 
by providing curators with a more coherent understanding of both the 
relations and the relata which such ontologies involve" 

It is convincing that, in general, being more precise and consistent 
about the definitions of terms helps one construct terms in the 
database. The authors demonstrate that there are consistency checks 
that can be made on the basis of the definitions they propose - 

e.g. which relations are transitive and which not. What I was left 
wondering was whether having these checks in place would help in the 
maintence of GO or other available gene centered databases. What would 
help here would be reference to specific problems in such databases 
that would be caught by these checks. For instance, if these checks are 
applicable to GO, list a number of issues that are caught by them when 
applied to it. 

Again, some examples have been supplied, together with associated reasoning as to how the methodology will catch them. However the logical details will be supplied in the separate paper, mentioned above.

The third: 

"that the resulting framework can in addition support new types of 
automated reasoning with biological data and annotations, including 
reasoning about the spatial and temporal dimensions of biological 
phenomena." 

There are quite a few frameworks that allow one to do this in 
general. Automated reasoning is enabled by an specific enough rules 
about how things relate and progress. Of course there are many 
problems that arise when one tries to do this in practice. What would 
be new here would be to show that this particular set of relations is 
adequate to express interesting relations (at least for the audience 
of Genome Biology) among gene products involved in cellular processes, 
that it is tractable, and that the results of the reasoning are 
interesting and useful. 

The paper shows how a number of further relations can be defined in terms of the chosen set, especially the relations of derivation and transformation/development (which were run together e.g. in recent versions of OBO’s cell ontology), and the associated notions of fusion, fission, budding and absorption.

In summary, while I think the claims are true, I don't think that the 
authors provide enough examples or development in the paper to warrant 
them and make them of direct interest to your audience. 

Further examples, and further references to the literature on such examples, have been included.

I do think (and hope) that these issues can and should be addressed. Some of the author's other papers referenced here are much 
more specific about GO and raise issues that could be addressed by 
their framework or some modification to it. Showing how these problems 
are addressed by the proposals here would be an important 
contribution. 

One issue of particular interest is the issue of is_a relations and 
multiple inheritence in [13]. Given their discussion there (and 
suggested alternative) I was surprised that they chose to include is_a 
in their initial set of relations and wondered what prevented them 
from suggesting a more ambitious solution. In this manuscript they 
warn against using multiple inheritence. However the GO ontology uses 
multiple inheritence extensively. As gene products can have multiple 
functions in multiple cell components this is a real problem that 
needs to be constructively addressed. There is mention of the 
temporary parthood relationship "C is part_of C1 at some time in C's 
existence" in the discussion of the part_of relation. I think it would 
be engaging if the authors asserted, for instance, that this 
relationship should be the underpinning of the GO component ontology 
and demonstrated some of the consequences. 

Some further discussion of multiple inheritance has been added, including a statement of the reasons why many of the problems commonly associated with multiple inheritance are solved by the methodology described in the paper.

Another area that I think could be developed profitably would be their 
account of processes and particularly the derives_from relation which 
seems more complicated to me than the space devoted to it. 

The initial treatment of derives_from was in fact unconscionably compressed; it has now been expanded, and the Figures amended accordingly.
How 
processes are to be recorded is of very practical interest to those of 
us who collect and mine biological knowledge. Take the process of  
protein 
degradation that is of high interest to Millennium. Various signals 
can direct the ubiquination of proteins which targets them for 
degradation by the proteasome. Some aspects of this process is  
understood, 
some not. The question is, how is the curator of a biological 
knowledge database to represent information about this process? How 
can we mine existing databases to collect information relevant to 
understanding the problem? 

Finally, a point that caught my attention was the use of the term 
"genuine biological classes". It would be helpful to explain what they 
mean by this and elaborate on how it fixes the contingent inclusion 
problem. Given the audience it would also be helpful to replace the 
example "mammal in saarbruken is_a mammal" with a more compelling 
example taken from molecular biology. 

The example has been changed and some further explication added. 

Hope this helps, 

Sincerely, 

Alan Ruttenberg 
Millennium Pharmaceuticals 
alanr@mpi.com 

Referee 3 

>THIS SHEET IS FOR COMMENTS THAT THE AUTHORS MAY SEE >(The referee may elect to sign this form) > >Re: Manuscript by Smith and colleagues; reference number S1004201ap  Referees are asked to ensure that the following questions are addressed in their reports: 

1. Does the work include all necessary controls? YES/NO 

This is not a standard experimental manuscript, but more of a treatise on the logical definition of terms that have heretofore been used in a somewhat haphazard manner. As such, it is a useful contribution, that will be of interest primarily to bioinformatics professionals. 

I would encourage the authors to address the important question of whether logic is the right language to use for biology?  Biology is in many ways all about ambiguity and uncertainty.  A two-valued logic system seems necessarily doomed to inadequacy.  Instead, a more probabilistic (fuzzy?) formalism may be more realistic.  It would be very useful, therefore, for the authors to discuss these issues and  their "take" on this tradeoff. It may be that the definitions  provided can be used even for multi-valued logics? 

The logic used is deliberately left unspecified, and the definitions presented are in fact compatible with a variety of many-valued logics. However, we have preferred to leave such matters unexpressed in order to maximize the readability of this text among biologists.  (Our goal is that biologists will use the definitions here supplied in authoring ontologies – and response from biologists in this connection thus far has been very positive.)

Also, I am very worried about the introduction of temporal aspect to this logic. The authors do not refer to the work of Allen and others in creating logical relations based on time points and intervals.  
The treatment in this paper seems a bit superficial, and this could lead to problems down the line. Thus, I would appreciate some discussion of the broader scope of temporal reasoning, and what the authors believe is reasonable in the present study. 

The treatment of time has deliberately been kept logically simple (again, in order to maximize accessibility especially among biologists); but the treatment given is not superficial, and has been confirmed as such by logicians with expertise in the corresponding areas. Use of Allen’s interval logic was considered at an early stage, but was found to be logically less tractable and biologically less useful than the instance-based approach here offered. A combined approach may be necessary at the point where more detailed relations involving processes need to be added; this will be an unproblematic extension, but was avoided, here, again, in order to maximize readability for non-logicians.

Finally, I think the authors should provide some further  
justification about why the relations listed on page 4 are chosen,  and if they can really be expected to form a true basis set of  relations required for biology?  If so, why? If not, what process  will be used to identify the additional relations? 

Relevant text has been added.

> 2. Are the conclusions drawn adequately supported by the data shown? YES/NO 

Yes, for the most part the paper is lucid and clear.  The logic is strong! 

>  3. Are sufficient details provided to allow replication and comparison by interested readers with related analyses that may have been performed? YES/NO 

Yes. 

> 4. Does the manuscript adhere to the field standards for experimentation, nomenclature, and public availability of data (or any other significant standards)? YES/NO 

Yes, although again this is a non-standard type of publication. 

>  5. Does the software represent a significant advance over previously published tools (as demonstrated by direct comparison with available related software) and is it likely to be of broad utility? 

No software, as far as I can tell. 

>  6. If you feel that the paper is of broad interest to others in the field, or of outstanding interest to a broad audience of biologists, please say so, with a brief explanation of why. YES/NO 

This is of great interest (or should be) to bioinformaticians who are  involved in annotation, or the creation of software systems that are  expected to be highly interoperable.  The contribution is the clear definition of some basic relations, in a manner which attempts to remove all semantic fuzziness. 

I am enthusiastic about this paper.