Sample Reviews

Below are examples of helpful and less helpful comments in reviews of Research Reports, for submissions that would be accepted as well as those that would be rejected.

Helpful Reviews

A Helpful Review of an ACCEPTED Paper A Helpful Review of a REJECTED Paper

Rationale and research question(s):

The research question is clearly stated, is interesting, and the author provides sufficient context to make clear the significance of the problem.

Rationale and research question(s):

The research questions are overly broad, and the rationale includes too much of the author's opinion and insufficient connection to educational issues/context.

Theoretical framework and related literature:

This is a small-scale study that builds on prior work by ---- and ---- on the learning of rational number concepts. The theoretical framework is clearly laid out and provides justification for the choice of research questions.

Theoretical framework and related literature:

The literature review on beliefs is not connected to the study undertaken, as described in the statement of purpose. There is no theoretical framework provided beyond the literature review.

Methodology:

The study is primarily descriptive, with ongoing analysis of the data. The descriptions of students’ work, together with the verbal data, are well selected to support the points made by the author, although I would have preferred to see other evidence provided in addition to that here.

Methodology:

The scales used to measure beliefs were not based on any theory, nor was any information provided on the reliability of the scales. One wonders about the validity of the study.

Results:

The authors make a clear connection between their analysis of student learning and of the theoretical framework presented. The results are clearly presented and well substantiated by the data.

Results:

The authors make a number of unsubstantiated claims; for example, they say that males believe that all word problems should be able to be solved quickly, whereas females do not believe this, yet no data is provided to show this claim to be true.

Clarity:

For the most part, the report is well written and easy to follow. The table, however, appears to be missing some of the data intended to be there (see column 2).

Clarity:

The authors have not made good decisions about how to wisely use the eight-page limit. The introductory section contains too much detail, while the results are presented in a cursory fashion.

Relevance to PME:

This work is well connected to previous PME work in this area, both in terms of its topic and development, and in terms of references to other PME work.

Relevance to PME:

The purposed study would be of interest to many PME members but the manner in which the study was undertaken is not appropriate.

Please select your final recommendation:

Accept as Research Report

Please select your final recommendation:

Reject as Research Report

Reasons for recommendation:

The paper is interesting to read, contains original data, and adds to our understanding of how students learn rational numbers. The authors are encouraged to see the recent article by --------- which would provide an interesting comparison to the work in this paper. The presentation should contain a corrected table.

Reasons for recommendation:

I am sorry to reject this paper because the research question is interesting and relevant. However, the lack of any theoretical framework, the poor design of the study, and the questionable results make it unacceptable.

Unhelpful Reviews

An Unhelpful Review of an ACCEPTED Paper An Unhelpful Review of a REJECTED Paper

Rationale and research question(s):

Interesting questions.

Rationale and research question(s):

Question has been addressed in other research.

Theoretical framework and related literature:

Framework not one I’m familiar with.

Theoretical framework and related literature:

None here.

Methodology:

Appears to be fine.

Methodology:

Poorly designed.

Results:

Interesting findings.

Results:

Appears to be fine.

Clarity:

Well written.

Clarity:

Poor

Relevance to PME:

Of interest to some PME members.

Relevance to PME:

Ok, if better designed.

Please select your final recommendation:

Accept as Research Report

Please select your final recommendation:

Rejected as Research Report

Reasons for recommendation:

Good paper. Should be accepted.

Reasons for recommendation:

Doesn't meet PME standards.