ognizant Communication Corporation

TECHNOLOGY
A Journal of Science Serving Legislative, Regulatory, and Judicial Systems
Human Advancement · Environmental Protection  · Industrial Development

ABSTRACTS
Volume 11 - 2

Technology, Vol. 11, Issue 2
Copyright © 2008 Cognizant Communication Corp.

An Independent Peer Review Process Developed by the American Society of Mechanical Engineers for the U.S. Department of Energy

Charles O. Velzy, Ernest L. Daman and Nathan H. Hurt
Past Presidents of the American Society of Mechanical Engineers

Anibal L. Taboas and Yvette Collazo
Chicago Operations Office, Department of Energy

A. Alan Moghissi,* Betty R. Love, and Sorin R. Straja
Institute for Regulatory Science, Alexandria, Virginia

Peer review is the foundation of acceptability of scientific and engineering information. If properly managed, it provides the manager of governmental, industrial, and other organizations with credible and timely technical materials. This paper summarizes the results of a program developed by the American Society of Mechanical Engineers (ASME) in cooperation with the Institute for Regulatory Science (RSI). The program was funded by the U.S. Department of Energy (DOE) and used primarily by several offices of DOE. However, it was also used by other organizations. The Peer Review Committee (PRC) formed by the ASME was responsible for development and enforcement of policies and procedures to ensure the integrity and credibility of the Program. Review Panels (RPs) appointed by the PRC were responsible for peer reviewing specific projects. The day-to-day operations of the RP were managed by RSI. Judging by the large-scale recognition of the Program, it can be concluded that it was credible, timely, economical, and applicable to virtually all scientific and engineering activities.

Key words: Peer review

*To whom correspondence should be addressed at Moghissi@nars.org




Technology, Vol. 11, Issue 2
Copyright © 2008 Cognizant Communication Corp.

Independent Peer Review and Independent Scientific Assessment: The Process and Its Application

A. Alan Moghissi,* Betty R. Love, and Sorin R. Straja
Institute for Regulatory Science, Alexandria, Virginia

Independent peer review is the foundation for acceptability of scientific claims. Virtually all credible scientific (including engineering) publications rely upon peer review in their decisions to publish or reject a manuscript. Similarly, at least in the United States, most government agencies rely upon peer review to fund a project or prioritize projects for funding. In addition, many government agencies and others perform scientific assessments in response to their respective needs. The process for peer review and scientific assessment described in this paper was developed in cooperation with a number of organizations, notably the American Society of Mechanical Engineers. It has a clear and unambiguous structure. An independent committee oversees the entire process and is responsible for development and enforcement of policies, procedures, and appointment of Review Panels and Assessment Panels. Policies and procedures established during the development of the process ensure that members of the Panels are qualified for the specific review or assessment and are independent, as demonstrated by lack of conflict of interest. Reviews and assessments performed for national and international organizations, including federal, state, and local governmental agencies as well as for the U.S. Congress, demonstrated the validity of the process. Judging by the large-scale recognition of the program, it can be concluded that it was credible, timely, economical, and applicable to virtually every scientific and engineering activity.

*To whom correspondence should be addressed at Moghissi@nars.org




Technology, Vol. 11, Issue 2
Copyright © 2008 Cognizant Communication Corp.

Multiple Attribute Decision Making: Application to Prioritize Projects for Peer Review

Sorin R. Straja*
Institute for Regulatory Science, P.O. Box 7166, Alexandria, VA 22307

Charles L. Nalezny
U.S. Department of Energy, Germantown, Maryland

Yvette T. Collazo
U.S. Department of Energy, Aiken, South Carolina

The American Society of Mechanical Engineers and the Institute for Regulatory Science were asked by the U.S. Department of Energy (DOE) to peer review various research and development projects funded by DOE. During the initial phases of the peer review program, it became clear that the number of projects was too large for the program to review every one of them annually or even periodically. In conjunction with these activities, a study was initiated to develop a logical process to screen all projects, prioritize those that required peer review, and eliminate those that did not need to be peer reviewed. A Multiple Attribute Decision Making technique was used to generate a composite score and prioritize the projects. The methodology described in this paper has general applicability beyond DOE-funded projects and provides managers with an approach to screen their respective projects and prioritize them for peer review. Once the process of information collection for the chosen attributes is in place, the computation will be largely automated.

*To whom correspondence should be addressed at sstraja@nars.org




Technology, Vol. 11, Issue 2
Copyright © 2008 Cognizant Communication Corp.

Stakeholder Participation: A New Process and tts Application to Environmental Decisions

Betty R. Love, Sorin R. Straja, James R. Streeter, Sharon D. Jones, and A. Alan Moghissi*
Institute for Regulatory Science, Alexandria, Virginia

Virtually all government agencies at the federal, state, and local levels and many private organizations realize the benefit of engaging the public in general and stakeholders in particular in their decision-making processes. However, there has been confusion on who is a stakeholder and how various stakeholders are to be included in a decision. In order to ensure that all stakeholders have a reasonable chance to participate in a decision, stakeholders are categorized into several groups consisting of: 1) Personally Impacted Stakeholders; 2) Administratively Impacted Stakeholders; 3) Generally Concerned Stakeholders; 4) Process Concerned Stakeholders; and 5) Remainder of the Public. The purpose of this categorization is to ensure that all stakeholders are reached and their views are appropriately considered in the decision process. Experience with the proposed stakeholder participation process indicates a high-level of agreement on the fairness of the process by virtually all stakeholders.

Key words: Stakeholder participation, stakeholder categorization, stakeholders, peer review

*To whom correspondence should be addressed at Moghissi@nars.org




Technology, Vol. 11, Issue 2
Copyright © 2008 Cognizant Communication Corp.

Metrics for Assessment of Environmental Benefits: Results of an Independent Scientific Assessment

Melvin W. Carter
Georgia Institute of Technology, Atlanta, Georgia

Peter Maggiore
North Wind Inc., Los Alamos, New Mexico

A. Alan Moghissi,* Betty R. Love, Sorin R. Straja
Institute for Regulatory Science, Alexandria, Virginia

Environmental protection is considered to be one of the key challenges of the twenty-first century. However, measurement of success of various actions to enhance environmental protection is a major challenge. This paper reports the results of a major effort to develop metrics for various aspects of environmental protection by convening several panels of independent experts to develop metrics and apply them to selected grants awarded by the U.S. Environmental Protection Agency (EPA) to certain nonprofit organizations. These metrics cover human and ecological health, concentration of pollutants in environmental media, emission of pollutants, and public information. The results of these efforts were presented in a hearing convened by the U.S. House of Representatives.

Key words: Environmental metrics, peer review, peer review at the EPA

*To whom correspondence should be addressed at Moghissi@nars.org




Technology, Vol. 11, Issue 2
Copyright © 2008 Cognizant Communication Corp.

Comparison of Two Peer Review Processes

Sorin R. Straja*
Institute for Regulatory Science, Alexandria, Virginia

Michael C. Kirkland
Aiken, South Carolina

This study compares the External Independent Review (EIR) and the Independent Peer Review (IPR), which are two review types used to evaluate large Department of Energy projects. This quantitative study collected and evaluated data that reflected attitudes of project management personnel regarding the relative effectiveness of the EIR and IPR for addressing a project's technical issues. The Generalized Linear Model revealed that study participants preferred the IPR to the EIR for addressing many aspects of their project's technical issues. Factor Analysis showed that in order to manage the EIR and IPR processes it is necessary to take into account the following five factors: the benefits provided by EIR and IPR; the technical impact of EIR and IPR; the appropriateness and specificity of the review criteria of EIR and IPR; the "value added" of EIR and IPR; and the level of detail and conflict of interests of EIR and IPR.

*To whom correspondence should be addressed at sstraja@nars.org.




Technology, Vol. 11, Issue 2
Copyright © 2008 Cognizant Communication Corp.

Commentary: Peer and Organizational Review in the Post Vannevar Bush Era

Dennis K. McBride
Potomac Institute for Policy Studies, Arlington, Virginia

This commentary concentrates on research and development (R&D) enterprises, with a focus on peer review, primarily as used in manuscript review. This commentary also examines the technology transformation process and how peer review and organizational review serve each other, quite effectively. The key arguments made are that (1) peer review is not without its own problems; (2) it should be understood that peer review is focused largely on the most basic of scientific endeavors (i.e., discovery and not technology development), and (3) technical reporting (i.e., scientific or engineering writing in which there is organizational/vertical review) plays the major role in the actual transformation of ideas into wealth.

Address correspondence to dmcbride@potomacinstitute.org