Draft Best Practices
Guide for VAMAS Technical Activities
A continuing goal of VAMAS is to share ideas about successful
practices for the organization of the VAMAS technical committees
or Technical Working Areas (TWAs) and the design and conduct
of round robin intercomparisons. The following ideas represent
a compilation of views expressed by several TWA chairmen
based on their experiences. These ideas do not represent
an all-inclusive list but could form the basis for developing
an internal VAMAS "best practices guide". Comments
and suggestions for additions and/or revisions are welcome
and encouraged and should be sent to the VAMAS Secretariat
(VAMAS Secretariat).
1. Setting up and organization of TWAs
-
It is important when setting up a TWA
to keep the objectives of VAMAS in mind. There must be
an industrial need, and proposers should involve industry
at the outset to establish their "real" needs.
-
Regular feedback from industry is beneficial
in assisting with determining priorities and the output
should be rapidly disseminated to industrial users, who
have not been actively involved, by their inclusion on
the mailing list as "observers".
-
Good communication is essential between
collaborators. This can be achieved through the web or
by e-mail. Continual dissemination of information (particularly
through the web) should help draw in new "participants"
and "observers".
-
To maintain momentum, progress meetings
should be held at least once a year with adequate time
for discussion, minutes issued and actions identified.
Distinct progress needs to be demonstrated between meetings.
It is probably counter productive to hold meetings too
frequently.
-
For large projects it may be appropriate
to form sub-task groups with leaders.
-
It may be beneficial in some cases to
adopt a shorter term and less generic approach on the
scope of TWA subject areas to encourage "new initiatives"
and "new blood" and instill a more dynamic structure
for VAMAS. TWAs may be set up to tackle perhaps only one
project and then disband on conclusion of the activity.
This approach could run alongside TWAs with a more generic
remit but the latter should be subject to regular review
to ensure that momentum is being maintained.
2. Design and operation of intercomparisons
-
Ensure that there is a well defined and
focused objective.
-
A route for the output into a Standards
Development Organization (SDO) for the final document
should be identified as early as possible.
-
It is helpful to prepare a framework at
an early stage of the final document it is intended to
produce. This framework can then be reviewed and updated
as the activity progresses.
-
Run a smaller exercise initially with
a limited number of participants.
-
Keep the work expected from each participant
down to a maximum of about one man week.
-
Ensure as far as possible that the material
used is consistent and uniform.
-
Give the participants extra material to
allow for trials and errors.
-
Ensure that the instructions are written
simply and clearly.
-
Do not add too many "interesting"
side issues.
-
Start with no fewer than 6-8 participants
to allow for a number who will not respond.
-
Make it clear that you will adhere to
deadlines for submission of data.
-
Expect that the analysis will probably
have to cover issues not originally anticipated.
-
Reports and data release should be made
in a timely fashion.
-
All participants should have the opportunity
to review and correct a draft version of a final report.
-
Consult a statistician if there is any
doubt about experimental design or treatment of data.
|
