Review and Search for Items

Searching for Items

FastTest has extensive item search capabilities. You can search for items that you want to find for editing/review purposes, or if you have a test open for assembly, you can search for items to be added to a test.

• The item search options are accessed from the area in the upper right of the Item Explorer bank view (not when an item is already open for editing).
• There is a quick search box immediately available, which searches for the specified string in the item’s name, description, keywords, author, and source.
• The Advanced Search link brings up a new dialog (Figure 3.17) to search on additional fields.
  • This dialog is useful in test assembly, as you can search for items that meet your statistical/psychometric specifications.
  • Advanced searches can be saved by clicking Save at the bottom of the dialog.
  • To access saved searches open the Advanced Search and click Open at the bottom of the dialog.

Figure 3.17: Advanced Item Search Dialog

 

Item Review: Set up

Item Review is an essential step in the development of high-quality assessments.  For example, after an item is written by a content expert, you might need it reviewed by another content expert, a psychometrician, and an English editor, who all sign off on a checklist.

There are several steps to this:

1. Create the Custom FIelds that you want as part of the checklist (e.g., “Language OK” “Key is correct” “Bloom’s Taxonomy”

2. Create the custom Workflow /  Status stages that you want to use (“Content review” “Psychometrician review” “English review”).

3. Assign items to users and set the Status appropriately.

4. Users perform the review.

To learn how to define custom item fields used for review, see Custom Fields on the Workspace page. The workspace administration area is also where the Workflow is defined.

Next, you or Bank Managers should select items and Assign them to users. You can assign to a specific person with a due date and email notification. You should also change the status appropriately.

Performing item review

• To review an item, right click an item in the item list and select Review Item. Users can also filter for items that are assigned to them using the Advanced Search screen.
• A dialog will appear that shows the item alongside the input fields (Figure 3.16).
• Each reviewer will have his or her values saved for each item.

Figure 3.16: Item Review Dialog

• The check box labeled “Automatically save when clicking next or previous” will make reviewing faster by allowing users to enter values and immediately continue reviewing without being prompted to save.
• Users are also provided with a check box labeled “Show Answer Key” and can choose to turn the feature on or off.

The data for a specific custom field can be exported in a CSV for all items in a bank and/or category.

• Simply right click the bank or category containing the items that have been reviewed, and select Export → Review Field.
• A dialog will appear that allows you to select the custom field that you want to export.

Note: only custom item fields used for review will appear here. In the generated CSV, each row represents an item and each column represents a user.

Modified-Angoff Standard Setting

Note that the Item Review functionality above is easily used to automate the complex process of modified-Angoff standard setting.  Set up a review field called Angoff rating like you see before, and require input from 6 or more experts.  You can then Export Review Field for a set of items, and get a complete spreadsheet of Angoff ratings.  This can then be analyzed for inter-rater reliability and other important statistics.  See the Angoff Analysis Tool for a free software to perform this.