READ_ME.DOC/READ_ME.TXT

Break Out Plus

Virus Protection Management

You may have to set your Security Level to Medium to have this macro powered Excel program run properly. This means you will be asked if you wish to run the program each time. The Security Level window will also list Virus scanner installed if you have one (and you had better have one!) The virus scanner will then scan the program every time it is run to insure it does not pick up a virus from your computer. Click Tools | Macro | Security | Medium to set your security level. If your campus system strips the macros out of the .xlt program, save the program as .xls for successful operation.

Three Sample Answer Sheet Files

Nurse1.ANS contains answer sheets from a class preparing for a standardized test using traditional right-mark scoring. The test instructions included, Mark the best answer to every question. The pass/fail point was set at 75% to insure mastery by the class. 

BioSci2.ANS contains a 24 student sample from a general studies class with a pass/fail point of 60%. The test instructions included, Mark an answer to every question or mark only when you know the answer. Omit was assigned a value of 0.5 or 50%. These students were struggling; only about half elected Knowledge and Judgment Scoring on their first test.

Fall88.ANS is a complete set of 106 answer sheets from a class in which all but five students have accepted Knowledge and Judgment Scoring and most students are functioning at the mastery level at the end of the semester. Few wrong answers are marked.

When you click Import, the program looks for files ending in .ANS. If none are found, you must find your answer sheet data. Use these files to proof that your computer delivers the correct results. 

Program Installation, Operation, and Licensing


You have three ways to install and operate the software:

1. Copy, unzip, and run the software in a new folder on your PC. Copy the .xlt file to your template folder or a new folder. Save results as .xls files.
2. As above but have a scoring service put the scanner output on a disk or in your account. The scanner output will need converting before importing into the software if one of the above formats cannot be produced. On one campus four different outputs are produced depending upon who is running the scanner and if it is the first or second scan.
3. As above but have the campus scoring service do everything and deliver you the results. You only run the software to quickly correct item or scoring errors.

License:  A free license is granted to individual users of Break Out Plus. A site license, for the number in the group, is required when Break Out Plus is part of a scoring or scanner service provided to a group of users.

Uninstall

Delete the template file and all other files you have produced with it to uninstall. 


Nine-Patch Software

(Nine-Patch Software was sold to Nine-Patch Multiple-Choice, Inc, in April, 2006.)

This is the fourth reincarnation of Nine-Patch Software. Each new major version incorporates new insights from research and classroom experience.  The goal remains to help create better, faster and easier ways of assessing students, test items, and instruction (the main parts of a functional classroom instructional system). A greater emphasis is placed on student counseling. The audience addressed extents beyond students and teacher to professional development personnel (consultants, trainers, and administrators). This generation is so different from the past that new names are used as most of the old programs have been gleaned and condensed into two. The more that is known, the simpler things get. All multiple-choice scoring methods have one common base.

Scoring Methods

Know/dont know. Students only mark an answer to report what they know. They omit (do not mark) to report they dont know. This practice collects the most usable information from multiple-choice tests. The value for omit can range from 0 to 1 (0 to 100%). All other scoring methods can be derived from this one by the value assigned to omit. The starting test score equals the omit value expressed as a percentage. Right and wrong answers are then added to or subtracted from the starting score. This method generates two student scores, one for quantity and one for quality. It generates four classes of item performance, expected and guessing (traditional easy and difficult) plus student discriminating and misconception from the mark distribution.

High-risk. The value for omit (dont know) is set at 75% or higher. Few if any wrong answers are expected. If an examinee is unsure about an answer he/she should omit (on the job he/she should ask for help or clarify the situation before proceeding where an error would be costly or life threatening). Example: omit set at 0.8, starting score is 80%, right answer add 0.2, wrong answer subtract 0.8, pass/fail set at 92%. There are few if any wrong marks. The 2% grade range can be rescaled to 10% to match classroom grades.

Knowledge-and-judgment (K&J). The value for omit (dont know) is set between 30% and 50%. The higher value entices students to switch from traditional to knowledge-and-judgment scoring, from random guessing to reporting what they know or can do at higher levels of thinking. An honest value for omit is always higher than the design value with classroom tests. As a rule of thumb, 0.33 is the lowest value as most classroom tests have, on average, three functional options. Example: omit set at 0.50, starting score is 50%, right answer add 0.50, wrong answer subtract 0.50, pass/fail set at 60% to 75%. The number of wrong marks ranges from none to the same as in traditional scoring (when all items are marked). The range of scores is compatible with traditional scoring. This permits using both on the same test. The judgment score (quality) can range from the knowledge (quantity) score to 100% (no wrong answers).

Knowledge, traditional, or right only (K). The value for blank is set to zero. The student is not asked to differentiate, judge, between what he/she knows and doesnt know, only to mark all items. This method generates a single test score with little ability to tell which right marks are from what is known or from guessing. Traditional example: blank set at zero, starting score is zero, right answer add 1.0, wrong answer subtract 0.0, pass/fail set at 60% to 75%. On a test with four-option items this really is omit set at 0.25, potential score is 25%, right answer add 0.75, wrong answer subtract 0.25. This method produces the maximum number of wrong answers and identical knowledge and judgment scores. It produces the least and the lowest quality information from a multiple-choice test because forced choice (guessing) must be used to complete the test. 

One of Two Software (.xlt) Files

BreakOut.xlt scores traditionally and know/dont know (omit) for the teacher who only needs test scores for setting grades. It is designed to help students and teachers break out of the grip traditional multiple choice scoring has in most schools. For students to improve their performance they need to improve the quality as well as quantity of their thinking, that is, move from lower levels, rote, to higher levels, define, relate, and visualize. The value for omit (chance or judgment) can be set from 0 to 99%. This corresponds to students taking little to full responsibility in reporting what they know on a test that has an honest value for omit. A best estimate for the value of omit is calculated for each test. Use this version to help students make the transition from forced-choice to reporting what they actually know. They can elect which scoring method they are prepared for. Students should be informed that answer sheets are checked for independent marking and your policy for handling answer sheets that fail the check. Valid test statistics and grades can only be obtained from accurate data. This is critical for doing item discrimination analyses.

PowerUp.xlt adds a powerful item discrimination analysis to assist in improving student counseling, test items, and instruction. The descriptive analyses are valid for relating how a student, item, and the class performed on a test. A low scoring student may miss an item most other students marked correctly. This points to the student as the source of the problem. If most other students omitted the item, this points to the item or instruction. If most students marked the item incorrectly, a misconception is being pointed out. If most students omit but those who do mark are correct, the item is discriminating between those who know and those who dont know. It is a good item. There is a failure in instruction or learning. All of this information is printed on one sheet for easy use. The Test Performance Profile item discrimination data can be used to revise items (make predictions) when a total of about 75 students exists or a consensus is obtained from three tests.

Features and Benefits             Break Out Plus Power Up Plus
Software name                      BreakOut.xlt   PowerUp.xlt
Scoring Method                       K    K&J       K    K&J

Imports scanner data                 *     *        *     *
Imports text(.txt) files             *     *        *     *
Manual data entry                    *     *        *     *

On screen corrections                *     *        *     *
Drop bad item                        *     *        *     *
One point to each student            *     *        *     *
Tally without scoring                *     *        *     *

Quantity and quality scores                *              *
Ranked by test score                 *     *        *     *
Ranked by student ID                 *     *        *     *
Score mean, median, mode             *     *        *     *
Standard deviation of test mean      *     *        *     *

Student mark distribution            *     *        *     *
Right answer distribution            *     *        *     *
Average number of options marked     *     *        *     *
Value for omit, chance, judgment     *     *        *     *
Fitness, educated guessing score     *     *        *     *
(E)xpect student performance               *              *
(D)iscriminating                           *              *
(G)uessing                                 *              *
(M)isconception                            *              *

Student counseling mark matrix       *     *        *     *
Score and item difficulty            *     *        *     *
Plus EDGM analysis (above)                 *              *
Plus MUD analysis (below)                           *     *

Test Performance Profile                            *     *
(M)astery/Easy Items                                *     *
(U)nfinished Items                                  *     *
(D)iscriminating Items                              *     *
Item mean, median, mode                             *     *
Standard deviation of item mean                     *     *
Point Biserial r item discrimination                *     *
Kuder-Richardson test reliability                   *
Alpha test reliability                                    *
Test reliability with 50 items                      *     *
Test reliability with 100 items                     *     *

Independent mark matrix (cheating)   *     *        *     *
Independent mark matrix filtered     *     *        *     *

License, single user                   Free           Yes
License, site or multiple users        Yes            Yes

This work started in 1962 with a desire to find out what other people know. The first formalization in 1990 used the patch-work quilt design my mother frequently used: nine-patch. This brought together the three groups directly involved in institutionalized education (students, teachers and administrators) who can each function at three levels of thinking (cyclic, self-correcting; linear, concrete; and random chance). There are nine ways that anything done in a classroom can be interpreted. Finding out what other people know is not a simple matter. When students report a course is both too easy and too difficult or not enough to study and too much to study they are right in respect to their position in the matrix. 

I must first acknowledge the cooperation of some 3000 NWMSU students in the process of empirically developing what became knowledge-and-judgment scoring. It was their reports of success in other classes when they changed from their middle-school study and test habits and developed the sense of responsibility needed to learn and function at higher levels of thinking that indicated we were on the right track. Knowledge-and-judgment scoring gave them the practice and the insight that it is better to know and understand a few things well than try to know a bit about everything. Long term understanding is the basis for building knowledge, not quick rote memory. Know what you know and know what you do not know. You are 96% right in selecting questions to answer for your score of 70%.

I must also acknowledge a unique campus where everyone could program (teach) the campus computer system on campus and off campus.  Time, teaching assignments, and portable computers were provided to complete the project including a Teacher of the Year award. I thank all who had a hand in this.

No final publication was made as the quality of research was too anecdotal for someone trained in the physical and biological sciences. Student quality continually changed from year to year. Fall classes could not be compared with spring classes. The night section that formerly drew less than 30 students not only filled to over 120 but was the first to fill. It became one hour written recitation, one hour lecture, and one hour student project reports with 24 hour online tutoring.

Health prompted early retirement and a change in academic environment from biology students to nursing students or more accurately nurse educators. Again in empirical fashion, additions were made that reflected the needs of serious faculty who prepare students for standardized tests. I must thank the chairperson, faculty and support personnel of the Department of Nursing, SEMSU, for their leadership, support, and constructive critique in creating the Test Performance Profile, an ANA paper presentation, a MLN workshop and a publication in Nurse Educator.

I must thank members of the School of Nursing, Washburn University, for sharing their data, running the scoring software of the day on their personal computers, and a workshop.  These experiences plus the high-risk scoring patent by Knowledge Factor, July 2005, resulted in the current programs.  At last the math that related all forms of multiple-choice, by way of the value for omit, was solved.

I must thank the members of the Educational Software Cooperative, Inc (non-profit) for their support and encouragement as the software was updated and distributed (last Google count had several hundred sites around the world). A special thanks for the Life Time Award, July, 2005 and their continuous vote for Treasurer from the beginning in 1994.

And last and foremost, I must thank my family for putting up with me and at times rescuing me by way of a number of diversions. Setting up their home networks so I could work in my office with the grandkids in three different states made finishing this work possible.

Richard A. Hart, Ph.D.
Professor of Biology, Emeritus
Northwest Missouri State University
803 Somerset Drive
Columbia, Mo 65203-6436
Voice 573-808-5491
Fax 573-445-7275
rahart@mchsi.com
Educational Software Cooperative, Inc
Treasurer, www.edu-soft.org

28 February 2006

(Nine-Patch Software was sold to Nine-Patch Multiple-Choice, Inc, in April, 2006.)

16 October 2007
