Adult Reading Assessment
Development
Stay on top of the latest developments with our blog on the homepage.
This page outlines details about the Assessment Resource Project. Use the following links to jump to the information you are looking for:
Basic Design of the Reading Assessment Resource Tests
Segmentation of Reading Skill Levels
Providing Feedback to Learners
The Assessment Resource Project Description
The Centre for Foundational Learning at Bow Valley College has developed an adult reading assessment resource, referenced to the International Adult Literacy Survey (IALS). This is an easy to use, pencil and paper tool designed to compliment any reading or language course in programs such as literacy, basic education, upgrading, GED, apprenticeship, essential skills, workplace and college.
The tests contained in the resource are based on current research and developed by experts in the fields of reading, literacy, essential skills and statistics. Practitioners and learners alike will find the paper and pencil format and the one hour length of each test very program friendly. The content of the test items will cover the “everyday” of the adult world including items from home, community and work. Tests will include prose and document texts. Doing these tests will give learners and their instructors a sense of accomplishment and an idea of what their skill levels are related to a meaningful reference framework (IALS levels). This cost-effective resource will present the tests in a format that can be photocopied and thus used over and over.
The project has included two piloting phases to test out the material. The initial pilot was conducted internally at Bow Valley College. The second pilot was national and consisted of partners from various organizations across Canada, including colleges, community programs and workplace programs.
Key Elements to the Project
- Contains testing materials for use by instructors of adult classes in programs including: literacy, upgrading, GED, apprenticeship, transition to work and workplace
- Low stakes tests referenced to general IALS levels
- Total of 6 levels of tests segmenting IALS mid-level 1 to low-level 3
- Provides 5 tests at each of the 6 successive reading skill levels
- Simple to use paper and pencil format
- Step-by-step practitioner guide
- Easily integrated into existing reading/language programs
Basic Design of the Reading Assessment Resource Tests
There are six successive test levels with five tests developed at each level; this allows readers a variety of practice in order to master the prose reading and document use skills at each level. The result is a series of 30 paper-and-pen tests that are designed to be written in one hour, in either group or individual settings.
The texts at all levels are evenly divided into three topic categories that reflect everyday reading: home, community and workplace. All texts have been created for this resource and are intended to replicate the types of texts learners would come in contact with in their day-to-day lives. The layout and accompanying graphics are done to support the reading. The resource has been designed in colour as this assists in test situations; part of the design included careful consideration of how these colours would reproduce in black and white, as it is anticipated that most instructors would not use colour photocopies.
All questions are short answer, requiring the reader to input information on an answer line following the question or directly on the text. Their responses can vary from one word to a few sentences, depending on the test level being used. It is not necessary for students to use complete sentences to answer the questions. The reason there are not a variety of question styles; such as oral, multiple choice, or essay, is because the IALS framework that the project is based on uses short answer questions exclusively.
Segmentation of Reading Skill Levels
The broad range of reading skills captured in the International Adult Literacy Survey (IALS) mid-Level 1 to low-Level 3 has been divided into six successive segments. Below is a description of the format of each test level. For more details about specific reading skills at each level, please see the individual posts on the blog portion of our website (links included).
Test A
- Referenced to IALS mid-level 1
- First test in our series
- Each test contains 30 questions
- Each text corresponds to one question
- Test divided into two components: vocabulary and reading for meaning
Test B
- Referenced to IALS high-level 1
- Second test in our series
- Each test contains 30 questions
- Each text corresponds to three questions
- Even divide between prose and document texts
Test C
- Referenced to IALS low-level 2
- Third test in our series
- Each test contains 30 questions
- Each text corresponds to three questions
- Even divide between prose and document texts
Test D
- Referenced to IALS mid-level 2
- Fourth test in our series
- Each test contains 30 questions
- Each text corresponds to three questions
- Even divide between prose and document texts
Test E
- Referenced to IALS high-level 2
- Fifth test in our series
- Each test contains 20 questions
- Each text corresponds to 3 – 5 questions
- Even divide between prose and document texts
- Texts can span more than 1 page
Test F
- Referenced to IALS low-mid-level 3
- Sixth test in our series
- Each test contains 20 questions
- Each text corresponds to 3 – 5 questions
- Even divide between prose and document texts
- Texts can span more than 1 page
Answer keys accompany the resource to assist the instructor in marking the tests. Due to the design of the overall resource, there are some specific considerations when it comes to marking. First is that there are no part marks. Each question is worth a total of one point and is scored as either correct or incorrect. This means that if a question asks the reader to list four people named in the text and the reader writes out three names, it would be scored 0. This reflects the fact that each question is designed for specific skills and by only providing part of the answer the reader may only be demonstrating a subset of skills that correlates to a lower testing level.
Second, this resource is designed to assess reading only and not writing. This means that learners do not have to use complete sentences to answer questions and that errors in spelling and grammar do not result in a loss of a mark, as long as the instructor can identify it as a correct answer.
Depending on the test level being used, the learner will have a total score out of 20 or out of 30 points.
Providing Feedback to Learners
As the purpose of this formative assessment is to provide guidance to the learners about their reading skills, the resource includes an individual learner feedback form. This form is to be completed by the instructor after marking the tests. Each learner would receive a form which contains information about how they did on the test, the skills being assessed at this level, and specific comments from the instructor as to how to improve.
Part of the emphasis of this resource is a scale score which helps learners see their progression through each level rather than focusing on moving from one level to another. Uncommon for most instructors and learners alike, is that with this resource a learner scoring from 21% – 79% should continue working on that same level of skill. As most people are used to using 50% as a guideline, this requires some explanation and understanding.
- 0% – 20% – Learners have not yet mastered the skills at the previous test level. They should continue to work with the previous test level in the series until reaching a level of mastery of those skills.
- 21% – 79% – Learners are using the correct test level for their reading skills. They need to continue working on material at this level and using these skills. The resource and the feedback form will help both the instructor and the learners see progress within this range.
- 80% – 100% – Learners have mastered the skills targeted at this test level and are ready to move on to the next test level in the series.
This breakdown is a result of the design of the tests. The questions at each test level represent a blend of three levels: 60% of questions are at the specific level, 20% are at the previous level, and 20% are at the next level. For example, if a learner is using Segment B, 60% of the questions will assess Segment B level skills, 20% will assess Segment A level skills, and 20% will assess Segment C level skills.
Also included in the resource is a user guide to help instructors understand the purpose of the project and how to use the resource. It covers topics similar to what has been addressed on this page along with more tips on how to get the most out of this reading resource.
This project included two pilot phases to test the material with learners studying in a variety of programs at a variety of levels. The initial pilot was conducted internally at Bow Valley College. The second pilot was national and consisted of partners from various organizations across Canada, including colleges, community programs and workplace programs. For more details on our pilot partners and results from both pilot phases, please see the Pilots page.
There were two different forms of analysis completed on both the questions and texts which were piloted, to determine which test level each question belongs in. All texts and questions piloted in Segments B – F were analyzed. Segment A was excluded due to the nature of the design; there are a limited number of words in these tests, so item analysis would not be effective.
Below, in a simplified form, is an explanation of the two review processes: item analysis and results analysis. An ‘item’ is a question and item analysis refers to the aspects that were taken under consideration when determining the level of difficulty of each question. Each of these aspects was reviewed individually for each question by two different analysts. In total, 177 different texts and 650 unique questions were piloted.
Item Analysis
To understand how to answer a question, an understanding of the text the question is based on is needed. Each text was analyzed for each of the following aspects:
- Readability: the Flesch-Kincaid reading ease score and the estimated grade level of the most difficult word
- Length: the number of characters, words and sentences
- Context: was the subject matter about the home, community or workplace
- Familiarity: was the text completely new or something readers would read on a daily, weekly, or monthly basis
- Type: was the style of writing a type of prose or a type of document or a mixture of various styles (multiple types were considered, for example: descriptive prose, instructive prose, entry documents)
- Process: the steps readers need to follow to determine the answer
After these six elements of the text were determined, each question was analyzed for the following aspects:
- Information structure: if the reader needs more than one piece of information from the text, are those pieces of information in the same block of text or not
- Comparison: if there is a comparison, is it determining similarities or differences
- Distracting information: if there is distracting information inside the text or from background knowledge, is it likely to interfere with determining the correct response
- Response: how many pieces of information are needed to provide a complete answer and is that stated explicitly in the question
- Inference: do readers need to infer anything about the information in the text or about what is required for the answer; if so, what type of inference is needed
- Type of information: is the information in the text and the information required in the answer concrete or abstract
Results Analysis and Final Output
Results analysis refers to recording whether each learner who wrote a piloted test got the question correct or incorrect and interpreting what this means for the question in terms of difficulty. The results of all 330 tests piloted were recorded. The last step was taking the item analysis and merging it with the results analysis to determine to which of the six segments each question fit best; the calculation and interpretation of this was done by Fernando Cartwright.