Clinical and Technical Questions

  1. How do I use FCE results to determine 8-hour functioning?
  2. The terms frequent and infrequent that are used in the Dictionary of Occupational Titles and in the PDC Chart don’t seem to conform with the usual meaning of those terms. How did that happen?
  3. I am looking for some definitions of frequent, occasional work. The classic time-related DOL definitions are well known. I am aware that rate based definitions are also used; ie, so many lifts per hour or quarter hour. What I need is a reference to substantiate the existence of those rate based definitions.
  4. How were the RPC-I ranges established?
  5. If your client’s RPC-I scores do not fall within the range for each PDC level (suggesting that his/her responses were not consistent), are the reliability and validity of the RPC-T score significantly altered or affected? If so, how should I document these findings?
  6. I am concerned about the recorded blood pressure rates I am seeing in the African American population at my rehabilitation center in Florida. Have you or any EPIC evaluators have come across this problem before?
  7. We recently bought the EpicRehab Hand Function Sort. I was wondering how to answer this question from a client: “Do I answer these questions as if I was trying to do the task with my injured hand or with the hand I would typically use to do the task?”
  8. An arbitrator’s decision in an Ontario case revolving around job simulation functional testing supported the right of the company to do specifically job-related testing when an employee moves from one job to another, provided that the physical demands of the jobs are “significantly” different. However, the arbitrator also ruled that the employee does not have to participate in any prescreening, such as resting blood pressure, completing a PAR-Q, or even seeing his or her doctor. This leaves me and many others in a bit of a bind. As certified kinesiologists we always do a prescreening; doing so is taught in our courses, including the Matheson system.
  9. When an FCE is performed, does the correct diagnosis need to be entered into the record to get an accurate score?
  10. Is there a resting heart rate limit (restriction) for taking the FCE?
  11. With my FCE testing as of late, I have come across several situations that involve the relationships between HR, blood pressure, and subjective pain ratings. I have had discussions with Workers’ Compensation Case Managers suggesting that I should report an FCE invalid due to the fact that the patient continually reports high levels of pain, despite HR and blood pressure being “normal”. Is there any data out there that would help in this situation? To my knowledge, significant research does not exist that suggests that HR and BP should get to a certain level to consider pain and/or effort valid as an entire rating for the FCE. I know that during dynamic lifting, there is the 10 bpm cuttoff to empasize whether the lift is considered valid or invalid, but how does/can this play a role in circumstances where the patient reports that he cannot perform a lift and declines?
  12. Question #33 on the EpicRehab health questionnaire is unclear to me. Some people find it difficult to find time to perform regular exercise. I am not sure whether the intent of the question is to include these people or whether it is to focus on people who have COPD or Fibromyalgia/chronic fatigue syndrome that can’t perform regular exercise.
  13. Can you assist me with suggestions of a protocol for a carrying test? Can these be replaced or assessed from any Hanoun or EPIC results? What correlation does the Hanoun Static Pushing/Pulling testing have to dynamic lifting?
  14. How high should the table be when administering the Doll Chair Assembly Task (DCAT)?
  15. In what order should the DCAT tests be administered?
  16. Will normative data be developed for the Structured Work Activity Groups (SWAGs)?

The following are comments pertaining to PODs and SWAGs (responses are by Dr. Matheson). They refer to the DCAT manual’s Appendix G, which can be found here.

  1. I’m fully into administering the PODS and SWAG assessment tools with my clients and finding it very interesting and helpful. I have started to form a list of typical behaviours seen under each of the 7 or 10 “Demonstrated Abilities”.
  2. I find it important to write under each number at least one behavioural example that supports my Employability Rating. If you are revising the forms, it would be helpful to have a small space under each number to provide behavioural example(s). I think this will force the evaluator to have rationale and support for each individual rating and improve reliability of the overall Employability Rating.
  3. I have also found this prevents double under scoring (i.e. scoring a client down for the same observed behaviour under different sections).
  4. I have been scoring clients as requiring Environmental Accommodation if their behaviour does not support a “Competitive” rating, yet they did not require cueing or guidance (Personnel Accommodation) or Personal Assistance.
  5. I have changed the term Environmental Accommodation in my most recent report to “Environmental Accommodation/Self Adjustment” as I find behaviours do not always reflect something related to the environment. For example, sometimes a client changes their method or makes a self adjustment, thus demonstrating learning as they proceed through the task. I have been scoring clients as requiring Environmental Accommodation if their behaviour does not support a “Competitive” rating, yet they did not require cueing or guidance (Personnel Accommodation) or Personal Assistance. I have changed the term Environmental Accommodation in my most recent report to “Environmental Accommodation/Self Adjustment” as I find behaviours do not always reflect something related to the environment. For example, sometimes a client changes their method or makes a self adjustment, thus demonstrating learning as they proceed through the task. Environmental Accommodation refers to adapting one’s physical environment to improve task performance. Self Adjustment refers to adapting one’s approach (or method of doing) during the task to improve performance (i.e. anticipating and attempting to prevent undesirable circumstances or problems from recurring or persisting).
  6. Upon further reflection, scoring (weighting) would not reflect a client showing a recurring or persist behaviour of this nature. I will mention the Assessment of Motor and Process Skills tool again here to illustrate this concept, as I don’t want to take credit for concepts that I did not develop. With the AMPS, clients are scored down further if the same types of behaviours persist throughout the task and also considers impact on task outcome (e.g. undesirable versus task breakdown).
  7. I just wanted to thank you Len for a very helpful second meeting. Using the term, “Vocational Executive Function” will really help OTs differentiate the difference in our testing (assessment through work sample and situational assessments) to neuropsychologists and how both assessments and professions offer equally valuable findings, although assessment of executive function from a different angle.
  8. (With regard to Environmental Accommodation): I have not included the line under definition of this Employability Rating “works at a speed and with productivity and error rate equal to other workers performing similar tasks” as I have found these accommodations can result in reduced productivity (e.g. use of sit-stand stool can slow work rate or even if client can multi-task they may be slow to shift back to the assembly task).

All responses by L. N. Matheson, PhD

1. How do I use FCE results to determine 8-hour functioning?

This is impossible to do unless you are using a standardized FCE that has been demonstrated to be valid in prediction of full-day workplace tolerance, like the EPIC Lift Capacity. One of the benefits of the EPIC Lift Capacity research that is important to professionals is that it has been demonstrated in peer-reviewed research to be valid as a predictor of full-day workplace tolerance. If the evaluator chooses, it is also possible to conduct a full-day FCE at a particular level of work demand, although you should first perform a standard EPIC Lift Capacity to determine the evaluee’s safe work demand level. Because this is so demanding and expensive, it is rarely undertaken.

Index

2. The terms frequent and infrequent that are used in the Dictionary of Occupational Titles and in the PDC Chart don’t seem to conform with the usual meaning of those terms. How did that happen?

In the original Handbook for Analyzing Jobs (1972), published by the United States Department of Labor (DOL), the strength demands of work were presented in terms of five levels. Each level carried a connotation of maximum number of pounds to be lifted on a frequent basis. However, frequent and infrequent were not defined. In A Guide to Job Analysis (1982), published by the Materials Development Center at Stout Vocational Rehabilitation Institute at the University of Wisconsin Stout, several major improvements of this job analysis technique were introduced, including modification and definition of these terms. Instead of infrequent, the term occasional was substituted, defined as “under 20 percent of the time.” Frequent was defined as “between 20 percent and 80 percent of the time.” Constant was defined as “over 88 percent of the time.”

This is the format incorporated into the Physical Demands Job Analysis (1981) and in the 1982 supplement to the fourth edition of the Dictionary of Occupational Titles. In the 1986 supplement to the Dictionary of Occupational Titles, a further revision of the rating structure for strength was introduced. This is presented in the current PDC chart. Occasionally is defined as up to 33% of the time. Frequently is defined as 34% to 66% of the time. Constantly is defined as more than 66% of the time. In my opinion these references use the term frequency incorrectly. If we think of frequency as a qualifier for work demands in terms of human performance, the appropriate reference is the number of repetitive cycles over a period of time before a rest break. For example, I would define frequency in terms of number of repetitions per minute for the duration of a task, also expressed in minutes.

Unfortunately, the DOL took a position that is relevant for people who are performing job analysis but not the best position for people who are evaluating human performance. I argue that the job analysts should follow the lead of the human performance experts rather than the other way around. I may have contributed to the problem by incorporating these terms and definitions into the PDC Chart, but I thought it necessary to follow the lead of the Department of Labor. Before the DOL had defined these terms in this manner, I had been using the definitions from the National Institute of Occupational Safety and Health (NIOSH).

Index

3. I am looking for some definitions of frequent, occasional work. The classic time-related DOL definitions are well known. I am aware that rate based definitions are also used; ie, so many lifts per hour or quarter hour. What I need is a reference to substantiate the existence of those rate based definitions.

The use of adjectives to describe degrees of frequent is a poor convention. The actual use of frequency in human performance measurement is on a times-per-minute basis for handling or times-per-hour for lifting.

The United States Dept of Labor was ill-advised when they adopted their convention for the DOT and related publications. It is used nowhere else in the world, nor does it have any basis in science or ergonomics or industrial psychology.

Index

4. How were the RPC-I ranges established?

The RPC-1 ranges were based on research that demonstrated the likely scores in each sector for a given total score. It is one more way that you can internally validate the evaluee’s responses.

Index

5. If your client’s RPC-I scores do not fall within the range for each PDC level (suggesting that his/her responses were not consistent), are the reliability and validity of the RPC-T score significantly altered or affected? If so, how should I document these findings?

It all depends. If I get scores that do not fall within the range for each PDC level, I ask the evaluee about item scores that seem inconsistent in the affected sector. Often, I hear a very good explanation that allows me to accept the response as accurate. In these cases, I report that “internal consistency checks identified an abnormally low score in (name the sector). On further inquiry, it was established that Mr. Jones’ balance is also affected, along with right-side weakness as a consequence of hemiparesis due to the stroke.” This enriched information can only come from post-test inquiry. I never miss such opportunities!

On the other hand, if the evaluee gives me a response that suggests less than full effort, I report that “internal consistency checks identified an abnormally low score in (name the sector). Review of selected item responses indicates that Mr. Jones’s responses are suggestive of less than full effort/over-limiting due to fear/discouragement and depression…,” etc. Of course, I would be sure to “constructively confront” the evaluee so that we can make progress in the latter case. Often, a few days after such a confrontation, RPC scores are a lot higher and the evaluee appears to be making much more rapid progress in the rehab program.

Use the item responses, not just the score; the evaluee is able to tell you much more than the score indicates. The HFS test experience is a therapeutic encounter that can be very powerful!

Index

6. I am concerned about the recorded blood pressure rates I am seeing in the African American population at my rehabilitation center in Florida. Have you or any EPIC evaluators have come accross this problem before?

We have a similar problem here in St. Louis. The blood pressure readings for many African Americans tends to be higher than for the rest of the population, for a wide variety of reasons. Nevertheless, the guidelines developed by the American College of Sports Medicine for nonphysician supervised tests are consistent across all racial categories. For this reason, and we do not use a different standard related to racial or ethnic categories. When we have a person whom we are about to evaluate whose blood pressure exceeds the criteria for the EPIC Lift Capacity test, we do not proceed. If it appears that the person’s blood pressure is temporarily elevated due to the stress of the exam or recent coffee intake or other issues, we work with the person to attempt to bring down the blood pressure to a true resting level. If the person’s blood pressure is within guidelines addressed, we proceed. If, after having taken these measures, the person’s blood pressure continues to exceed the limits in the test protocol, we defer testing and recommend to the person that he or she seeks medical care before returning to be tested.

Aside from the safety and liability issues that this would lead to, using different standards would also create many other problems with the acceptability of the test results in legal settings in which employment is considered.

One of the key safety features of this test protocol that has allowed us to provide services without any injuries for tens of thousands of people is strict adherence to the cardiovascular guidelines. If you want to test people with hypertension or other cardiovascular diseases, it must be done under physician supervision, with the ability to respond immediately to cardiac emergencies. In the long run, this will pay off for everybody although in the short run, it will force us to not provide services to a significant proportion of the people who could otherwise benefit from the services.

As an aside, before the EPIC Lift Capacity test was developed, I participated in reviving two of my clients who had experienced cardiac arrest. When I began to work with my team to develop the EPIC Lift Capacity test, one of my goals was to never have to go through that experience again. When we applied for a patent, this particular issue was one of the most important to the patent examiner. I hope that you appreciate that, although this will limit your ability to provide services, and therefore limit your revenue, in the long run it will be the best approach for you to take in your practice.

Index

7. We recently bought the EPIC Hand Function Sort. I was wondering how to answer this question from a client: “Do I answer these questions as if I was trying to do the task with my injured hand or with the hand I would typically use to do the task?”

In response to this question, explain to the evaluee that “This is an indicator of your current level of ability to perform the task in the writtern description. You do not have to do the task exactly as the drawing.”

Notice that the tasks are not described in such a way as to completely specify the manner in which the person is to do the task. This ambiguity is intentional, and allows us to get at the person’s perception of how their functional impairments have produced disability.

In addition, some of the items are designed in such a way that even a person who has mid-cervical quadriplegia is able to perform the task at some level. The example I like to use is HFS Item No. 10: “Turn a lever knob to open a door.” It does not read, “Turn a lever knob to open a door with your right hand.” It is also a lever knob, which was selected over a round knob because a person with mid-cervical quadriplegia normally uses this type knob to open a door. So a person who has a severe right hand injury who indicates that they are “unable” is quite different from a person who has the same severe right hand injury who indicates that they are “able” or “slightly restricted.” This also gets at the person’s perception of how their functional impairments have produced disability.

The HFS approach is based on a subtle but important emphasis in how we look at our patients' impairment, and its impact on their ability to work. Are we more interested in finding out the specifics of the impairment, or whether the impairment has convinced him/her that he/she is unable to work? I have taken the position that the latter is more important. If a person with a severe right hand injury says that he is “able” on Item No. 10, I know that the person is working around his impairment in order to minimize his disability. If, on the other hand, the person says that he is “unable” on Item No. 10, I know that he is taking the opportunity to communicate how disabling the impairment is, without regard to whether or not there are alternative approaches to open the door that would not impact him very much. This is one important indication of symptom magnification.

In practice, I find out if the person is prone to symptom magnification in several ways. One of the first indicators is if a person asks me during the administration of one of the Sorts, “Do I have to do this exactly as the picture shows?” This tells me that the person is thinking in terms of using the instrument to describe his impairment rather than how he is able to perform tasks using whatever strategy works for him. My response to this question is, “Respond like you normally would in your life. I want to find out how you are doing.”

Another way I find out if the person is prone to symptom magnification is by reviewing items afterwards with the person that appear to be more restricted than I expect and asking, “Is there another way that you can do this task that doesn't restrict you so much?” Let me give you an example.

Mr. Smith responds “unable” to item No. 10, “Turn a lever knob to open a door.” He also responds “unable” to Item No. 2, “Sort a deck of playing cards.” and “unable” to item No. 13, “Drink from a bottle of juice.”

Now, I see sitting in front of me a person who is well fed, with his fly zipped up and his shirt buttoned and his shoes tied. Unless he had his butler dress him that morning, I know that he can use both hands and I surmise that he is able to drink from a bottle of juice and open a door. I may even have observed him doing these things in the clinic; using a paper cup to get water from the fountain or going in and out of the bathroom door or front door of the clinic. So, I challenge him by asking why he gave me “unable” on the item that is likely to be easiest, usually item No. 10. In the normal course of events, the patient will back down and admit that this is something that he is able to do with little or no limitation as long as he uses his unimpaired left-hand. I then take the next most easy item, often No. 13 and ask him same question. Again the patient will back down and admit that this is able to be done with little or no restriction. Item No. 2 is placed where it is because it actually is fairly difficult to sort a deck of playing cards unless you have two pretty functional hands. So next we get to that item in my debriefing with the patient, and he says that this is actually difficult because he has a severe right hand injury. I point out that we are interested in what he can do, rather than what he can't do. We talk about this and all of the other patients in the clinic have had the same talk in one form or another, so the patient knows that we are different from the clinic down the block that wanted to know every little detail about what was WRONG and we are different from his attorney who wants him to point out every little detail about what he is UNABLE TO DO. We are interested in having him do the most with what he has left.

Index

8. An arbitrator’s decision in an Ontario case revolving around job simulation functional testing supported the right of the company to do specifically job-related testing when an employee moves from one job to another, provided that the physical demands of the jobs are “significantly” different. However, the arbitrator also ruled that the employee does not have to participate in any prescreening, such as resting blood pressure, completing a PAR-Q, or even seeing his or her doctor. This leaves me and many others in a bit of a bind. As certified kinesiologists we always do a prescreening; doing so is taught in our courses, including the Matheson system.

The basic issue is to separate the need to maintain safety in the evaluation from using medical information to determine the person's ability to work. These must be kept separate.

The standard of care throughout the United States is to screen for basic risk factors that can put a person in jeopardy in a test that is physically demanding. We routinely collect resting heart rate and resting blood pressure and have guidelines that allow us to objectively determine whether or not a person is able to be tested. To not collect such data would leave one open to malpractice liability in this country, notwithstanding the rules and procedures that various jurisdictions may have about prohibiting use of “medical tests” in preplacement screening and other such testing.

Index

9. When an FCE is performed, does the correct diagnosis need to be entered into the record to get an accurate score?

It depends on which functional capacity evaluation you're using, or which test you’re referring to, and whether or not the diagnosis is pertinent to the factors that you’re testing. For example, with a test of grip strength, whether the person has a back injury or not would not be important, but whether the person has a hand injury would be important.

Index

10. Is there a resting heart rate limit (restriction) for taking the FCE?

In every good FCE there is a resting HR limit or guideline; it depends on the FCE. The basic idea is that if you have a too-high resting HR, you are at greater risk for a “cardiovascular event” or other similar problem, such as a stroke, and extra care must be taken to keep you safe.

In FCEs that use the EPIC Lift Capacity test, the resting HR limit is 90 bpm. This is derived from the American College of Sports Medicine guidelines for “non-physician supervised tests”.

The ACSM publishes new editions of their guidelines every few years. Check on their website for the latest.

Index

11. With my FCE testing as of late, I have come across several situations that involve the relationships between HR, blood pressure, and subjective pain ratings. I have had discussions with Workers’ Compensation Case Managers suggesting that I should report an FCE invalid due to the fact that the patient continually reports high levels of pain, despite HR and blood pressure being “normal”. Is there any data out there that would help in this situation? To my knowledge, significant research does not exist that suggests that HR and BP should get to a certain level to consider pain and/or effort valid as an entire rating for the FCE. I know that during dynamic lifting, there is the 10 bpm cuttoff to empasize whether the lift is considered valid or invalid, but how does/can this play a role in circumstances where the patient reports that he cannot perform a lift and declines?

I wish it were so simple.

We need to look at many characteristics of the person and of his/her test data to make a good decision concerning validity. For instance, the chronicity of the person’s pain is important. Often, people with high levels of chronic pain have blunted HR responses to nociceptive stimuli (such as FCE demands), while, in contrast, people whose pain is of recent vintage experience a fear avoidance response in response to the same stimuli.

I look at all of the patient characteristics and also their performance data before offering an informed opinion.

Always remember that the amateurs want you and me to come up with a simple system even they can use when, in reality, it takes seasoned judgment to make good decisions in this area.

Index

12. Question #33 on the EpicRehab health questionnaire is unclear to me. Some people find it difficult to find time to perform regular exercise. I am not sure whether the intent of the question is to include these people or whether it is to focus on people who have COPD or Fibromyalgia/chronic fatigue syndrome that can’t perform regular exercise.

Use this as an indicator of likely fitness.

Index

13. Can you assist me with suggestions of a protocol for a carrying test? Can these be replaced or assessed from any Hanoun or EPIC results? What correlation does the Hanoun Static Pushing/Pulling testing have to dynamic lifting?

The Cal-FCP integrates the ELC with a good carry test. Dr. Matheson’s ToolKit, which has the Cal-FCP video and protocol at no extra charge, can be purchased here. The relationship between IM strength and lift capacity is weak but significant for vertical pulls only, not for horizontal push or pull. For the latter, there is no dependable relationship.

Index

14. How high should the table be when administering the Doll Chair Assembly Task (DCAT)?

We performed the normative study with each subject standing at a table 29 inches high. We recommend a table between 27 inches and 31 inches. If you want to use a high shelf or low shelf or the floor to have a client perform an assembly task that is tied to his or her work demands, this would be acceptable, but the norms should only be used as a guideline.

Index

15. In what order should the DCAT tests be administered?

If the purpose of the testing is to identify problems with executive dysfunction, the Situational Assessment is best administered without prior Work Sample test experience, but I find it acceptable to use the Work Sample on the first day of the evaluation, with the Situational Assessment on the second day in order to select the “just right challenge” for the latter, using Table 18 in the Professional Manual. If I select a sufficiently demanding level from this table, I find that I can easily identify executive dysfunction issues. Used in this manner, the evaluee’s performance on the work sample test can provide the evaluator with guidance in selecting the proper demand level for the Situational Assessment. If I have only one evaluation day and it is important to identify executive dysfunction if it is present, I use the Situational Assessment near the end of the evaluation day without a prior Work Sample, using other performance information from tests that have been completed to help me select a “just right challenge” level from Table 18. If executive dysfunction is not of concern and I am primarily interested in learning about the evaluee’s performance in terms of bi-manual dexterity and coordination, I use the Baby Bear Chair Work Sample test by itself. I often video-record clients’ performance for later analysis.

The Placement test is always performed before the Assembly test when the Work Sample is administered. The Placement test is never used with the Situational Assessment, nor is the parts mat used with the Situational Assessment because we want to require the client to apply whatever organization and problem-solving abilities are available. For example, there are extra parts in the Situational Assessment that are used as distractor parts. If the parts mats were used, the distractor parts would be immediately identified and would therefore not provide a challenge to the evaluee.

With regard to application of the normative data, because there is a learning effect with the assembly tests, the most valid norms for general application are the Baby Bear Chair Placement and Assembly tests. When I am working with clients on an extended basis, after collecting baseline data with the Baby Bear Chair, I can choose to do follow-up testing with any of the three chairs, at which time the appropriate normative comparisons can be made.

With regard to the time lapse between the first and second trials of Placement tests, 90 minutes was used for the normative data study. In a clinical setting, the only reason to wait to do a second trial of the Placement test would be to look at consistency of effort.

With regard to the Situational Assessment and integrating the audio materials, the reiteration of instructions occurs after the “Ready, Set, Go!” provided by the evaluator.

With regard to how much to intervene with cues and assistance, refer to Appendix G, Work Sample Employability Definitions. I generally do not intervene until it is clear that “Personnel Accommodation” is required. After the accommodation has been provided, it is possible to determine whether active assistance is necessary. My own preference is to allow the evaluee as much time as is necessary to complete the task, observing whether there is improvement in performance as the task proceeds; video-recording is helpful here. In the future, I would like to have our users’ group develop guidelines, perhaps using a nomogram or some other simple table to integrate the Employability Rating with the time to completion. I would welcome your help with this.

With regard to the pictures of the chairs, these were developed at the request of a clinician in California who wanted to provide additional visual cueing in order to compare use of the Labeled Photo with the Assembly Manual with the Assembled Model. I will leave it to the users’ group to provide more guidance with this and possible integration of the labeled photos on a formal basis in the future.

With regard to the DCAT Situational Assessment Calculators, in practice you usually will be able to collect data on all of the factors, but when this does not happen, you can use an average score, which the calculators provide. For example, with the Situational Assessment, it will happen that a client will focus on avoiding and identifying errors and perform those at a competitive level, obviating your ability to determine whether he or she can correct errors. This might happen if the Situational Assessment was too simple or the evaluee might have not been able to shift from one task to the other, sticking with the Telephone Message Recording task until it is completed and then returning to the Assembly task. As we mature in our sophistication of evaluation of executive dysfunction, we may resolve this issue, but I want to be careful to not over-structure these tasks so that the experience becomes insensitive.

Index

16. Will normative data be developed for the Structured Work Activity Groups (SWAGs)?

This is not in the works. What you will find in the next few months [Note: This was posted February 2011] will be normative data for the Progressive Occupational Demand (POD) analogue to SWAG 1 Activity 6 and Activity 7. We selected components from SWAG 1 that would be most conducive to normative studies, reconfigured them, and used them for the reliability and validity studies that have been concluded. We will distribute these norms in a new professional manual. We plan to develop a training and certification program that will involve the certificant in testing healthy normal subjects on a test retest basis, aggregate those data and provide them as additional norms, such as was done with the ELC. We will certainly invite guidance from the users' group with regard to which POD and SWAG activities to include.

Index

17. I’m fully into administering the PODS and SWAG assessment tools with my clients and finding it very interesting and helpful. I have started to form a list of typical behaviours seen under each of the 7 or 10 “Demonstrated Abilities”.

I’d like to develop a means to have us all collate examples of typical behaviors because that will help to improve interrater reliability.

Index

18. I find it important to write under each number at least one behavioural example that supports my Employability Rating. If you are revising the forms, it would be helpful to have a small space under each number to provide behavioural example(s). I think this will force the evaluator to have rationale and support for each individual rating and improve reliability of the overall Employability Rating.

I fully endorse the discipline of at least one behavioral example for each rating. I actually video-record all of my clients and when I write the report, I have the video in front of me and also justified each of my ratings that indicate some difficulty with an example. I will revise the rating scales so that there is room for notation of an example behavior, and send the revised forms to you. I will post the revised forms in the CRC section of the EpicRehab.com website for others to use, as well.

Index

19. I have also found this prevents double under scoring (i.e. scoring a client down for the same observed behaviour under different sections).

I am not avoiding using the same example in two constructs and I would love to have a discussion with you about this because sometimes I find that the same behavior contains information that I can interpret as pertinent to more than one construct. Obviously, we need to develop a consensus on this, but I don't want to adopt this as a formal decision rule unless we can actually develop a consensus. That will be for the months to come.

Index

20. I have been scoring clients as requiring Environmental Accommodation if their behaviour does not support a “Competitive” rating, yet they did not require cueing or guidance (Personnel Accommodation) or Personal Assistance.

Yes, I score clients in this category based on the same behaviors as you, reflecting less than a “Competitive” rating but they do not require input from a person.

Index

21. I have changed the term Environmental Accommodation in my most recent report to “Environmental Accommodation/Self Adjustment” as I find behaviours do not always reflect something related to the environment. For example, sometimes a client changes their method or makes a self adjustment, thus demonstrating learning as they proceed through the task. I have been scoring clients as requiring Environmental Accommodation if their behaviour does not support a “Competitive” rating, yet they did not require cueing or guidance (Personnel Accommodation) or Personal Assistance. I have changed the term Environmental Accommodation in my most recent report to “Environmental Accommodation/Self Adjustment” as I find behaviours do not always reflect something related to the environment. For example, sometimes a client changes their method or makes a self adjustment, thus demonstrating learning as they proceed through the task. Environmental Accommodation refers to adapting one’s physical environment to improve task performance. Self Adjustment refers to adapting one’s approach (or method of doing) during the task to improve performance (i.e. anticipating and attempting to prevent undesirable circumstances or problems from recurring or persisting).

I will have to give some more thought to “Self Adjustment” because I do not want to mark people down for learning, which would reflect executive function. What I mean to say is that if a person self-corrects or self-organizes, then there should be no penalty. If the person spontaneously adjusts to the demands of the task after having made an early error, I would not rate that as requiring “Environmental Accommodation”. An example of an Environmental Accommodation would be if the client were to need something like a pencil and paper to develop a checklist or need the parts mat as methods to promote organization. The question I ask myself as the client proceeds is, “would more external structure allow the person to proceed without having difficulty on this construct?” This reflects my earlier experiences in sheltered workshops with the highest functioning clients who, when the task was sufficiently organized, would not require input from others, but who could not work in the competitive labor market because the structure the person needed was somewhat more than would normally be available on a regular basis. The problem for us in these relatively brief situational assessments is that we need to have the client demonstrate first for us that he or she is at a less-than-competitive level before we suggest or make available these sorts of “props”. I think the first pass would be purely a professional judgment call; if confirmation is necessary, repeating the task or a parallel task at a later date with the prop available to find out whether this improved performance would be necessary.

Let’s talk more about the “Self Adjustment” option; perhaps we can have a debriefing after the conclusion of the task to ask the client, “If you were to do this again, are there any tools or guides that you would find useful?” If the person recognizes this, a self adjustment rating would be appropriate. If necessary, confirmation could be provided through a repeat of the task or a parallel task with the accommodation.

Index

22. Upon further reflection, scoring (weighting) would not reflect a client showing a recurring or persist behaviour of this nature. I will mention the Assessment of Motor and Process Skills tool again here to illustrate this concept, as I don’t want to take credit for concepts that I did not develop. With the AMPS, clients are scored down further if the same types of behaviours persist throughout the task and also considers impact on task outcome (e.g. undesirable versus task breakdown).

Yes, such a decision process as the AMPS employs makes a lot of sense to me. We need to consider both the behavior and the effect of the behavior on task outcome, always asking ourselves how this would show up in the competitive work environment and whether the productivity that the person is displaying would be acceptable.

Index

23. I just wanted to thank you Len for a very helpful second meeting. Using the term, “Vocational Executive Function” will really help OTs differentiate the difference in our testing (assessment through work sample and situational assessments) to neuropsychologists and how both assessments and professions offer equally valuable findings, although assessment of executive function from a different angle.

Thank you for your kind comments about the second meeting. I was not so generous with myself and hope to do better at the third meeting. I am glad that you understood the importance of this new term because it is a profound difference from what neuropsychologists and, in turn, all of the rest of us clinicians normally consider. The gap between neuropsychological executive function and vocational executive function is almost too great to bridge, which leads to many false negative opinions when neuropsychological findings are used to predict vocational competence and competitive employment. It is literally true that only about 1/5 of the neuropsychologists I have worked with since the sub-specialty was established are able to interpret their findings in terms of vocational outcome with any degree of accuracy, and these all are neuropsychologists who work in interdisciplinary settings and have input from occupational therapists and vocational evaluators.

Index

24. (With regard to Environmental Accommodation): I have not included the line under definition of this Employability Rating “works at a speed and with productivity and error rate equal to other workers performing similar tasks” as I have found these accommodations can result in reduced productivity (e.g. use of sit-stand stool can slow work rate or even if client can multi-task they may be slow to shift back to the assembly task).

We need to have a discussion about this as a larger community because you are correct in indicating that some accommodations will result in diminished productivity. However, I would like to distinguish diminished productivity as a consequence of executive function separately from diminished productivity as a consequence of pain or weakness or problems with physical endurance. I’m not sure how to handle this and look forward to future discussions. I will make time in the upcoming CRC meeting and would invite you to raise issues such as this.

Index