Appendix E to Part 240 - Recommended Procedures for Conducting Skill Performance Tests

Appendix E to Part 240 - Recommended Procedures for Conducting Skill Performance Tests

FRA requires (see § 240.127 and § 240.211) that locomotive engineers be given a skill performance test prior to certification or recertification and establishes certain criteria for the conduct of that test. Railroads are given discretion concerning the manner in which to administer the required testing. FRA has afforded railroads this discretion to allow individual railroad companies latitude to tailor their testing procedures to the specific operational realities. This appendix contains FRA's recommendations for the administration of skill performance testing that occurs during operation of an actual train. It can be modified to serve in instances where a locomotive simulator is employed for testing purposes. These recommended practices, if followed, will ensure a more thorough and systematic assessment of locomotive engineer performance.

The Need for a Systematic Approach

There are numerous criteria that should be monitored when a designated supervisor of locomotive engineers is observing a person to determine whether that individual should be certified or recertified as a qualified locomotive engineer. The details of those criteria will vary for the different classes of service, types of railroads, and terrain over which trains are being operated. At a minimum, the attention of a designated supervisor of locomotive engineers should concentrate on several general areas during any appraisal. Compliance with the railroad's operating rules, including its safety directives and train handling rules, and compliance with Federal regulations should be carefully monitored. But, in order to effectively evaluate employees, it is necessary to have something against which to compare their performance. In order to hold a locomotive engineer accountable for compliance, a railroad must have adequate operating, safety and train handling rules. Any railroad that fails to have adequate operating, safety, or train handling rules will experience difficulty in establishing an objective method of measuring an individual's skill level. Any railroad that requires the evaluation of an individual's performance relative to its train handling rules needs to have established preferred operating ranges for throttle use, brake application, and train speed. The absence of such criteria results in the lack of a meaningful yardstick for the designated supervisor of locomotive engineers to use in measuring the performance of locomotive engineers. It also is essential to have a definite standard so that the engineer and any reviewing body can know what the certification candidate is being measured against.

Evaluating the performance of certain train operation skills will tend to occur in all situations. For example, it would be rare for a designated supervisor of locomotive engineers to observe any operator for a reasonable period of time and not have some opportunity to review that engineer's compliance with some basic safety rules, compliance with basic operating rules, and performance of a brake test. As the complexity of the operation increases, so does the number of items that the operator must comply with. Higher speeds, mountainous terrain, and various signal systems place increased emphasis on the need for operator compliance with more safety, operating, and train handling rules. Accounting for such variables in any universal monitoring scheme immediately results in a fairly complex system.

FRA therefore recommends that designated supervisors of locomotive engineers employ a written aid to help record events and procedures that as a minimum should be observed for when conducting a skills performance test. FRA is providing the following information to assist railroads in developing such a written aid so as to ensure meaningful testing. When conducting a skills performance test, a designated supervisor of locomotive engineers should be alert to the following:

- Does the employee have the necessary books (Operating Rules, Safety Rules, Timetable, etc.)?

- Are predeparture inspections properly conducted (Radio, Air Brake Tests, Locomotive, etc.)?

- Does the employee comply with applicable safety rules?

- Does the employee read the bulletins, general orders, etc.?

- Enroute, does the employee:

- Comply with applicable Federal Rules?

- Monitor gauges?

- Properly use the horn, whistle, headlight?

- Couple to cars at a safe speed?

- Properly control in train slack and buff forces?

- Properly use the train braking systems?

- Comply with speed restrictions?

- Display familiarity with the physical characteristics?

- Comply with signal indications?

- Respond properly to unusual conditions?

- At the conclusion of the trip, does the employee:

- Apply a hand brake to the locomotives?

- Properly report locomotive defects?

Obviously, the less sophisticated the railroad's operations are, the fewer the number of identified practices that would be relevant. Hence, this list should modified accordingly.

The Need for Objectivity, Use of Observation Form

It is essential that railroads conduct the performance skills testing in the most objective manner possible, whether this testing is the locomotive engineer's initial qualification testing or periodic retesting. There will always be some potential for the subjective views, held by the designated supervisor of locomotive engineers conducting the testing, to enter into evaluations concerning the competency of a particular individual to handle the position of locomotive engineer. Steps can be taken, and need to be taken, to minimize the risk that personality factors adversely influence the testing procedure.

One way to reduce the entry of subjective matters into the qualification procedures is through the use of a document that specifies those criteria that the designated supervisor of locomotive engineers is to place emphasis on. The use of an observation form will reduce but not eliminate subjectivity. Any skill performance test will contain some amount of subjectivity. While compliance with the operating rules or the safety rules is clear in most cases, with few opportunities for deviation, train handling offers many options with few absolute right answers. The fact that an engineer applies the train air brakes at one location rather than a few yards away does not necessarily indicate a failure but a question of judgment. The use of dynamic braking versus air brakes at a particular location may be a question of judgment unless the carrier has previously specified the use of a preferred braking method. In any case the engineer's judgment, to apply or not apply a braking system at a given location, is subject to the opinion of the designated supervisor of locomotive engineers.

A railroad should attempt to reduce or eliminate such subjectivity through use of some type of observation or evaluation. For railroads developing any evaluation form, the areas of concern identified earlier will not be relevant in all instances. Railroads that do not have sophisticated operations would only need a short list of subjects. For example, most smaller railroads would not require line items pertaining to compliance with signal rule compliance or the use of dynamic brakes. Conversely, in all instances the observation forms should include the time and location that the observer started and ended the observation. FRA believes that there should be a minimum duration for all performance skills examinations. FRA allows railroads to select a duration appropriate for their individual circumstances, requiring only that the period be “of sufficient length to effectively evaluate the person.” In exercising its discretion FRA suggests that the minimums selected by a railroad be stated in terms of a distance since the examination has to be of a sufficient duration to adequately monitor the operator's skills in a variety of situations. FRA also suggests that the format for the observation form include a space for recording the observer's comments. Provision for comments ideally would allow for the inclusion of “constructive criticism” without altering the import of the evaluation and would permit subjective comments where merited.