2023-02947. Revised Carrier Safety Measurement System

AGENCY:

Federal Motor Carrier Safety Administration (FMCSA), Department of Transportation (DOT).

ACTION:

Notice; request for comments.

SUMMARY:

Since 2010, FMCSA has used its Safety Measurement System (SMS) to identify motor carriers for safety interventions. The National Research Council of the National Academy of Sciences (NAS) recommended on June 27, 2017, that FMCSA develop and test a new statistical model. This notice explains FMCSA's analysis and the Agency's proposed changes to SMS, announces FMCSA's preview of the proposed changes, and requests comments and input on the Agency's system to identify motor carriers for safety interventions.

DATES:

Comments must be received on or before May 16, 2023.

ADDRESSES:

You may submit comments identified by Docket Number FMCSA-2022-0066 using any of the following methods:

Federal eRulemaking Portal: Go to https://www.regulations.gov/​docket/​FMCSA-2022-0066/​document. Follow the online instructions for submitting comments.

Mail: Dockets Operations, U.S. Department of Transportation, 1200 New Jersey Avenue SE, West Building, Ground Floor, Room W12-140, Washington, DC 20590-0001.

Hand Delivery or Courier: Dockets Operations, West Building, Ground Floor, Room W12-140, 1200 New Jersey Avenue SE, Washington, DC 20590-0001 between 9 a.m. and 5 p.m., Monday through Friday, except Federal holidays. To be sure someone is there to help you, please call [phone] or [phone] before visiting Dockets Operations.

Fax: [phone].

To avoid duplication, please use only one of these four methods. See the “Public Participation and Request for Comments” portion of the SUPPLEMENTARY INFORMATION section for instructions on submitting comments.

FOR FURTHER INFORMATION CONTACT:

Mr. Catterson Oh, Compliance Division, FMCSA, 1200 New Jersey Avenue SE, Washington, DC 20590-0001, [phone], [e-mail]. If you have questions regarding viewing or submitting material to the docket, contact Dockets Operations, [phone].

SUPPLEMENTARY INFORMATION:

Public Participation and Request for Comments

If you submit a comment, please include the docket number for this notice FMCSA-2022-0066, indicate the specific section of this document to which each comment applies, and provide a reason for each suggestion or recommendation. You may submit your comments and material online or by fax, mail, or hand delivery, but please use only one of these means. FMCSA recommends that you include your name and a mailing address, an email address, or a phone number in the body of your document so the Agency can contact you if it has questions regarding your submission.

To submit your comment online, go to https://www.regulations.gov/​docket/​FMCSA-2022-0066/​document, click on this notice, click “Comment,” and type your comment into the text box on the following screen.

If you submit your comments by mail or hand delivery, submit them in an unbound format, no larger than 8 1/2 by 11 inches, suitable for copying and electronic filing. If you submit comments by mail and would like to know that they reached the facility, please enclose a stamped, self-addressed postcard or envelope.

FMCSA will consider all comments and material received during the comment period and may change this notice based on your comments.

Confidential Business Information (CBI)

CBI is commercial or financial information that is both customarily and actually treated as private by its owner. Under the Freedom of Information Act (5 U.S.C. 552), CBI is exempt from public disclosure. If your comments responsive to the notice contain commercial or financial information that is customarily treated as private, that you actually treat as private, and that is relevant or responsive to the notice, it is important that you clearly designate the submitted comments as CBI. Please mark each page of your submission that constitutes CBI as “PROPIN” to indicate it contains proprietary information. FMCSA will treat such marked submissions as confidential under the Freedom of Information Act, and they will not be placed in the public docket of the notice. Submissions containing CBI should be sent to Mr. Brian Dahlin, Chief, Regulatory Analysis Division, Office of Policy, FMCSA, 1200 New Jersey Avenue SE, Washington, DC 20590-0001. Any comments FMCSA receives not specifically designated as CBI will be placed in the public docket for this notice.

Viewing Comments and Documents

To view any documents mentioned as being available in the docket, go to https://www.regulations.gov/​docket/​FMCSA-2022-0066/​document and choose the document to review. To view comments, click this notice, then click “Browse Comments.” If you do not have access to the internet, you may view the docket in person by visiting Dockets Operations in Room W12-140 on the ground floor of the DOT West Building, 1200 New Jersey Avenue SE, Washington, DC 20590-0001, between 9 a.m. and 5 p.m., Monday through Friday, except Federal holidays. To be sure someone is there to help you, please call [phone] or [phone] before visiting Dockets Operations.

Privacy Act

DOT solicits comments from the public to better inform its processes, in accordance with 5 U.S.C. 553(c). DOT posts these comments, without edit, including any personal information the commenter provides, to www.regulations.gov, as described in the system of records notice (DOT/ALL 14—Federal Docket Management System), which can be reviewed at www.transportation.gov/​privacy.

Background

SMS Overview

In December 2010, FMCSA implemented SMS to identify high risk motor carriers for investigations (75 FR 18256, April 9, 2010). Section 5305(a) of the Fixing America's Surface Transportation (FAST) Act, Public Law 114-94 (Dec. 4, 2015; 129 Stat. 1312) requires FMCSA to ensure, at a minimum, that a review is conducted on motor carriers that demonstrate, through performance data, that they are among the highest risk carriers for four consecutive months. FMCSA and its State enforcement partners also use SMS to identify and prioritize motor carriers for inspections and less resource-intensive interventions, such as automated warning letters.

SMS determines a carrier's prioritization status ( i.e., prioritized or not prioritized) in each Behavior Analysis and Safety Improvement Category (BASIC) based on the carrier's on-road performance and/or investigation results. A carrier's relative on-road performance is indicated by its BASIC percentile. Investigation results reflect if any Acute and/or Critical (A/C) violations are found in a given BASIC during investigations. A carrier can be prioritized for interventions because its percentile is at or above the Intervention Threshold and/or it has one or more A/C violations related to a particular BASIC.

SMS also provides motor carriers and other stakeholders with safety performance data, which is updated monthly, through the public website at http://ai.fmcsa.dot.gov/​SMS. Under section 5223 of the FAST Act, FMCSA removed SMS percentiles and alerts from the public SMS website for motor carriers transporting property. Passenger carrier percentiles and alerts remain publicly available, as well as inspection, investigation, crash, and registration data for all carriers.

SMS quantifies the safety performance of motor carriers using data available in FMCSA's motor carrier database, the Motor Carrier Management Information System (MCMIS). This database includes violations found during inspections, traffic enforcement, and investigations, as well as crash and motor carrier census data. For detailed information on the current structure of SMS, see the SMS Methodology at http://csa.fmcsa.dot.gov. A copy of the SMS Methodology is available in the docket for this notice.

FMCSA's analysis has shown that SMS is effective in helping the Agency identify high crash risk carriers for interventions. FMCSA's SMS Effectiveness Test (ET) found that the group of carriers that SMS identified for intervention in one or more BASICs had a crash rate that was 61 percent higher than the group of carriers not identified for intervention. In addition, the group of carriers that met FMCSA's high risk criteria had a crash rate that was 178 percent higher than the national average crash rate. A copy of FMCSA's ET, which was first published in 2014 and updated in 2018, is available in the docket for this notice.

Section 5221 of FAST Act required that NAS conduct a study of FMCSA's Compliance, Safety, Accountability (CSA) program and SMS. Specifically, the FAST Act required that NAS:

(1) shall analyze—

(A) the accuracy with which the Behavior Analysis and Safety Improvement Categories referred to in this part as “BASIC”)—

(i) identify high risk carriers; and

(ii) predict or are correlated with future crash risk, crash severity, or other safety indicators for motor carriers, including the highest risk carriers;

(B) the methodology used to calculate BASIC percentiles and identify carriers for enforcement, including the weights assigned to particular violations and the tie between crash risk and specific regulatory violations, with respect to accurately identifying and predicting future crash risk for motor carriers;

(C) the relative value of inspection information and roadside enforcement data;

(D) any data collection gaps or data sufficiency problems that may exist and the impact of those gaps and problems on the efficacy of the CSA program;

(E) the accuracy of safety data, including the use of crash data from crashes in which a motor carrier was free from fault;

(F) whether BASIC percentiles for motor carriers of passengers should be calculated separately from motor carriers of freight;

(G) the differences in the rates at which safety violations are reported to the Federal Motor Carrier Safety Administration for inclusion in the SMS by various enforcement authorities, including States, territories, and Federal inspectors; and

(H) how members of the public use the SMS and what effect making the SMS information public has had on reducing crashes and eliminating unsafe motor carriers from the industry; and

(2) shall consider—

(A) whether the SMS provides comparable precision and confidence, through SMS alerts and percentiles, for the relative crash risk of individual large and small motor carriers;

(B) whether alternatives to the SMS would identify high risk carriers more accurately; and

(C) the recommendations and findings of the Comptroller General of the United States and the Inspector General of the Department, and independent review team reports, issued before the date of enactment of this Act.

NAS Study

On June 27, 2017, NAS published a report titled “Improving Motor Carrier Safety Measurement.” FMCSA commissioned this report under Section 5221 of the FAST Act. The report is available at https://www.nap.edu/​catalog/​24818/​improving-motor-carrier-safety-measurement. However, NAS did not complete all the reviews requested by the FAST Act. The NAS report notes, “This study is not concerned with non-SMS aspects of CSA, and it is concerned only with CSMS [Carrier SMS], not with DSMS [Driver SMS], but we will refer to our topic as SMS in the remainder of this report.” The NAS report concluded that SMS, in its current form, is structured in a reasonable way and its method of identifying motor carriers for alert status is defensible. NAS agreed that FMCSA's overall approach, based on crash prevention rather than prediction, is sound. NAS provided FMCSA with six recommendations to improve the system. This notice focuses on FMCSA's actions in response to the first NAS recommendation to develop an Item Response Theory (IRT) model. FMCSA will update its full corrective action plan addressing all six NAS recommendations after reviewing comments to this proposal. The corrective action plan is available on FMCSA's website at https://www.fmcsa.dot.gov/​mission/​policy/​nas-correlation-study-corrective-action-plan-report-congress.

Pursuant to the FAST Act, FMCSA submitted the results of this study to both Congress and the DOT Office of Inspector General (OIG) on August 7, 2017. FMCSA also submitted the corrective action plan required by the FAST Act to Congress on June 25, 2018. Copies of the NAS report and FMCSA's action plan are available in the docket for this notice.

OIG reviewed FMCSA's action plan as required by the FAST Act and on September 25, 2019, provided its report titled “FMCSA's Plan Addresses Recommendations on Prioritizing Safety Interventions but Lacks Implementation Details,” available on the OIG's website at https://www.oig.dot.gov/​library-item/​37465. The OIG made two recommendations for FMCSA to provide additional details to improve the corrective action plan in relation to three of the NAS recommendations. FMCSA partially concurred with both recommendations, stating that the Agency would first decide how to move forward with its prioritization methodology before providing the cost estimates and benchmarks recommended by OIG. The OIG considers the recommendations resolved but open pending completion of planned actions. As one of those planned actions, FMCSA conducted a full review of the IRT model and made a decision on how to move forward with the prioritization methodology, which is described in this FRN. A copy of the OIG report is available in the docket for this notice.

IRT Modeling

The NAS report recommended that FMCSA develop an IRT model and “[i]f it is then demonstrated to perform well in identifying motor carriers for alerts, FMCSA should use it to replace SMS in a manner akin to the way SMS replaced SafeStat.” FMCSA contracted with NAS for the establishment and operation of a standing committee of experts, as well as with subject matter experts with experience in large-scale IRT modeling, to provide advice and guidance to the Agency during the development and testing of the IRT model. The IRT model was designed and tested using inspection data from FMCSA's MCMIS database. The full modeling report titled, “Development and Evaluation of an Item Response Theory (IRT) Model for Motor Carrier Prioritization,” which details the statistical methodologies that were applied in developing and testing the IRT model, is available in the docket for this notice.

The Agency's IRT modeling work revealed many limitations and practical challenges with using an IRT model. As a result, FMCSA has concluded that IRT modeling does not perform well for the Agency's use in identifying motor carriers for safety interventions, and therefore, does not improve overall safety. First, IRT is heavily biased towards identifying smaller carriers that have few inspections with violations and limited on-road exposure to crash risk. When the safety event groups and data sufficiency standards used in SMS were applied to the IRT model, IRT produced similar results to SMS.

Second, IRT does not use Vehicle Miles Traveled (VMT) or Power Units (PUs) to adjust for differences in on-road exposure in the Unsafe Driving BASIC. As a result, IRT identified carriers with much lower crash rates in that BASIC compared to SMS.

Third, IRT modeling is not readily understandable by most stakeholders or the public. IRT's inherent complexity makes it challenging for the industry and public to replicate and interpret results. While SMS results can be reproduced and explained using simple math, IRT requires an advanced understanding of statistical modeling and analysis.

Fourth, a motor carrier cannot independently compute its IRT results. IRT results can be computed only for the entire carrier population. A carrier would not be able to identify how specific violations or areas of regulatory noncompliance impacted its prioritization status or how it could improve its status.

Finally, IRT's runtime is incompatible with FMCSA's operational needs. The IRT model takes four weeks to run as compared to two days for SMS. The long runtime would make it difficult to make even minor changes to the system.

Because IRT is overly complex and adopting the IRT model would reduce transparency without improving safety, FMCSA will not replace SMS with an IRT model. Instead, FMCSA continues its commitment to continuously improving SMS to identify motor carriers that present the highest crash risk through a transparent and effective system.

Changes to SMS

The Agency conducted analyses during the IRT modeling study that revealed areas in which SMS could be improved to better identify high risk carriers for intervention, without the complications inherent in adopting an IRT model. Those improvements include reorganizing the BASICs, now called “safety categories,” to better identify specific safety problems and combining the 959 violations used in SMS, plus 14 additional violations not currently used in SMS, into 116 violation groups. In addition, the changes include simplifying violation severity weights, removing percentile jumps that occur when carriers move into a new safety event group, and adjusting the Intervention Thresholds. FMCSA also previously published proposed changes as part of its efforts to improve SMS (81 FR 69185, Oct. 5, 2016). The previously proposed improvements included moving certain Out-of-Service (OOS) violations to the Unsafe Driving BASIC, segmenting the Hazardous Materials (HM) Compliance BASIC, focusing on recent violations, and updating the Utilization Factor.

Taken together, FMCSA proposes the following combined improvements to SMS: (1) reorganized and updated safety categories, including new segmentation; (2) consolidated violations; (3) simplified violation severity weights; (4) proportionate percentiles instead of safety event groups; (5) improved Intervention Thresholds; (6) greater focus on recent violations; and (7) an updated Utilization Factor.

FMCSA conducted the ET to measure the impact of the proposed changes on potential future crash reduction. In addition, the Agency analyzed other measures such as the A/C violation rate, which measures egregious and systemic safety issues found during in-depth investigations. Thus, a high A/C violation rate among prioritized carriers affirms the ability of the prioritization system to identify carriers that are more likely to exhibit these egregious safety issues. In addition to the safety impacts measured with the ET and A/C violation rate, the proposed changes were guided by FMCSA's continuing commitment to enhance the accuracy, fairness, and clarity of its prioritization system.

A document which describes the newly proposed changes and provides additional analysis to support the proposed changes, titled “Foundational Document” and dated March 2022, is available in the docket for this notice.

Reorganized and Updated Safety Categories

During the development and testing of the IRT model, FMCSA gained valuable insight and concluded that reorganizing the BASICs, now called “safety categories,” could make it easier for FMCSA and motor carriers to pinpoint and address safety issues. FMCSA proposes reorganizing the Controlled Substances/Alcohol, Unsafe Driving, and Vehicle Maintenance safety categories as described below. FMCSA also proposes to segment the Driver Fitness and HM Compliance safety categories to account for differences in carrier operations.

The new safety categories would be: (1) Unsafe Driving; (2) Crash Indicator; (3) Hours of Service (HOS) Compliance; (4) Vehicle Maintenance; (5) Vehicle Maintenance: Driver Observed; (6) HM Compliance; and (7) Driver Fitness. A copy of the complete list of violations in each safety category is available in the docket for this notice and can also be found in Appendix A of the Foundational Document.

Controlled Substances/Alcohol and Unsafe Driving

FMCSA conducted an Exploratory Factor Analysis (EFA) to identify potential new groupings of violations by highlighting statistical relationships between the violations in each BASIC. Controlled Substances/Alcohol has the fewest violations of any BASIC, and those violations are also cited relatively infrequently. The EFA demonstrated that controlled substances and alcohol violations were strongly associated with the Unsafe Driving BASIC and supported removing the Controlled Substances/Alcohol category as a standalone BASIC. The new Unsafe Driving safety category now includes the drug and alcohol violations that were previously captured in the Controlled Substances/Alcohol BASIC.

In addition, FMCSA's analysis found that violations for operating while under an OOS Order issued under the Commercial Vehicle Safety Alliance North American Standard OOS Criteria belong in the new Unsafe Driving safety category. Currently, SMS places these types of violations across multiple BASICs based on the underlying OOS violation. For example, a carrier that had a violation cited against its driver who operated after being placed OOS for an HOS violation and another driver who operated after being placed OOS for a vehicle violation would now have both violations placed in the new Unsafe Driving safety category, rather than one in the HOS Compliance safety category and the other in the Vehicle Maintenance safety category. Moving and consolidating these violations to the new Unsafe Driving safety category would allow motor carriers and enforcement officials to more effectively identify and correct driver-based safety problems related to disregarding OOS Orders.

FMCSA's evaluation of the new Unsafe Driving safety category, illustrated in the table below, showed that this new combined safety category identified more carriers for intervention that were involved in more crashes and had a higher crash rate and A/C violation rate than the groups of carriers identified in the current Unsafe Driving and Controlled Substances/Alcohol BASICs.

Category	Crash rate (crashes per 100 PUs)	Number of crashes	A/C violation rate (violations per 100 investigations)	Number of carriers
Current Unsafe Driving BASIC	10.32	27,255	114.1	12,786
Current Controlled Substances/Alcohol BASIC	5.51	182	84.8	805
Proposed Unsafe Driving Safety Category	10.63	27,550	116.8	13,353

Vehicle Maintenance

Vehicle Maintenance is the largest BASIC in terms of both the number of violation identifiers ( i.e., CFR provisions or unique enforcement codes) included in the BASIC and the number of violations cited during inspections. The EFA results showed that breaking this category into two separate categories would provide greater specificity to help carriers improve and enforcement officials to conduct targeted investigations.

Therefore, Vehicle Maintenance violations would be divided into two separate categories: Vehicle Maintenance: Driver Observed, which includes violations that may be identified by a driver during a pre- or post-trip inspection and/or while operating the vehicle; and Vehicle Maintenance, which includes all other vehicle maintenance violations.

FMCSA's evaluation showed that although splitting Vehicle Maintenance into two separate categories identifies groups of carriers with a lower crash rate in each category, more carriers with more crashes are identified for intervention and those carriers have a very similar A/C violation rate, as illustrated in the table below.

Category	Crash rate (crashes per 100 PUs)	Number of crashes	A/C violation rate (violations per 100 investigations)	Number of carriers
Current Vehicle Maintenance BASIC	8.06	23,675	108.4	18,764
Proposed Vehicle Maintenance Safety Category	7.55	19,039	103.8	11,019
Proposed Vehicle Maintenance: Driver Observed Safety Category	7.44	23,618	109.7	17,167
Combined Proposed Vehicle Maintenance and/or Proposed Vehicle Maintenance: Driver Observed Safety Category*	7.47	31,666	107.1	22,092
* Carriers in this row have percentiles above the 80th percentile threshold in one or both proposed new Vehicle Maintenance safety categories. This row is not the sum of the prior two rows since some carriers are prioritized under both new safety categories.

Segmentation in Driver Fitness and HM Compliance

SMS accounts for differences in carrier operations in the Unsafe Driving and Crash Indicator BASICs by segmenting carriers according to whether they primarily operate Combination vehicles ( i.e., more than 70 percent of their total PUs) or Straight vehicles. Carriers that are not considered Combination carriers are considered Straight carriers. This segmentation ensures that carriers are compared to other carriers with fundamentally similar exposure to crash risk when operating their vehicles. FMCSA tested whether applying segmentation to other safety categories would improve the identification of the highest risk carriers in those categories. Based on its analysis, FMCSA proposes to segment the Driver Fitness and HM Compliance safety categories to more effectively pinpoint safety issues relating to each operation type. FMCSA determined that segmenting HOS Compliance, Vehicle Maintenance: Driver Observed, and Vehicle Maintenance would not improve those safety categories.

In the Driver Fitness BASIC, carriers that operate Straight trucks and similar vehicles have much higher violation rates than motor carriers that operate Combination vehicles. Segmenting the Driver Fitness BASIC into Straight and Combination segments more effectively identifies carriers with higher crash rates in both segments. Although fewer carriers were prioritized for intervention in the Driver Fitness BASIC, the carriers that were removed from prioritization had a lower crash rate, which allows the Agency to better focus on those carriers that pose a higher risk to public safety.

The current HM Compliance BASIC compares Cargo Tank carriers to non-Cargo Tank carriers, but these carriers have fundamentally different operations. A carrier is categorized as a Cargo Tank carrier for purposes of segmentation if more than 50 percent of its inspections indicated the vehicles were Cargo Tanks. FMCSA's analysis found that segmenting carriers as Cargo Tank carriers and Non-Cargo Tank carriers in the HM Compliance safety category in conjunction with adjusting the HM Compliance threshold from the 80th to 90th percentile identifies a group of carriers that has (1) an HM inspection violation rate that is 22 percent higher and (2) an HM A/C violation rate that is 46 percent higher than carriers identified for intervention under the current HM Compliance BASIC.

Consolidated Violations

Over the past decade, the number of CFR provisions or distinct enforcement codes used as violations in SMS has grown from about 650 violations to 959 violations. Most of the new violation codes provide more specific descriptions for existing violations and do not reflect new Federal safety regulations. For example, an inspector could cite an inoperative vehicle brake by citing §§ 393.48(a) (Inoperative/defective brakes), 393.45UV (Brake tubing and hose adequacy under vehicle), or 393.45PC (Brake tubing and hose adequacy—connections to power unit).

FMCSA's analysis during IRT modeling confirmed that similar violation provisions could be consolidated to mitigate differences that result from inspectors citing different violation codes. Grouping similar violations together would also allow motor carriers and enforcement officials to identify and address specific safety issues more easily. The following table shows a summary of the consolidated violations by safety category.

Violations in category	Number of violation provisions/codes in SMS	Number of consolidated groups in new system
Unsafe Driving	* 59	32
HOS Compliance	73	9
Vehicle Maintenance	406	15
Vehicle Maintenance: Driver Observed	N/A	35
Controlled Substance/Alcohol	11	N/A
HM Compliance	369	14
Driver Fitness	55	11
Total	973	116
* Number includes 14 additional violations for operating while under an OOS Order that are not used in the current SMS methodology.

A report titled, “New Prioritization System: Proposed Violation Groups,” which maps the consolidation of the violations, is available in the docket for this notice.

Severity Weights

SMS assigns each violation a specific severity weight that is intended to correlate with the crash risk associated with that violation. The assignment of severity weights to violations in SMS on a scale of 1 through 10 has been criticized as overly subjective. FMCSA tested many different models to improve the severity weights attached to violations in SMS, including models that applied regression analysis and IRT. Based on that analysis, FMCSA proposes to simplify violation severity weights by assigning each consolidated violation group a weight of either one or two. OOS violations and violations in the Unsafe Driving safety category that are disqualifying offenses under 49 CFR 383.51 would be assigned a weight of two and all other violations would be assigned a weight of one. If an OOS violation is combined with a non-OOS violation in the consolidated violation grouping, the consolidated group would be assigned the higher weight of two.

FMCSA's evaluation found that simplifying the severity weights identifies carriers with higher crash rates. This change would maintain the safety focus on those violations severe enough to result in an OOS Order while removing the subjectivity and complications of distinguishing each violation by severity on a scale of 1 through 10.

Proportionate Percentiles

FMCSA places motor carriers into safety event groups in SMS based on their number of inspections and crashes. For example, carriers in the HOS Compliance BASIC with 3 to 10 driver inspections are compared to each other, while carriers with 11 to 20 driver inspections are compared to each other, and so forth. SMS uses violations and crashes to calculate a quantifiable “measure” of a motor carrier's safety performance. SMS then ranks carriers within safety event groups by assigning each carrier in the safety event group a percentile rank that compares their measure to the measure of other carriers in the same safety event group. A higher percentile rank in a BASIC indicates that a carrier has a worse measure than other carriers in that safety event group. Safety event groups allow FMCSA to provide safety oversight of carriers of all sizes. Some carriers, however, have experienced large percentile jumps based solely on a no-violation inspection that places them in a new safety event group.

FMCSA proposes to use a new method of “proportionate percentiles” that will remove sudden jumps in percentiles, which can occur when a carrier moves into a different safety event group. By removing those percentile jumps, FMCSA would be able to more accurately evaluate whether a carrier's safety performance is improving or declining from month to month. The proportionate percentile approach would use safety event groups only to calculate the benchmark median value of each grouping, which would be calculated periodically. A carrier's proportionate percentile would be calculated from a weighted average of percentiles based on those benchmark medians. After the benchmark run has been established, any changes to a carrier's percentile would be based solely on the carrier's own safety performance and would not be impacted by the safety performance of other carriers.

The table below provides an illustration of how proportionate percentiles more accurately reflect a carrier's change in safety performance. For a detailed description of the method used to calculate the proportionate percentiles, see the Foundational Document in the docket for this notice.

Example carrier	Current methodology	Proposed methodology
Carrier with 10 inspections	Measure: 1.51	Measure: 1.51.
	Percentile: 53.0%	Percentile: 67.4%.
Same carrier after receiving 1 additional inspection with no violations (and moving to next largest safety event group, with 11 total inspections)	Measure: 1.37 (↓ .14)	Measure: 1.37. (↓ .14).
	Percentile: 75.0% (↑ 22%)	Percentile: 67.0%. (↓ .4%).

FMCSA's analysis showed that this approach would reduce the number of unexpected jumps in a carrier's percentiles. In addition, the proportionate percentile method would more closely align a carrier's percentile ranking to changes in its safety performance, ensure stable monthly results for carriers, and provide customized results that are specific to the carrier's exact number of inspections or crashes.

Improved Intervention Thresholds

FMCSA prioritizes carriers for safety interventions when their SMS percentiles reach or exceed pre-established levels called Intervention Thresholds. Because higher percentiles represent worse safety performance, a lower Intervention Threshold in a BASIC represents a more stringent safety criterion. FMCSA's ET found that the Unsafe Driving, Crash Indicator, and HOS Compliance BASICs have the strongest correlation to crash risk. Therefore, those BASICs have lower Intervention Thresholds than the other BASICs, at 65 percent for property carriers, 60 percent for HM carriers, and 50 percent for passenger carriers. The Intervention Thresholds for the Vehicle Maintenance, Controlled Substances/Alcohol, and Driver Fitness BASICs currently are set at 80 percent for property carriers, 75 percent for HM carriers, and 65 percent for passenger carriers, and the HM Compliance Intervention Thresholds are set at 80 percent for all carriers.

FMCSA examined whether adjusting the Intervention Thresholds for the Driver Fitness, HM Compliance, Vehicle Maintenance, and Vehicle Maintenance: Driver Observed safety categories could improve the Agency's focus on carriers with the highest crash risk. FMCSA's updated ET continues to show that the Driver Fitness and HM Compliance safety categories have the lowest correlation to crash risk. FMCSA believes raising the Intervention Thresholds in those safety categories, as shown in the table below, would allow the Agency to focus on populations with a greater safety risk.

FMCSA also considered lowering the Intervention Thresholds in the Vehicle Maintenance and Vehicle Maintenance: Driver Observed safety categories. However, because the Agency is now proposing to split Vehicle Maintenance into two safety categories, FMCSA determined that more carriers would be prioritized for vehicle maintenance issues by applying the current Intervention Thresholds to the new Vehicle Maintenance and Vehicle Maintenance: Driver Observed safety categories than are prioritized in the current Vehicle Maintenance BASIC. FMCSA, therefore, does not propose to change the Intervention Thresholds for the Vehicle Maintenance safety categories, as shown in the table below.

Category	Current intervention thresholds			Proposed intervention thresholds
	Passenger carrier	HM	General	Passenger carrier	HM	General
Vehicle Maintenance	65	75	80	65	75	80
Vehicle Maintenance: Driver Observed	N/A	N/A	N/A	65	75	80
HM Compliance	80	80	80	90	90	90
Driver Fitness	65	75	80	75	85	90

Focusing on Recent Violations

SMS currently assigns percentiles in the HOS Compliance, Vehicle Maintenance, and Driver Fitness BASICs if the last inspection in the past two years resulted in a violation. Under this standard, a carrier may be prioritized for intervention even if the carrier had no recent violation. FMCSA proposes to sharpen the focus on carriers with more recent violations by assigning percentiles only to carriers that had at least one violation in the safety category in the past 12 months. This change means that if all a carrier's violations in a particular safety category are 12 months or older, the carrier will not be assigned a percentile in that category.

FMCSA's evaluation showed that this change would result in 1,081 carriers no longer having a safety category at or above the Intervention Threshold and that those carriers had a crash rate that was 13 percent lower than the national average. Removing carriers with no recent violation in those safety categories would allow the Agency to focus its resources on carriers that pose a greater safety risk.

Updated Utilization Factor

The Utilization Factor in SMS helps to account for a carrier's exposure in the Unsafe Driving and Crash Indicator BASICs. Carriers with higher-than-average exposure to safety events, as measured by VMT per PUs, receive an adjustment in those BASICs. The Utilization Factor currently covers carriers that drive up to 200,000 VMT per PU per year. FMCSA's analysis found that more carriers are reporting higher VMT now than when the Utilization Factor was developed in 2009, and the 314 carriers with 200,000 to 250,000 VMT per PU were involved in about three times as many inspections per PU than the national average. This result indicates that these carriers exhibit much higher exposure to inspections than most carriers. FMCSA proposes to extend the Utilization Factor to carriers that drive up to 250,000 VMT per PU in the Unsafe Driving and Crash Indicator safety categories to more accurately account for carriers with increased exposure.

Other Changes Considered and Not Proposed

FMCSA analyzed other potential changes to SMS and determined that they would not improve safety, as described below.

Geographic Variation

A consistent criticism of SMS has been that differences among State enforcement agencies in commercial motor vehicle (CMV) inspection and violation rates may lead to unfair SMS results for carriers that operate primarily in States with higher-than-average enforcement rates. During the IRT model design, FMCSA explored a statistical model to better account for enforcement variation among States. That model is detailed in the report titled “Development and Evaluation of an Item Response Theory (IRT) Model for Motor Carrier Prioritization,” which is available in the docket for this notice.

FMCSA determined that incorporating a model to account for geographic variation would not improve the Agency's ability to identify high risk carriers and would run contrary to the goals of the Motor Carrier Safety Assistance Program (MCSAP), the Agency's grant program to support State and local efforts to reduce crashes involving CMVs. States face varying challenges to reducing crashes due to different road types, congestion, topography, and weather conditions, among other factors. Through MCSAP, FMCSA encourages States to tailor their crash reduction strategies by addressing local conditions and challenges. Applying a model that de-emphasizes enforcement in certain States would disincentivize FMCSA's MCSAP partners from undertaking enforcement initiatives that are intended to address particular safety issues in their States. FMCSA believes that it should encourage all States to continually raise the bar for safety rather than discounting the safety efforts of certain States.

Crash Indicator

The Crash Indicator BASIC applies severity weights to reportable crashes and places more weight on crashes involving an injury or fatality and crashes involving the release of HM than on tow-away crashes. FMCSA analyzed whether removing severity weights to simplify the calculation would improve this BASIC. Because removing the severity weights from the Crash Indicator BASIC has a minimal impact on the group of carriers identified for intervention, FMCSA does not propose to make this change.

FMCSA also studied the impact of raising the minimum number of crashes required to assign a percentile in the Crash Indicator BASIC from two to three. FMCSA's ET results, however, showed that carriers with exactly two crashes have a future crash rate that is more than twice the national average future crash rate. Approximately two-thirds of those carriers were not prioritized in another BASIC, meaning they would not receive any safety interventions from FMCSA if the data sufficiency standard in the Crash Indicator BASIC were increased from two to three crashes. FMCSA has concluded that raising the minimum number of crashes from two to three in the Crash Indicator BASIC would not improve safety. Crashes that are reviewed through FMCSA's Crash Preventability Determination Program and found to be Not Preventable will continue to be excluded from the prioritization methodology.

Preview

With the February 2023 SMS update, the Agency provided a preview opportunity of the system before implementation, as it has historically done with SMS implementation and enhancements, to allow motor carriers, law enforcement, and other interested stakeholders to see the impacts of these proposed changes on measures, percentiles, and alerts. Motor carriers can log in to the preview at https://csa.fmcsa.dot.gov/​prioritizationpreview/​ or through the CSA website or the FMCSA Portal to see how the proposed methodology may impact their prioritization results. The public can view the new methodology using an example carrier. To support the preview, FMCSA will hold a series of question and answer (Q&A) sessions for the industry and the public, where participants will be able to ask questions about the proposed changes and receive real-time responses. All sessions will have closed captioning. The dates and times for these sessions will be announced on the Agency's website. Before the Q&A sessions, participants have the opportunity to view the preview website and additional resources at https://csa.fmcsa.dot.gov/​prioritizationpreview/​_ where they can learn more about the proposed changes and review their results under the proposed methodology. FMCSA encourages all stakeholders to participate in these Q&A sessions.

FMCSA requests comments on the above proposed enhancements, as well as the changes that were considered but are not proposed. In addition, input is requested on other changes that should be considered. Submitters should provide data to support their recommendations.

Robin Hutcheson,

Administrator.