Jim Kouri
Anti-terrorism: evaluating Federal Air Marshal Service difficult
By Jim Kouri
By deploying armed air marshals onboard selected flights, the Federal Air Marshal Service, a component of the Transportation Security Administration (TSA), plays a key role in helping to protect approximately 29,000 domestic and international flights operated daily by U.S. air carriers.
The US Congress directed the Government Accounting Office to examine FAMS's operational approach or "concept of operations" for covering flights, to what extent this operational approach has been independently evaluated, as well as initiatives FAMS established to address workforce-related issues.
GAO analyzed documented policies and procedures regarding FAMS's operational approach and a July 2006 classified report based on an independent evaluation of that approach. Also, GAO analyzed employee working group reports and other documentation of FAMS's processes and initiatives for addressing workforce-related issues, and interviewed the FAMS Director, other senior officials, and 67 air marshals (selected to reflect a range in levels of experience).
This report is the an unclassified version of a restricted report (GAO-09-53SU) obtained by the National Association of Chiefs of Police's Terrorism Committee. It was originally compiled last December, but was made available this month.
Because the number of air marshals is less than the number of daily flights, FAMS's operational approach is to assign air marshals to selected flights it deems high risk — such as the nonstop, long-distance flights targeted on September 11, 2001.
In assigning air marshals, FAMS seeks to maximize coverage of flights in 10 targeted high-risk categories, which are based on consideration of threats, vulnerabilities, and consequences. In July 2006, the Homeland Security Institute, a federally funded research and development center, independently assessed FAMS's operational approach and found it to be reasonable.
However, the institute noted that certain types of flights were covered less often than others. The institute recommended that FAMS increase randomness or unpredictability in selecting flights and otherwise diversify the coverage of flights within the various risk categories.
As of October 2008, FAMS had taken actions (or had ongoing efforts) to implement the Homeland Security Institute's recommendations. GAO found the institute's evaluation methodology to be reasonable. To address workforce-related issues, FAMS's previous director, who served until June 2008, established a number of processes and initiatives — such as working groups, listening sessions, and an internal Web site — for agency personnel to provide anonymous feedback to management on any topic. These efforts have produced some positive results.
For example, FAMS revised its policy for airport check-in and aircraft boarding procedures to help protect the anonymity of air marshals in mission status, and FAMS adjusted its flight scheduling process for air marshals to support a better work-life balance. The air marshals GAO interviewed expressed satisfaction with FAMS efforts to address workforce-related issues.
Also, the current FAMS Director, after being designated in June 2008 to head the agency, issued a broadcast message to all employees, expressing a commitment to continue applicable processes and initiatives. Also, FAMS has plans to conduct a workforce satisfaction survey of all employees every 2 years, building upon an initial survey conducted in fiscal year 2007.
Although the 2007 survey indicated positive changes since the prior year, it was answered by 46 percent of the workforce, well short of the 80-percent response rate that the Office of Management and Budget encourages for ensuring that results reflect the views of the target population. OMB guidance gives steps, such as extending the cut-off date for responding, that could improve the response rate of future surveys. Also, several of the 2007 survey questions were ambiguous, and response options were limited. Addressing these design considerations could enhance future survey results.
To facilitate continued progress in identifying and addressing issues that affect the ability of FAMS personnel to perform the agency's aviation-security mission, the FAMS Director should take appropriate actions to increase the usefulness of the workforce satisfaction surveys that FAMS plans to conduct biennially.
Such actions could include, for example, ensuring that the survey questions and the answer options are clearly structured and unambiguous and that additional efforts are considered for obtaining the highest possible response rates, according to the GAO report.
© Jim Kouri
February 21, 2009
By deploying armed air marshals onboard selected flights, the Federal Air Marshal Service, a component of the Transportation Security Administration (TSA), plays a key role in helping to protect approximately 29,000 domestic and international flights operated daily by U.S. air carriers.
The US Congress directed the Government Accounting Office to examine FAMS's operational approach or "concept of operations" for covering flights, to what extent this operational approach has been independently evaluated, as well as initiatives FAMS established to address workforce-related issues.
GAO analyzed documented policies and procedures regarding FAMS's operational approach and a July 2006 classified report based on an independent evaluation of that approach. Also, GAO analyzed employee working group reports and other documentation of FAMS's processes and initiatives for addressing workforce-related issues, and interviewed the FAMS Director, other senior officials, and 67 air marshals (selected to reflect a range in levels of experience).
This report is the an unclassified version of a restricted report (GAO-09-53SU) obtained by the National Association of Chiefs of Police's Terrorism Committee. It was originally compiled last December, but was made available this month.
Because the number of air marshals is less than the number of daily flights, FAMS's operational approach is to assign air marshals to selected flights it deems high risk — such as the nonstop, long-distance flights targeted on September 11, 2001.
In assigning air marshals, FAMS seeks to maximize coverage of flights in 10 targeted high-risk categories, which are based on consideration of threats, vulnerabilities, and consequences. In July 2006, the Homeland Security Institute, a federally funded research and development center, independently assessed FAMS's operational approach and found it to be reasonable.
However, the institute noted that certain types of flights were covered less often than others. The institute recommended that FAMS increase randomness or unpredictability in selecting flights and otherwise diversify the coverage of flights within the various risk categories.
As of October 2008, FAMS had taken actions (or had ongoing efforts) to implement the Homeland Security Institute's recommendations. GAO found the institute's evaluation methodology to be reasonable. To address workforce-related issues, FAMS's previous director, who served until June 2008, established a number of processes and initiatives — such as working groups, listening sessions, and an internal Web site — for agency personnel to provide anonymous feedback to management on any topic. These efforts have produced some positive results.
For example, FAMS revised its policy for airport check-in and aircraft boarding procedures to help protect the anonymity of air marshals in mission status, and FAMS adjusted its flight scheduling process for air marshals to support a better work-life balance. The air marshals GAO interviewed expressed satisfaction with FAMS efforts to address workforce-related issues.
Also, the current FAMS Director, after being designated in June 2008 to head the agency, issued a broadcast message to all employees, expressing a commitment to continue applicable processes and initiatives. Also, FAMS has plans to conduct a workforce satisfaction survey of all employees every 2 years, building upon an initial survey conducted in fiscal year 2007.
Although the 2007 survey indicated positive changes since the prior year, it was answered by 46 percent of the workforce, well short of the 80-percent response rate that the Office of Management and Budget encourages for ensuring that results reflect the views of the target population. OMB guidance gives steps, such as extending the cut-off date for responding, that could improve the response rate of future surveys. Also, several of the 2007 survey questions were ambiguous, and response options were limited. Addressing these design considerations could enhance future survey results.
To facilitate continued progress in identifying and addressing issues that affect the ability of FAMS personnel to perform the agency's aviation-security mission, the FAMS Director should take appropriate actions to increase the usefulness of the workforce satisfaction surveys that FAMS plans to conduct biennially.
Such actions could include, for example, ensuring that the survey questions and the answer options are clearly structured and unambiguous and that additional efforts are considered for obtaining the highest possible response rates, according to the GAO report.
© Jim Kouri
The views expressed by RenewAmerica columnists are their own and do not necessarily reflect the position of RenewAmerica or its affiliates.
(See RenewAmerica's publishing standards.)
