VA OIG: End User Concerns with Integrated Financial and Acquisition Management System Training

OIG Survey The OIG issued a web survey to a statistical sample of 400 individuals who attended and passed at least one iFAMS training course.6The OIG received 326 completed surveys, of which 71 employees were excluded for not having used iFAMS since completing the training. The review team estimated projections from the eligible 255 sampled employees’ responses to quantify the results in the estimated 1,221 eligible employees in the population. The OIG survey was different from the FMBTS survey in two key areas: the OIG survey (1) included specific questions designed to solicit end users’ opinions on training content for role-specific tasks and preparation for day-to-day tasks and (2) was administered four months or more after the go-live date for the relevant iFAMS wave.7This timing helped ensure end users had worked with the system. The OIG survey questions were designed to solicit end user satisfaction with the overall training course as well as with the information provided specifically to facilitate the completion of tasks and daily activities in the new system.8Survey questions on overall training satisfaction focused on the general aspects of the course, such as its timing, the resemblance of training screens to live-version screens, hands-on system practice, and the instructor’s ability to answer questions. Survey questions about the training’s facilitation of role-specific tasks focused on end users’ preparation for completing tasks in the new system, whether enough information was provided to perform their duties, and their ability to handle errors or exceptions in the actual system. When the aggregate estimated survey responses for “strongly disagree” and “disagree” met or exceeded 30 percent for a question, the review team identified the collective responses as “a significant negative response” in the OIG survey results. This threshold represents approximately one-third of FMBTS end users having concerns with the training and suggests there are opportunities for FMBTS to improve the training program. The estimated responses to the OIG survey are detailed below. Survey Results: Overall Training Course The estimated survey responses from the statistical sample of 255 respondents indicated that the majority of end users appeared satisfied with the overall training course but expressed specific concerns with the information provided to enable them to successfully complete role-specific tasks and daily activities. The review team estimated that 61 percent of end users found the time between taking the training and using iFAMS for work “appropriate,” 29 percent found it “too long,” and 10 percent found it “too short” (table 1). Regarding the overall satisfaction with the training, the OIG estimated that 41 percent of end users were “very satisfied” or “satisfied,” 34 percent were “neutral,” and 24 percent were “very dissatisfied” or “dissatisfied” (table 2)…

Requested Action The OIG requests that the Office of Management inform the OIG of what actions, if any, are taken to collect additional feedback or modify iFAMS training courses. No further action or recommendations will be made by the OIG at this time. Office of Management Response In the response provided by the Office of Management, the assistant secretary for management/chief financial officer stated that the OIG’s surveys, user responses, and conclusion are all extremely helpful as FMBTS continues to improve and evolve their training approach. He also acknowledged that user adoption is greatly influenced by effective training and is arguably their most important determinant of success. He stated that FMBTS will heavily leverage the information in this OIG management advisory memorandum as they ramp up training for the next wave and will provide a written update outlining the improvements that they make based upon this management advisory memorandum and other feedback. See appendix B for full comments. OIG Response The OIG thanks the Office of Management for their response and for the courtesies extended to OIG staff during the course of this review. The OIG looks forward to receiving the written update.

Read the report here.

🔊 Listen to Story