Animal & Veterinary
CVM Improves Drug Review, Other Activities Through Project Management “Lessons-Learned” Process
by Madeline Vanhoose, PMP, Director, CVM Project Management Staff, and Petra Garosi, PMP, Supervisory Project Manager, Office of New Animal Drug Evaluation
FDA Veterinarian Newsletter 2007 Volume XXII, No III
The Center for Veterinary Medicine is using project management for several projects, including new animal drug reviews. A key part of project management is the analysis of the project, usually conducted after it is complete, to see what went well and what could be improved for future project performance. The analysis is called the “lessons-learned” process, and it is one of the best ways to ensure that good practices are repeated and bad practices are eliminated. It is also instrumental in making CVM a learning organization and helps build the Center’s intellectual capital.
Lessons learned is a process of documenting the experience gained during a project. Lessons-learned meetings are typically part of the project plan and are conducted near the end of the project as part of the project close-out process. Although lessons-learned meetings are usually held at or near the end of a project, they can also be useful at key interim points during longer projects.
Lessons learned for a project are documented in one or more meetings that involve the project team and may include other stakeholders, including other CVM organizations and executive management when appropriate. The goal is to document the “wins” and the “challenges” by discussing what worked well during execution of the project, what problems were encountered, and what the project manager and the team did to resolve those problems. The lessons learned part of the project can also show where the team deviated from the project plan during plan execution and help the team document the reasons for the deviations.
Lessons learned from projects
The CVM Project Management Staff has conducted lessons-learned exercises on many of CVM’s completed projects and conducted an interim lessons-learned exercise on an ongoing project of longer duration. The lessons learned identified the successes and challenges of the team’s practices in the following major categories.
The project team’s size, mix of skills, and the team members’ roles will all have an impact on the ability of the team to create the project’s deliverables efficiently and effectively, and to work together smoothly.
The processes that the project team establishes for managing meetings, handling issue escalation, and communicating within the team can impact how smoothly the team works together and the effectiveness of its interactions with stakeholders, especially the internal CVM organizations represented by project team members.
Documentation of the project team’s composition and processes should be incorporated into a project team charter at the beginning of the project, so that the team knows what is expected of it and of each team member.
Definition of the scope (i.e., goals, objectives, deliverables, timeline, assumptions, constraints, risks) of a project will help project teams avoid conflict later in the project over what should be in the final deliverable and avoid unnecessary work or rework. The scope of a project also includes the environment in which a project is implemented. For project management purposes, the environment of the project is the other organizations or teams with similar goals and objectives, as well as other external factors that could impact the project.
A project’s scope and environment should be incorporated into a project definition document at the beginning of a project.
Project management tools have helped project leaders and project managers to efficiently and effectively initiate a project, manage the activities of the project against a plan, and support efforts to deal with the ever present challenges created by interactions among the project team members and the stakeholders.
Project planning software tools at CVM (Microsoft Office Project® and Microsoft Office Excel®) have been used to plan projects of different priority, complexity, duration, and scope. Excel has been used to plan smaller, shorter projects with limited scope, while Microsoft Project has been used for larger, complex projects of longer duration.
Project planning template tools created for projects repeated often throughout the Center (e.g., writing standard operating procedures, developing guidance documents, or producing annual reports and FDA Veterinarian) have been used to increase the efficiency of project planning for similar projects.
Project management communication tools (e.g., a project’s charter, project definition document, project plan and schedule) for use in interfacing with stakeholders and internal organizations can support the resolution of conflicts around scope, schedule, and performance of plans.
Project management high-level monitoring tools (e.g., Microsoft Office Project®, Microsoft Office Excel®) can be used to develop “milestone” reports for management. (A milestone represents the achievement of a significant part of the project. When a project milestone is achieved, it is an important step in the progress of the project.) Operational level monitoring tools (e.g., checklists and action item tables) can be used to help track action items for the project team.
Project management process tools for meeting management, decision-making, and team-membership management have been identified as important for smooth operation of project teams and as a way to support conflict resolution in these areas.
Public meetings are used by CVM project teams to communicate with their external stakeholders on issues of common interest and to gain stakeholder feedback. (For example, CVM’s project team that is developing an Animal Feed Safety System has held four public meetings.)
Public meeting preparation requires detailed planning and careful management of numerous activities and is facilitated by the use of checklists and a logistics coordinator.
If the public meeting incorporates special features, such as breakout sessions, which the Animal Feed Safety System Team used for its first two meetings, it is important to conduct training of facilitators to define roles and ensure adherence to the timeline and agenda during the meeting.
Sponsors, ONADE benefit from lessons-learned meetings
One of the more exciting new areas of lessons learned at CVM is the Office of New Animal Drug Evaluation (ONADE’s) use of lessons-learned meetings to evaluate the processes used for recent approvals granted to animal drug sponsors. The lessons-learned process is one of the project management tools that ONADE is bringing to bear on submissions to improve the quality of the submissions and to enhance the abilities of the team leaders and review staff to manage them. It is also a significant communications tool between ONADE reviewers and application sponsors.
Lessons-learned meetings can be held for any significant approvals (new chemical entities, or for supplements for new indications, species, or routes of administration).
ONADE has developed a procedure that outlines the project manager’s role for scheduling, preparing for, and holding lessons-learned meetings with sponsors. The meetings are voluntary. Upon completion of a significant approval, the project manager works with the sponsor and the CVM/ONADE team that reviewed the drug product to determine an appropriate date and time for the lessons-learned meeting. These meetings do not require the sponsors to make official meeting request submissions to the file. A meeting date is selected that provides at least 30 days’ advance notice. The project manager then schedules the meeting (which lasts 2 hours or so) and notifies the sponsor and the review team.
Approximately 3-4 weeks in advance, the sponsor sends an overview of the points it wants to make during the lessons-learned meeting. Sponsors are encouraged to organize their points into two categories—what went well, and what needs improvement for next time.
The project manager schedules a CVM/ONADE-only pre-meeting prior to the lessons-learned meeting, giving the review team time to review the sponsor’s points and discuss any additional items to bring up at the lessons-learned meeting.
The project manager serves as the facilitator for the lessons-learned meeting.
Recently, sponsors of three significant approvals have volunteered to conduct lessons-learned sessions with CVM staff. Discussions at the meetings focused on the areas of protocols, conduct of studies, summarization of statistical analyses (the data package), communication, and best practices for the future.
ONADE, along with the sponsors, has benefited from these discussions. The biggest area of future best practices identified during these meetings has been communication (e.g., best way to communicate during protocol development, best time to come in for meetings, how to handle e-mails, etc.). Other areas are the best ways to handle analyses and the best format for submissions. ONADE encourages sponsors of significant submissions to use this format to help ONADE manage its workload in order to enhance its ability to get safe and effective drugs to market faster.
Lessons learned – the future
In these cases, the lessons learned applied to only one or a few projects, but not across the board to all projects. However, as more key lessons are collected, patterns will emerge. As that happens, some of these lessons can be raised to the level of what, in project management, is called a “best practice.” A best practice statement implies that the benefit can be gained for all projects, not just the few that reported it.
CVM’s and ONADE’s project management staffs will establish processes for converting lessons learned into best practices for the benefit of all staff and project teams and, if appropriate, for incorporating them into CVM’s and ONADE’s project management methodologies.
Project Management Terms
Project managers sometimes speak their own language that others may not fully understand at first. They use several terms that have specific project management meanings. To help non-project managers better work with the project managers, here is a list of terms and their meanings.
Project Management Glossary
Project charter: A document that gives the project team the authority to use organizational resources for project activities and that formally recognizes the existence of the project. The document includes the goals and objectives of the project and lists the members of the project team and their roles. The document is approved by Center or Office Management.
Project close out: A process to provide for completion and retention of essential project and project management records.
Project deliverable: Any measurable, tangible, verifiable outcome, result, or item that must be produced to complete a project or part of a project and is subject to customer approval.
Project documentation: All documents developed and maintained during the project that are placed in storage for retrieval and access as historical data that can be used for future project estimating and planning activities. These documents may include the project plan, project schedule, project progress reports, meeting agendas and minutes, lessons learned, and archived e-mails.
Project environment: The combined internal and external influences, both individual and collective, that assist or restrict attainment of the project objectives. These influences may be either business or project related or may be a result of political, economic, technological, or regulatory conditions. Everything outside the project that delivers input or receives output from the project is considered part of the environment.
Project issue escalation: A process used by a project leader or manager to raise an issue to the next higher level of decision-making authority to ensure that a problem does not linger in the hands of any one individual or group without resolution. The process ensures critical issues or problems are raised soon enough to prevent impacts to the project, and it ensures the appropriate parties are informed and involved in critical decision-making. The project should always strive to make decisions and address issues at the lowest possible level.
Project management best practices: Techniques or methodologies that, through experience and research, are proven to reliably lead to the desired project outcome. A project leader or manager’s commitment to best practices is a commitment to using all the knowledge and technology at his or her disposal to ensure project success.
Project management lessons learned: A review and evaluation of the successes and failures recently experienced during the execution of a project, upon completion of the plan or one of its major milestone, to learn what worked and did not work. The lessons learned results of the review are documented and made accessible to all interested parties as a reference and guide for future project planning and implementation.
Project management templates: A document or file that describes a preset format for a particular type of project or project management activity, used so that the format does not have to be recreated each time a similar project or project activity is undertaken. Microsoft Project templates for frequently repeated project types can be useful for accelerating project planning. Templates can be annotated with instructions to facilitate their use.
Project management: Application of common or specialized knowledge, skills, tools, and techniques to effectively and efficiently carry out project activities to achieve the project goal and objectives within the planned timeframe and budget.
Project milestone report: A report consisting of key events or milestones (critical accomplishments planned at time intervals throughout the project) that is used to monitor overall project performance. The reports usually contain minimal detail and are presented at a highly summarized level.
Project monitoring: Acquiring and analyzing project data on an ongoing basis to determine that the project is on track for timely completion, or so action can be taken when progress fails to match plans and meet objectives.
Project plan: A formal, approved set of documents used to guide both project execution and control. Documents usually include a project definition document, a project team charter, and a project schedule. The project plan documents planning assumptions and decisions, facilitates communication among stakeholders, and documents approved scope, cost, and schedule.
Project schedule: Time-sequenced plan of activities that project managers use to direct and control project execution. Usually shown as a milestone report, a Gantt (a time-related horizontal bar chart) or other bar chart, or a tabular listing of dates.
Project stakeholder: An individual or organization actively involved in the project or that has interests that may be affected, either positively or negatively, as a result of project execution or successful project completion.