Transcript
Transcript: Credible Costing in Decision-Making
[00:00:00 Animated CSPS logo appears on screen. Text on screen: Welcome.]
[00:00:07 Jocelyn Charron appears full screen]
Jocelyn Charron: Good morning, everyone. Good morning, everyone. If you're joining from Eastern Canada, maybe it's even earlier wherever you are across our beautiful country. My name is Jocelyn Charron. I am the Executive Director, Financial Management Community and Oversight, at the Office of the Comptroller General. I am pleased to present our latest session on the fundamental principles of financial management.
[00:00:37 Text appears on screen: Jocelyn Charron, Executive Director, Treasury Board of Canada Secretariat]
Jocelyn Charron: We've held four sessions to date. This is our last one. Welcome everyone. Happy to be with you this morning. Today, this morning we are looking at: Using credible information on costing for decision-making. A subject that fascinates me. For those who don't know, I joined the public service as a costing specialist. So I worked in the private sector doing costing for profit management.
[00:01:11 Jocelyn Charron appears split screen with the session title slide, as described.]
Jocelyn Charron: Decision-making within the company. So this is a subject that I am very passionate about and I am happy to be here with you to help to guide the conversation and hope that you will get some important information and perhaps find it exciting.
Before starting, I would just like to do a little land acknowledgement. So, I want to highlight that I am joining you from the traditional, unceded territory of the Algonquin Anishinabe Nation, from the Ottawa region.
[00:01:41 Jocelyn Charron appears split screen with First Nations land recognition slide, as described.]
As many of you are joining us from different parts of the country, I also invite you to take a moment to recognize the traditional Indigenous territory on which you are standing.
It's always nice to join across the country and as well to remember and celebrate the ancestral territories in which we are joining.
[00:01:41 Jocelyn Charron appears split screen with slide, as described.]
Jocelyn Charron: So, a quick reminder that today's session will be bilingual. There will be segments in French and also segments in English. We also are pleased to offer an interpretation service. To access this service, please use the interpretation feature of your virtual meeting platform and select your preferred language. I will also give notice during the session for when we switch to French, when we switch to English, just to give you a little heads up. But generally speaking, the material is also available in both official languages. So you can refer to it too, if you want to have the material on your screens in your preferred language during the session, and then obviously it will also be projected on the screen with us.
We're going to use Wooclap. So Wooclap is a way for us to interact with you. To join, you can go directly to www.wooclap.com and the access code is "26 costing." We are going to ask you some questions. So you need to be paying attention, and also let us know a little about who you are. It will also be a way for us to communicate with you. We have a few questions for you during the session.
The session consists of three presentations and also a follow-up discussion. We will have a question/answer format. I will also let you know when to send us your questions during the session.
[00:03:52 Jocelyn Charron appears split screen with slide, as described.]
Jocelyn Charron: So, we know the skills compass. You can see it here on the screen. The skills compass is a tool that was developed by the Office of the Comptroller General and it was approved by the Comptroller General, Annie Boudreau, last April. It is really for the financial management community but also for several other communities. It is rooted in extensive national and international research, as well as extensive consultation. It is a bit like trying to provide a dictionary of the skills we need as financial professionals to help us to constantly develop, grow and also be a high value-add community for the public service.
You can see it here on the screen. Today's session will essentially help us to add some emphasis and deepen your understanding of the situation.
the situational awareness, which is important because we need to understand the context in which we operate to be able to provide the strategic advice. Also with an innovative spirit, because when we look at costing, it is clear that we need methods. We could perhaps even use AI, we could use data analysis. So, a lot of innovative thinking, innovative tools for costing. We're going to emphasize that too, those skills. Leadership skills in terms of changes are also important, because costing is something that evolves over time. So it's important to remain relevant. So, we're going to talk about that and also about compliance, because ultimately we need to make sure that it is grounded in sound principles and also in integrity. All of this will be part of the skills that we will emphasize in the session. Then I also take a moment to explain the compass for those who haven't seen it yet, and ultimately to promote it a little more, to market the compass at the same time.
I'm going to proceed with the Wooclap. So I told you about Wooclap earlier.
[00:03:52 Jocelyn Charron appears split screen with wooclap survey screen, as described.]
Jocelyn Charron: So I would ask you to join me for a moment. We wanted to know a little bit about what kind of audience we have today. This is information that is useful for us when we conduct these sessions. This is the fourth session, as I told you.
So in each session we want to know a little bit about who is here, who is curious.
So here we are on the screen, and slowly, your information will appear. I'll give you a moment to enter your information. I wanted to know, we have many financial management experts, nearly 50 percent of our audience, but also experts in procurement, costing, so costing experts. You may have some pointed questions for us later, and also for internal verification. We also have people joining us from other communities. So, people who are curious, who want to learn from us this morning. It's wonderful.
So now we know a little bit about who you are. You will also get to know who our experts are for the conversation this morning.
As I was explaining, this is our fourth session. The intention of the series was really to try to increase essential knowledge in financial management but also, at the same time, to build bridges, because when we look at the function of comptroller, we often see that when we do the costing, it is often because we want to manage a big project. A project that may take several years, which will aim to deliver an important service or product for Canadians. So, at the same time, sometimes when we're going to do a project we need procurement. It's a bit like when you look at the comptroller function, there are a lot of links woven into it. You each have your own areas of expertise and the magic happens when we combine our areas of expertise to really ensure that we give the decision-maker the best information to support their decision and the strategic value.
So for me it's important that you understand this, that you realize that we are a team and, basically, that we want to have a knowledge exchange from one field of expertise to another.
Thank you, we can see that financial management is taking the lead, but there are also other curious people in the room. So, thank you very much.
So I told you that the session has three parts and we have three experts with us.
[00:08:48 Jocelyn Charron, Valerie Seeto, Michael Hanlon and John Kealey appear in video chat panels]
Jocelyn Charron: We have Valerie Seeto. Valérie is one of my colleagues. She is the senior director of the Costing Expertise Centre at the Office of the Comptroller General. She is the senior director who looks closely at the Treasury Board of Canada Secretariat submissions that will be presented to the minister and for which we have very complex costing, probably the most complex in the public service. So she really guides the expertise, alignment, tools, and frameworks to ensure the function is complete and fully functional, and she also sees a lot of submissions to the Treasury Board of Canada Secretariat. Very interesting indeed. I also find this interesting because Valérie is an engineer. So, chartered accountant, I'm an accountant, so where do we come from? Perhaps we could complement each other in terms of our expertise. She will present the Treasury Board of Canada Secretariat's point of view in the presentation.
We also have two other speakers who will give us a bit of the ministerial side of things. So what happens in a ministry when we talk about costing? We have Michael Hanlon, a manager in the human resources department at the Canada Revenue Agency's Centre of Expertise. So, a ministry where there is also a lot of costing. It's a matter of practicality; good costing is important for decision-making.
We also have John Kealey, Senior Director at the Department of National Defence (DND), where he leads the development of costing for major capital projects and also supports presentations to the Treasury Board of Canada Secretariat, so presentations to Cabinet.
Obviously, when we think of the Canada Revenue Agency and the Department of National Defence, we can imagine the complexity of costing, the information that must be taken into account to make informed decisions. How many small margins of error could add up and possibly change these decisions by a few million dollars, that type of decision-making. So this is the information we're going to share with you.
But before we get to our speakers, let me introduce Samantha Tattersall.
[00:11:08 Jocelyn Charron, and Samantha Tattersall appear in video chat panels]
Jocelyn Charron: Samantha Tattersall is one of our colleagues. We have three Assistant Comptrollers General who report to the Comptroller General. So Samantha is one of those three Assistant Comptrollers General. Her role is really to provide strategic direction regarding investment management, so really at the level of procurement, equipment management, costing, of course, project management and also capital costs. She is here today because she is our most senior executive in the federal government who provides strategic direction for costing. Who better to give us the opening words?
Thank you for being with us this morning, Sam, and over to you.
[00:12:00 Samantha Tattersall appears full screen]
Samantha Tattersall: Thank you. I didn't think about it, Jocelyn, when you said I'm one of three. Maybe I'm one of the three musketeers, I'm not sure. But good morning and welcome everyone. As Jocelyn said, my name is Samantha Tattersall.
[00:00:37 Text appears on screen: Samantha Tattersall, Assistant Comptroller General, Treasury Board of Canada Secretariat]
Samantha Tattersall: I'm the Assistant Controller General here at TBS, and I'm responsible for investment management, including project costing through the Costing Center of Expertise, and you'll hear from Val in a moment. So thank you for joining this Financial Management Essential series on Credible Costing in Decision-Making. And starting to set the stage for this session, I wanted to share a few observations from a costing diagnostic we conducted last spring across the government of Canada. So as part of this diagnostic, we interviewed 90 individuals from 18 organizations. And in addition, we sent out a survey and 99 organizations responded to it, and there are three key lessons that came out of all of it. And I hope that can help frame how you approach costing. So the first one is we should not cost to funding, but rather cost-informed decision-making. Second, there's merit to making use of cost estimate ranges that integrate risks into them. And the third is that we need timely and clear communication of costing information that decision-makers, not the costers, but the decision-makers, so the people like me or ministers, can understand.
I'm going to touch on each of these three in a bit more detail. I want to start with the point that creating realistic budgets isn't just a numbers exercise. It sets the costing frame for the life of an investment. The first cost estimate people hear really matters. It sticks and shapes expectations. This is what we call an "anchor bias". It's important to set a solid project baseline, one that's clear on what the project includes, so that's your scope, and how long it will take, that's your schedule. Too often, budgets start with how much money is available, not what the project will actually cost. When funding drives costing, it creates unrealistic expectations from the start. So why does that matter? When the financial information assumptions aren't clear or complete, this can cause confusion, unexpected costs, and reduce public trust. But I realize it's not that simple, because early on in a project, there's a lot we don't know, particularly for large, complex projects, and so I'm hoping John will talk about that with DND. With those large, complex projects, the cost and confidence may not be high. As a result, a single dollar value can give a false sense of precision, even when there's caveats provided.
A cost estimate range can show how cost may vary as risks play out over the project's life. It can also help give a more honest picture and in turn, help build trust. Sometimes a flat percentage is added as a buffer, and it feels straightforward, but in practice, we may miss early warning signs and the real risks. So instead of a flat contingency, you should be tying your contingency and cost estimate ranges to specific named risks and the quantified cost impacts. As risks change and you learn more, in turn, you should refresh the range. I can't understate this enough. Costing, and I see this a lot, is not a one-time exercise. I see a lot of departments that do a cost estimate to get an approval, like a Treasury Board approval, and then they don't actively update it. I cannot stress this enough: costing is a continuous process that supports sound decision making. On the same vein in the sense of not locking into a single number, you should be thinking about when the right time may be to share numbers publicly, as well as how you can be transparent about what's included, what's not, and what may cause the cost estimate to change.
Cost increases are often not the result of necessarily poor cost estimation. Rather, cost increases are usually a symptom of what is happening on a project. Some drivers are outside of our control, like inflation and global supply issues. However, some cost increases are a result of things we do control, and this includes our ability to actively manage the scope of the project. So You need to think about delivering in phases and building an exit point so you can adapt if conditions change. To keep everyone aligned, set clear roles, manageable steps, balanced risk sharing, and incentives tied to outcomes.
I would also point out that active vendor management is also a key factor in how effectively we manage our costs. I was happy to see some people here from procurement on the call because it can be a key factor. Your ability to communicate early and clearly with decision-makers: the known risks; uncertainties; and mitigation plans is important. Likewise, when risks materialize, you will also be judged by how they are actively managed. Val has heard me say this many, many times. We need to move away from the discussion that starts with "There's a cost increase or a cost overrun because..." and get into: "These are the potential risks on this project. And if [they] materialize, this is what we can do and are doing to manage those".
So one final thought. Building costing capacity is going to take time, but also collaboration across and within organizations. I'm also happy to see that there are people from procurement, costing, finance, other on this call. Because what I mean by that is costing is not the responsibility of just the Cost Estimators. It's the responsibility of the Project Managers and Sponsors, those who provide the inputs into the cost estimate, and those are the people who control how project costs are managed. We need to work together, the Cost Estimators and the Project Managers, to make sure we provide better and clearer information on costs, and then make the appropriate decisions at the right time to manage those costs. I'll just conclude by saying thank you very much for allowing me to open this event and speak to you today. I'm now going to turn it over to Valerie Seeto for today's session.
[00:19:18 Jocelyn Charron and Samantha Tattersall appear in video chat panels]
Valerie Seeto: Thank you, Samantha.
[00:19:23 Valerie Seeto appears split screen with the title slide]
Valerie Seeto: Hello everyone and thank you for joining us today. I am Valerie Seeto, Senior Director, manager of the Centre of Expertise on Costing at the Office of the Comptroller General of the Treasury Board of Canada Secretariat.
[00:19:45 Valerie Seeto appears full screen]
Valerie Seeto: It's my pleasure to be here. In general, resources are shifting, projects are becoming more complex, and digital delivery is accelerating. All of this raises expectations for credible costing. Whether you contribute to costing, either through data,
[00:20:08 Valerie Seeto appears split screen with Learning Objectives slide, as described]
Valerie Seeto: Assumptions or analyses, here is the key point. Costing is not just a technical exercise, it is a strategic lever. This information supports better decision-making, clarifies the trade-offs to be made, reduces surprises and strengthens ongoing cost management.
You might be wondering: How do I perform costing correctly? At the end of this session, we hope to equip you with practical knowledge and concrete tools to increase the value of your costing work.
Let's begin. Next slide, please.
[00:20:55 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: A quick heads up, our next Wooclap question is at the end of this slide. Those who are more forward-thinking can already go and take a look at it—purely for strategic purposes, of course. Before getting into the details, let's first establish a common understanding of what we mean by costing and how it is approached within the Government of Canada.
In simple terms, costing is about estimating how much something will cost before it is done. At the GC, costing is a strategic tool that supports decision-making. For example, to prepare options, plan budgets, or understand what will be needed to achieve results. We therefore see costing as a continuum; on one side there is cost accounting. We look back at what was actually spent in order to understand the costs and manage the budgets. In general, we are quite strong in this area.
On the other hand, there is the cost estimation. We are looking ahead to assess what we will need in the future. It's more difficult because there are more unknowns. In the federal public service, costing serves to support a wide variety of decisions and can take different forms, depending on our role.
So, a quick question: How do you work with costing-related information in your role? That's our Wooclap question. So I'll give you just a moment to add your answer, then we'll move on to the next slide.
[00:23:02 Valerie Seeto appears split screen with wooclap survey screen, as described]
Valerie Seeto: Once everyone's answered, we'll take a quick look.
So costing is like a team sport and a feedback loop. Other functions contribute to building the estimate, and once it is established, costing helps these same functions to make better decisions. We can already see that we have a mix, a good mix, of how people work with that information. That's great! Thank you. Organizations are responsible for their costing, but the models used vary within government.
One thing is clear, and as Samantha mentioned, our diagnostic last spring showed that the majority of organizations have centralized or partially centralized teams often within the chief financial officer's sector, i.e., the CFO. One thing is clear: to be credible, costing can never be done in isolation. Here's how the parts fit together.
There is the finance team which can play the role of costing designer, structuring the estimate, grouping inputs and translating them into dollars. They can also play a role in questioning, verifying calculations, validating assumptions and supporting the CFO's certification.
[00:24:43 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: Program and project teams know field work better than anyone; they bring the reality, the scope, timeline, dating approach, constraints and operational details that costing relies upon. In turn, costing helps them compare options, plan implementation, and understand the trade-offs to be made.
The data teams provide the evidence upon which the entire costing is based.
[00:25:19 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: Actual spending, volumes, unit costs, benchmarks and trends improve assumptions and help quantify risks.
Inversely, costing helps them focus on the data that matters for decision-making. The digital IM-IT teams help define the technical scope and complexity, especially when they provide expertise to build a digital solution for the GC. This is essential because digital choices strongly influence costs. In turn, costing helps to evaluate design options and life cycle impacts. Across all these functions, we all build or use the same cost estimate.
Quick question: What makes you confident in this estimate? This is what we will examine now.
[00:26:23 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: The six criteria that make costing credible and capable of informing decision-making. So these six criteria help us to determine credibility. Why is this important? Our spring survey confirmed that it is the CFO who approves costing in all organizations. It's someone… in some organizations, it's a joint approval where the CFO and the designated senior manager sign together. It is therefore directly linked to the collective management of financial affairs and leadership in investment matters. These criteria also support the CFO's certification, that is, the ability to certify the proposal, risks, resources and financing and to confirm that compliance and controls are in place. In simple terms, here are the six criteria:
So the first one is focused on the process. The importance of using a clear, structured and reproducible process that can be applied uniformly over time.
Secondly, it is exhaustive in order to open up the full scope of the project so that decision-makers have an overview.
Third is evidence-based, supporting assumptions with comparable data, facts, analyses and experiences, not guesses.
The fourth assesses risks and uncertainties since they can affect costs. Present this as a cost estimate range so that decision-makers can see what might vary and to what extent.
The fifth one: validations. Compare the cost estimate to historical data, similar projects, or even an independent estimate to strengthen controls.
And finally, the last one: document all key elements, information, scope, assumptions and constraints to ensure traceability and compliance.
In short, these criteria make the costing credible and strengthen the confidence of the CFO during the approval process. Let's move on to the next slide, please.
[00:29:01 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: I will continue in English.
We just covered what makes a cost estimate credible. Here's how we actually build it using the GC's repeatable seven-step process. It scales to the decision, complexity, risk, and materiality, and it is iterative.
As projects mature and new information comes in, expect to loop back and refine.
We start by being clear on the objective, what's in and out of scope, who is involved, whether the scope is fit for purpose. Then break the work into logical pieces, including the planning, design, implementation, operations and maintenance and disposal, checking that nothing is missing or double-counted. When developing the cost estimate, be clear and be specific. The ground rules and assumptions are important. Keep a log of baselines, constraints, metrics, hypotheses, and approvals upfront. For those who challenge costing, ask questions. When scope shifts or new data arrives, return to the assumptions to update them, and again, refine the cost estimate range. Ensure that data and evidence are available. This includes actuals, volumes, unit costs, benchmarks. Clean it, normalize it, and reconcile totals. Another important piece is quantifying risks and uncertainties. Separate what might happen, which are your risks, from what will happen, but isn't certain in terms of its impact, your uncertainty. Then show a cost estimate range: low; most likely; and high, and explain the drivers behind that range. Equally important, assure and validate. Ensure the math and logic makes sense, compare against history or an independent review, and confirm controls and alignment to the CFO attestation.
Finally, tell the story in plain language. The scope, methodology, key drivers, risks, and the range, so it is understandable and decision-ready for our senior management to understand.
A good cost estimate not only relies on a rigorous process, but also captures the full cost over time, not just the startup numbers. Let's turn to life cycle costing on the next slide.
[00:31:30 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: We often hear life cycle costing. What is it about? In plain terms, it means accounting for the full cost of an investment over its life, not just the startup number. We need to cost what it takes: design; implement; operate; support; maintain; upgrade; and at the end, dispose, divest, and close out. This reveals downstream costs, so we avoid costly surprises and plan again wisely. In practice, investments move through an investment life cycle with one or more projects as part of the journey. Investments typically unfold over many years, and throughout that life cycle, different projects may be undertaken.
An electrical refit of an office building is one example. Each project has a clear beginning and end, and together they contribute to the overall long-term investment.
Treasury Board Oversight lines with these stages. Departments seek project approval, PA, and expenditure authority, EA, at key gates. And that's when TBS looks closely at your cost estimate, the lifecycle view, key drivers and risks, and clearly explained cost estimate range. Early on, the range is wider. As data sources and evidence improve, you update assumptions and narrow the range, staying true to the iterative process. To streamline oversight and improve efficiency across the project life cycle, we recently upgraded our guidance. I'll show you what changed and what it means for costing on the next slide.
[00:33:09 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: Without lowering the bar on costing credibility, five key changes were recently made to the directive on the management of projects and programs. The focus is on the essentials from a costing perspective, a full lifecycle view. Key cost drivers and risks and a clear cost estimate range that narrows as more evidence becomes available. When an amendment doesn't change the project baseline, the document requirements are lighter. Financial thresholds for project complexity and risk assessment, PCRA, are higher. This means less paperwork on lower-value, lower-risk projects, and more attention where risk is highest. Early approvals are recalibrated. Document that departments no longer need to seek TB project approval for the definition phase of a project. This means you can advance definition work under expenditure authority and then come forward with a stronger evidence-based cost estimate when seeking project approval for implementation.
Throughout, departments remain responsible for developing and updating cost estimate ranges as evidence improves. For the most complex high-risk projects, cost estimate classifications reflect the project's characteristics. For a lower-risk project's requirements stay the same. The expectations under departmental authority were clarified to ensure that project management frameworks embed consistent costing standards and processes, treating costing as part of governance across the project's life cycle, not just at Treasury Board.
In short, lighter admin where it makes sense, stronger focus where it matters, and clearer lifecycle costs with ranges that narrow over time. And on to our next slide with our key takeaways.
[00:34:59 Valerie Seeto appears split screen with slide, as described]
Valerie Seeto: So just a few final thoughts to share with you before Mike and John take over with more interesting information to share with you. Costing drives decision-making. It helps decision-makers compare options, understand risks and uncertainties, and choose the best path forward. Credibility matters because costing supports decision-making, including CFO approvals. Establish clear costing standards and governance so cost estimates are consistent. Think beyond the upfront costs. Include what it takes to operate, maintain, and sustain over the life of an investment. Follow a rigorous repeatable process. This is where the six credibility criteria and the seven-step costing process are essential. Use a cost estimate range to show uncertainty so decision-makers can see what might change and by how much. Bottom line: there's no easy button for costing. With structure, collaboration, teamwork, we can produce costing that is credible and decision-ready. Back to you, Jocelyn.
[00:36:08 Jocelyn Charron, Valerie Seeto, Michael Hanlon and John Kealey appear in video chat panels]
Jocelyn Charron: Thank you, Val. You definitely are the master of costing when I listen to you. It's so interesting. But then what comes to mind is about the level of complexity and the nuances. I feel like raising the level of understanding of what costing is and the multitudes of factors that can impact credible costing. We're subject matter experts. We're dealing with programs. Often, they are not experts in costing. So how do we translate all of these technical considerations for them to basically follow along the methodology that are set and be able to come up with credible costing to feed approvals on multimillion dollar projects? Definitely your presentation indicates that it's not easy, but there's a lot of methods and tools available. I have a question to you. We spoke about the situational awareness. We spoke about the compass earlier on. What do you think are some of the most important competencies for our technical experts or our functional experts on the call when it comes to costing? What are some of your thoughts What do you need to be good in to be able to be a good cost there?
Valerie Seeto: I would say the analysis piece is really huge, but also to be able to translate that, to be able to communicate that, to work in the team approach that we talked about because so many different areas contribute to that credible cost estimate. But then to be able to translate that and communicate that to decision makers in a way that makes sense for them is an ongoing challenge that we're all working through and hopefully working together to get better at.
Jocelyn Charron: I agree. I do agree. It's about translating that technical into the "so what" for your audience. I have one more question. I'm going to go to Michael, but when we do presentations to Ministers, what comes back in terms of the feedback? Why is costing sometimes such a sensitive subject? Why do ministers get hung up on this?
Valerie Seeto: That's a great question. I think going back to what Sam had mentioned about we need to change the narrative from the cost increased... to these are the things that have changed since we have last come in with a cost estimate, and as a result, the costs have increased. When you talk about political world events impacting supply chain or other things, those obviously have lots of impacts a costing perspective. It's not to say that the cost estimate was not well done at the time, but the cost estimate is only as good as the information that helped inform it. When we're coming back two, three, five years later to TB ministers, it's important to explain what has changed so that they understand why the costs may have increased, in some cases, quite significantly.
Jocelyn Charron: I agree. The cost of living and supply chain, like you're saying, when we saw in COVID, for example, the cost of lumber going so high. If you're costing a project over 10 years, obviously what you cost it 10 at years ago is not the same price. The increase of cost of living would surely just make a difference on the bottom line. That cost over time as well. That's even in itself is an interesting explanation to provide. But then again, that's what it is. The price of the bread 10 years ago is not price of loaves of bed today. If you put that into multimillion dollar projects, yes, it can change a price tag. I agree.
Michael, you're on. Give us a bit of insight about what happens in CRA and what are some of your expertise in your area when we look at costing, shifting over to perception or perspective of a department. Michael will be giving his presentation in English, so for those who want to switch to listen to the translation, interpretation is also available. Go ahead, Michael.
[00:40:34 Michael Hanlon appears full screen]
Michael Hanlon: Hello, everybody. So again, my name is Mike Hanlon. I'm the manager of the Program Costing Center of Expertise in the CRA.
[00:40:45 Text appears on screen: Michael Hanlon, Manager, Canada Revenue Agency]
Michael Hanlon: So I've structured this presentation with the assumption that people on the call either are like me responsible for building that type of capacity within your departments, you're also either or challenging or developing cost estimates. So this will be a bit of a fire hose over the next 15 minutes. I apologize in advance.
[00:41:08 Michael Hanlon appears split screen with slide, as described]
Michael Hanlon: And I also apologize to the poor translator who has to try to figure out what I'm saying. First of all, we'll start off with context because costing is almost entirely understanding the context. So within the CRA, very large organization, about 14 branches with operations spread across the country. From a costing perspective, the context is more about within those branches, a lot of the work is interrelated. There's downstream impacts from one branch to another within areas within a given branch. And for the most part, most of our costing involves program changes and software development. So that's effectively our universe. And overall, our costing is low to medium risk, particularly in comparison to John, who's going to speak in a little bit.
And for the most part, our costing is done by non-experts. We have program managers, we have FMAs. Those are the people who do the costing, the bulk of the costing work within the CRA. Again, it's important to understand that context because that impacts everything else. What I'm going to try to do is at the end of each, I'm going to be setting it up so I'll talk about how it works in the CRA, and then try to provide some takeaways, mostly focused on people who actually do the cost estimating or the reviewing portion.
What is the takeaway from this slide? It is more important to know your context of your program when you're estimating cost than it is anything else. Understanding your program, how it functions from a cost perspective is going to greatly help your ability to do a cost estimate. What do I mean by that? Question is, what drives the costs in your area? Which parts of your costing or your estimates are slam dunk, easy to estimate, which parts are very risky? For example, can you predict easily how many people will call or how many of these volumes will come in? Or unfortunately, do you have to guess?
And what part of understanding that context really tells you where you need to put most of your focus. From a reviewer perspective, what you really need to know is, first of all, understand some of what the cost estimator understands in terms of a very relatively shallow knowledge. And what you also need to do is understand how the pieces interconnect. If you don't understand how Branch 1's work flows to Branch 2, or how those types of things flow together, it's extremely difficult to review and validate a cost estimate.
Now the question is, how do you build that capacity? I'm not going to mention, in terms of making sure that your organization can do these things and can do them within the time frames involved. Next slide, please.
[00:44:40 Michael Hanlon appears split screen with slide, as described]
Michael Hanlon: The first step, and again, I'm a bit of a policy nerd here, where the first part is making sure that you have a set of guidance in place. Now, the guidance in all times starts with the TBS Guide to Costing, but it is very high level, as by necessity it is. What you're going to need to do is to build a policy setup that works within your context. For example, John and I live in very different costing contexts. The government of Canada stubbornly refuses to give the CRA fighter jets, and because of that, I haven't had to be involved in any of that type of costing.
Again, costing context is completely different. You need to have a guidance that works for your setup. Now, what do you get for a set of guidance? One thing you don't get is your user group reading your guidance. They won't, for the most part, especially in my area where they have non-experts that are working on a handful of costings a day. You can't assume that they're actually going to read your policy documents.
What does it give you? First of all, it sets up a set of guardrails. It says these are the things you can and can't do. And particularly from a reviewer perspective, being able to say you can't do that because it's against policy is extremely effective. It allows you to enforce validation related to guardrails. At the CRA, any cost estimate that's leaving the agency has to have some level of finance review. And we put in some risk thresholds for some of these things, just so you're not wasting senior management time. But ultimately, there has to be some level of review. And because we have a policy requirement, you can enforce that.
Building a foundation for training, extremely important. We wrote the procedures for program cost estimation, and we quite literally, particularly the early stages of our training, was copied and pasted from our procedures. It provided the step by step process. And also it helps you defend costing decisions. I'll give you an example. A few years back, thankfully, quite a few years back now, when we're talking about contingencies. Effectively, the approach for contingency was, "The costing makes me feel a little uncomfortable, add 25%." Now, where the 25% came from, I could not tell you. But again, that was the knee-jerk approach, and it wasn't really based on anything. The problem is that becomes difficult to defend later on when, okay, why is 25% the right number? Why not 20? Why not 30? At that point, we worked with our stats experts. Luckily, CRA has stats experts. And we worked with them once we took some time to explain to them what we were actually looking for. That was an interesting conversation. And they actually created statistical models for us so that we could say, "okay, if your costing is plus minus X%, here's an appropriate contingency to add". And we were able to create guidance with a very simple table so that our non-experts can look at it and say, "okay, well, I'm up this. I should add that". And again, and this just happened, I think last week, if and when somebody comes back to challenge that, you're able to say, here's where it comes from. Now, I believe there's a wooclap question, if memory serves. we'll be discussing that in a second.
[00:48:35 Michael Hanlon appears split screen with wooclap screen, as described]
Michael Hanlon: Now, essentially, the takeaway here is review the guidance that's prepared for your area, for your department. Now, if, for example, you're in a situation where you're estimating costs, and there's always these blank areas, always these areas where instead of knowing how to do things, we're having to peel it back to, "Oh, well, this worked last time. Let's continue." This is where you need to identify to whomever is responsible for the guidance in your department. This is where we need guidance because we're guessing here, and odds are the person who's in the other area is guessing it as well.
So now for the wooclap question. Now, when estimating costs, work should continue until you reach the one perfect, beautiful answer. This is absolutely false. As we talked about earlier, essentially, a cost estimate is an input to a decision. That's all it is. If you extend beyond the time of that decision, if it arrives the day after the decision is made, everything you've done in terms of costing is absolutely wasted effort because you weren't there to support the decision. When we're training people, basically what we let them know is, first of all, every cost estimate is wrong. If anyone ever tells you that I did this complex cost estimate and two years later, I was right to the dollar, either they're lying to you or they've rigged the actuals to equal the estimates. Those are the only two reasonable possibilities. Every estimate is wrong. So what are we trying to do?
[00:50:47 Michael Hanlon appears split screen with slide, as described]
Michael Hanlon: We're trying to effectively get the best, most credible number we can do within the time frame we have available. And hopefully, your department has built that capacity in advance so that you can do so. So now we have the guidance in place written for the people who are actually doing your costing.
Now you have to give them tools to actually work with. Now, within the CRA, we have, and again, the two main tools that we use. We have, and it's all Excel, is one very simple tool that we use for gathering costing information and for doing the initial cost and calculation. To give you a bit of background, the user experience for this tool has not changed in about 15 plus years, aside from a little bit of back-end reporting. We have obsessively kept the user experience as close as possible to not changing, because that gives you a lot of capacity where you're not having to constantly train people, and it's as simple as we can possibly make it, because, again, in our context, we're dealing with non-experts.
Now, what we built also on top of that is a tool, we call it the CART, that's for consolidation and reporting. Essentially, it's for more super users and people who are trying to gather in cost costings from all of these 14 different branches and be able to present either combine them into one costing or present them from a reporting perspective. This is where the key takeaway is.
If you're building it, it's making sure it's simple, it's sustainable, easy to use. By easy to use, I mean, if your training has to focus 100% on using your tool, you probably don't have a simple enough tool. I disparage my own tool all the time by referring to it as a glorified calculator, and that's exactly how I want it to be. I want people's 100% of their effort to be on figuring out how many people I need, how much non-salary spending I'm going to do. And then you just stuff the numbers into your glorified calculator to add all of the standard costs where we go from an area's estimate to the cost of the agency. Now, as far as takeaways go, understand your tools. For cost estimators, understand how they work, take whatever training is available to you.
[00:53:45 Michael Hanlon appears split screen with slide, as described]
Michael Hanlon: And also for costers, one of the things that I include in tools is the data available to you. Another tool for you is making sure that when you're doing a costing, if you're having to estimate the costs based purely on professional judgment because you lack the data, that is a sign that you need to get better data, particularly if it's happening frequently.
Now we move on to training. And how do we build up the capacity in people? One of the things that training needs to be as it relates to cost estimating is it needs to be timely and it needs to be geared towards the people who are actually doing your costing. Again, cost in training that would work really well for John would fly over most of my costers heads by a good distance. Because, again, we work in different contexts. And it doesn't need to be expensive to set up. Essentially, our costing training started, again, pulling it from the procedures. And then because this was pre-COVID, effectively, our training budget was taxi chits to get me and one person from my team to travel around the city to give presentations. That was it. That was the budget. And the first one that we developed was probably [inaudible]. A lot of people probably were confused. Over time, what we would do is develop that training in such a way that when we had a lot of questions about subjects, we would add content to the extent that after a few years, we stopped getting questions.
Now, at that point, we decided that, really, do we need to be doing live presentations of no questions? So at that point, we just recorded some videos of that content, posted those videos to our internal site, and then basically said, "take the training first, and then we can have a more value-added discussion with your teams about how your exact costing works". That second part, we've had a few people we've had conversations with. It hasn't been a tonne, but there's always hope.
Now, as far as takeaways, the funny thing is whenever I say the word now, they switch slides. We're still on the training slide. Sorry. So apparently I'm very forceful when I say now.
Themes, as far as training, one of their goals there was to make sure that people understood that costing is not some arcane magical thing. It is just a process. And a process for which, as we talked about, it's more important to know your program or know the area they're working with than any financial knowledge. Understanding the purpose of the cost estimate, that's always your first step. If you don't understand what purpose and what decision you're trying to support, you cannot cost.
We already talked about every cost estimate is wrong, and you'll never have the time that you need. That will never happen. Particularly given our current government context, where the timeline seem to be tighter than even than they were previously. The question is, what is the best, most credible estimate that you can come up with within the given time frame that you have? And ultimately, and we talked about communication earlier, is being able to communicate to the cost to the decision maker, here's what we're able to do, and here's its limitations. Here's the risks. You wanted something in a week that should have taken a month. Here's what you got, and here's the quality of what you got. And again, sometimes it takes a bit of bravery to say that this cost estimate is ultimately preliminary, that we didn't have the information or the time necessary to give you a good, even a rough order of magnitude. So here's what we got. Next slide, please.
[00:58:24 Michael Hanlon appears split screen with slide, as described]
Michael Hanlon: This is near and dear to my heart: costing review, the validating of the cost estimates. This is the area where my team gets more directly involved.
As we talked about, based on policy requirements, all cost estimates leaving the agency have to experience some CFO review or a finance review. The thresholds that are on the screen right now, those are as it relates to agreements. So I'll give you a story. When I first started in this area, every single agreement went to the CFO for the sign off. As you can imagine, it provided me physical pain to send a $1,200 agreement for approval to the CFO of, at the time, a $4 billion organization. This is where your review process has to have a basis in risk, the riskiness of the cost estimate that you're dealing with. If you do the same review for every cost estimate, you are doing it wrong, ultimately. So again, one of the things that you ensure is that when you're doing that review and when you have that guidance in place to make sure that the review is necessary is you have consistency with the rest of the guidance in terms of what is allowed, what is not, what the cost estimate should look like, and the mitigations of financial risks.
Now, takeaway for the reviewers.
What I always tell people is that you will, again, you will never have the time to look at every little aspect of a cost estimate, at least in my world. Hopefully, John has a little bit more time. Then what do you look at? Well, again, when you understand the risk that you're trying to mitigate, which is, again, every cost estimate is wrong. You're not trying to prevent it from being wrong. What you're trying to prevent is it being really, really wrong. That's the risk that you're trying to mitigate. In that context, what do you care about in a cost estimate? You care about, effectively, the big stuff and the missing stuff. The big stuff, you care about it because if it is 30% of your costing and they get it really, really wrong, your bottom line is going to go way up or down. If something is 0.1% of your costing, even if they get it 100% off, are you really going to care? So again, the big stuff that you have to identify the larger items, focus your review there. What do you care?
The missing stuff. The reason you care about missing stuff is you don't know how big it is. This is where, as a reviewer, understanding how the various pieces work together, that is extremely important because quite often, an area that's submitting you a costing, they know their area really well. So you can have a lot of confidence that they haven't missed anything in their program. But all of the other areas that they had to gather in could very well, their things could be missing, and you quite frequently find it.
[01:01:43 Jocelyn Charron and Michael Hanlon appear in video chat panels]
Jocelyn Charron: There's a good point, Mike, in terms of the subject matter experts having their own expertise, but then often it's about the horizontal costs or the compounding across, the risk, the compounded risk across the organization. I know you have a few more slides. I'm just wondering if we can maybe give the audience one example of costing at CRA so we can let John as well come into some of the DND realities that I'm sure the audience is interested to hear.
Michael Hanlon: Kind of figured I'd get up on time. Can you go one slide forward, please?
[01:02:21 Michael Hanlon appears split screen with slide, as described]
Michael Hanlon: Okay, so this is the one example. As many of you are aware, during COVID, the CRA was asked to deliver a lot of benefits in a very short time period. And just what you may not be aware of is that almost every one of these benefits, quite often, they'd have different funding approaches to them. Now, I wasn't involved in direct discussions. I assumed that there was some logic to how it was being assigned. But as an outsider, it honestly seemed like somebody was spinning a wheel, and wherever the little thing landed on, that was going to be the funding approach. And you'd have things changing constantly, time frames changing constantly. Luckily, it was one key group that was really impacted of FMAs as far as being the OPI and a lot of these costings. And the fact that over the years prior to that, we had built up the capacity in people through the training. We had built simple tools that everybody knew how to use because, professionally, they'd all grown up using them, effectively from the user perspective. And their guidance was there so that if somebody was trying to do something outside of the lines, there was guidance to stop that because all of that was already in place. It definitely didn't make it easy, I would never say that, but it made it possible.
For example, we have one tool. It's the same tool that you use, whether you're recovering costs, Treasury Board submission, what have you. And it just requires a button click, where you - oh, Treasury Board has changed how this is going to be funded. Click. There, the estimate has changed. You didn't have to redo the whole costing in a different tool. Where we've built the CART on top of this simple tool, then it gave that poor group the capacity to just gather in all of these cost estimates, consolidate it, track it over time. Oh, something has changed, swap it out, consolidate it again, begin reporting for a variety of reporting purposes. Because the structure was there in advance, it didn't make it easy, but at least it made it possible. And that is it.
[01:04:51 Jocelyn Charron and Michael Hanlon appear in video chat panels]
Jocelyn Charron: Thank you, Michael. We'll come to John in a second. I just wanted to point out, ultimately, when you're costing big initiatives, especially you're coming into a crisis such as we had COVID, or you're coming in with a change of government. Right now, the government is facing important challenges in terms of geopolitical context, in terms of affordability, housing. So you're going to be coming in with very, very quick estimates. You're going to be coming in with proposals that require fast delivery and fast decision-making. So I think having those key foundational ingredients of costing, good frameworks, good estimators, calculators, methodologies allow you, as you say, to cost what you know really well. So you can have a bit of that space for what is unknown or is going to be discovered as you move along, hence the iterative part of costing that we spoke about with Val and Sam earlier. So thank you for that. Over to you, John, to share a little bit of those realities at DND.
[01:05:58 Jocelyn Charron and John Kealey appear in video chat panels]
John Kealey: Thank you very much. Hi everybody. My name is John Kealey.
[01:06:01 John Kealey appears full screen]
John Kealey: I am the senior director for the Department of National Defence for costing.
[01:06:06 Text appears on screen: John Kealey, Director General, Department of National Defence]
John Kealey: I'm going to do a quick overview of the first three slides, because I think everyone will be more interested in the last three, which are more about risks and things like that. The first three are more about policies, process and a little bit of context. So I'm going to go a little faster first, and after that I'm going to spend more time going back.
[01:06:33 John Kealey appears split screen with slide, as described]
John Kealey: Context: For DND we have about 80 professionals who do the costing. The majority are in my team and then we have a small group of four or five people. Next, J8 Mil Ops, they are responsible for estimating the costs for military operations.
So, we in my team support project teams in determining and developing cost estimates. When seeking approval from the Treasury Board of Canada Secretariat, the Cabinet or other managers for DND.
Mike has already mentioned that for DND it's very different than other ministries for several reasons. Firstly, the things we buy and the things we build are very different. We have ships, we have submarines, we have fighter jets, we have lots of interesting things like that.
[01:07:27 slide appears full screen, as described]
John Kealey: These big projects don't cost millions, but billions. As Mike mentioned, this was something that cost $1,200; I've never seen anything that cost thousands. Sometimes I see things that are millions, but most of the time I see things that are billions. So these are big projects, and these are big projects that will last 20, 40, 60, 80 years. As Valerie already mentioned, we have to estimate the cost for an 80-year life and that's really, really difficult. So that's something I don't think other ministries do like us.
We must also meet the operational needs of the military. Again, that's very different, and there's a lot of complexity there, if you take into consideration the political developments we're seeing right now.
[01:08:25 slide appears full screen, as described]
John Kealey: Next slide please. I'm not going to spend much time here because Valerie has really covered a lot of it.
[01:08:32 John Kealey appears split screen with slide, as described]
John Kealey: So we have the policy, the financial management, that comes from the Treasury Board of Canada Secretariat. But our ministry has the directive and order (output) from the Department of National Defence and that's where we establish the roles and responsibilities for CFOs compared to other organizations. We still have the guidelines from the Secretariat and also the institution's guide. Then, in my team we also have… I have a separate team that develops standards, internal policies and also provides training for all professionals in my team. So they analyze the information from the Treasury Board of Canada Secretariat and then develop our own standards that complies with the Secretariat's mandate. Next slide, please.
[01:09:18 John Kealey appears split screen with slide, as described]
John Kealey: Again, not spending much time here because what we're doing is really… it aligns a lot with what Valerie has already presented. But all this to say that what we have, we have the six steps. I know that Valerie presented seven, I have six. But in any case, we completed two steps together and it's more or less the same thing. We have cost planning. This is very important. Honestly, we spend a lot of time planning. When you have time to spare, you spend a lot of time planning. This is very important. Next, we establish the basic rules, the assumptions. That's really where our costers will be spending a lot of time, too.
[01:09:59 slide appears full screen, as described]
John Kealey: Because the assumptions are where we're going to achieve... if we don't make good assumptions, the results are going to be really bad. So we need to make sure that we spend a lot of time establishing the basic rules and then the assumptions. We're going to prepare the model. So, my team that I mentioned earlier, which does the training, they also developed a model, a template that we use. It's very complex, and it's also in Excel. But we have the model that our costers will then start to fill in with the information we receive.
[01:10:32 John Kealey appears split screen with slide, as described]
John Kealey: From the industry, from the PMO, other sources, other information. We will examine, analyze and update the model and if it aligns with the part that Valerie really... also she made the circle. This is really the part where we'll have conversations with the project team to review and update our information. Should we adjust our assumptions? Are there things that have changed?
Next, we must provide a guarantee, and then also present the results. That's interesting. Presenting the results is perhaps the most difficult part, and I'll explain why in a minute. But before I go too far, we have a Wooclap question coming up. So, if you want to start answering the question, that would be fantastic.
But presenting the results is really complex. I'm not a coster, honestly. I am a CE, not an FI. Therefore, presenting the results to the other management; present the results to the Secretary of the Treasury Board of Canada Secretariat. Valerie understands what I do, but the majority of people at the Treasury Board of Canada Secretariat understand absolutely nothing about what I do.
The same goes for the ministers, not to be insistent.
[01:11:49 John Kealey appears split screen with wooclap screen, as described]
John Kealey: But they too are not so strong on the financial side. So they too sometimes have difficulties.
So, presenting the results is really something that is particularly difficult. Now we come to our Wooclap question.
From here on in, I'll be going in English. When developing a cost estimate, for a major capital project, so think submarines, think a new fighter jet, something really complex, very DND-ish. Why do we apply contingency? Is it to cover the unexpected scope changes? Is it to account for identified risks and uncertainty and assumptions? Is it to ensure the project has extra funds for future upgrades? Or is it because Treasury Board requires a fixed percentage, and that's what Treasury Board tells us to do, and that's what we will do. And you could see that most of the people are answering correct, that it is B. So it's to account for unidentified, sorry, for identified risk and uncertainty in assumptions. What's amazing, though, is PMOs don't see it this way. They think contingency is a buffer, a slush fund, money for future upgrades. "Heck, if these risks don't materialize, we can buy more of the widgets that we need."
And that is not the case. Contingency is applied to reflect the financial impact of identified risks and unavoidable uncertainties within the project's defined scope, approved scope. It is not to do a scope creep and to buy more of the widgets.
[01:13:14 John Kealey appears split screen with slide, as described]
John Kealey: So, capturing risk uncertainty. So now I'm going to go a little bit slower because this is the part that I think is a little bit more interesting. And Mike was right. We do things a little bit differently at DND because DND is different. Capturing risk uncertainty. We have two pieces that I'd like to speak on. And similar to what Mike was mentioning that he's taking a risk-adjusted approach or risk-informed approach on how to engage the CFO, we do the same thing. So under my authority, I see all projects from 25 million to 200 billion. It's a very wide gam, and we don't do costing the same way for every single thing.
We have risk-informed and risk-adjusted. Risk-informed cost estimates. It's typically developed either by the business owner, the OPI, the PMO, and also reviewed, validated by a cost analyst within my team. It is still based on industry quotes or benchmarks. We do apply a flat contingency, but unlike Mike, I know where those rates come from, and I'll get to that in a second. And then we actually do what's called right sized costing for some projects. And what this is, is we take a a look at the estimated cost and the PCRA, project complexity risk assessment, and we do a heat map to determine what type of costing we should do, what type of due diligence we should do.
So back to that flat rate. What we did a few years ago is we engaged a third-party consultant, one of those big firms, to take a look at historical contingency usage by different project types for different complexity. Based on that information, we were able to see, Okay, well, PCRA level 2 file from an infrastructure piece that is under 100 million. Jeez, they usually have 10% of contingency that they use at project close-out. And what we did is we took that information and we developed a bit of a grid. And that helped us to determine very quickly for low complexity, low risk files, we could just apply a very quick contingency percentage. But it's still based on the risk and complexity of the individual file. And the reason we do this is to make it so my analyst can spend more time on the higher dollar value, higher complexity pieces called the risk adjusted cost estimates. This is the bread and butter. This is what gets me up at night in a good way.
So, the risk adjusted cost estimates. So again, here, think, again, the submarines, think of the fighter aircraft, new satellites, whatever the case may be. These cost estimates are all independently developed by cost analysts within my team. We might even engage third-party costers. So again, a big firm, a Deloitte, or whatever the case may be, to help develop an independent cost estimate or to review the costing that we developed.
We use various methodologies, tools, and techniques. We do parametric cost estimates. We use analogic information. We use top-down, engineering build-ups, we use various methods. We have different applications, different software that we also use. While our costing is predominantly done in Excel, we do subscribe to various databases from, for example, the US on how much a warship might cost, and we use that as a parametric tool for our own costing.
We do a rigorous risk and sensitivity analysis. People hear the buzzword of a "Monte Carlo simulation". We do Monte Carlo simulations. We do 5,000 different simulations based off dozens of different risks that are hopefully not correlated. And what's really interesting is that we have engineers, mathematicians, statisticians, accountants, actuarial scientists that all form parts of my team, and they bring different aspects to this. This is where we spend the most time is on that risk and sensitivity analysis.
We apply contingency based on different confidence levels. So our costing will develop an individual point estimate. So it's going to cost us $100 billion. That is how much we think it's going to cost. But if it hits the fan, it might cost us a lot more. This speaks to the range that Valerie was mentioning earlier, or things were really well, it might cost us a bit less. What we usually do is what we call a P80. We want to make sure there's an 80% level of confidence that we will have enough money. We assess, we do 5,000 Monte Carlo simulations, and we're like, Okay, the level of contingency that we need to ensure that we have enough funding for 80% of the situations is this. And that is the level of confidence, the level of contingency that we apply for that level of confidence, and that is part of the messaging. If I can move to the next slide, please.
[01:18:01 John Kealey appears split screen with slide, as described]
John Kealey: The costing challenges. I'm not going to go through all eight here, and some of them, Val and Sam already mentioned, but we have a lot of costing challenges within DND.
So, for example, tariffs and microeconomic uncertainty. DND, yes, there's a consumer price index. There's a defense-specific inflation. And so we have a Chief Economist within DND who tries to calculate what our own individual inflation rates might be. That being said, we're responsive to industry. And industry sometimes gives really high year-over-year wage increases to its members, and that directly impacts the cost that they charge us. I mentioned earlier, the communicating the estimates. That's extremely challenging. It's very hard to say in messaging to a Minister that's going to be publicly released, we think it's going to cost $100 million, but at the P-80, we think it's going to be $120, and here all the caveats. They don't want that. They want a number. So it's very challenging. And this is where we're working very closely with Val and her team.
Black Swan events. Typically, our risk and sensitivity analysis is more like the things that we think might actually happen. But as you've seen in the past decade or so, we have a global conflict. We have now seen a global pandemic. So we need to bring in, how do we account for these Black Swan events, which are less Black Swan and more risk events?
So there's a lot of challenges here on the contingency management. This is related to the PMO thinking that the contingency is their money to spend and not give back to the Department if risk events don't occur. Next slide, please.
[01:19:40 John Kealey appears split screen with slide, as described]
John Kealey: So, the key takeaways. A lot of these repeat some of the things that Val was mentioning or Mike was mentioning. I won't mention all five of them. But credible costing is foundational to evidence-based decision making. Not too long ago, it was actually around this time last year, I was in the heat of working on the costing for the NATO 2% target. Costing played a very key role. To Mike's point, all my numbers were very wrong. Like not wrong, but very wrong. I'm confident that I'm very wrong in every single figure I provided. That being said, it gave a sense to decision makers of what these things could cost. Is it going to be plus or minus a couple of billion? Sure. But you know what? They have a sense of what it's going to be. Every single organization is different, but DND is particularly different. DND is particularly complex, for the reasons I mentioned earlier, To account for those complexities, we follow a rigorous process to ensure we develop credible cost estimates. That is the thing that we ensure that we follow international standards, TBS standards. It's something that we are very focused on.
And the last one that I'll mention is the risk uncertainty analysis. This is where we spend the most amount of time. Anyone can actually put numbers into an Excel sheet, but you need very informed individuals to use evidence-based decision making to shape that risk uncertainty analysis. And I'll stop there. Thank you.
[01:21:15 Jocelyn Charron and John Kealey appear in video chat panels]
Jocelyn Charron: John, I felt your whole presentation was so interesting, and now I want to go work for DND. I'm just joking, but you gave really tangible examples, and as well, I think you illustrated the level of complexity that you're dealing with and even the multidisciplinary expertise that's required to fuel these cost estimates. So we took a subject that is such a technical subject and then brought it into really the value it can have, the challenges it can have as a public service, and the needs for us to work collaboratively to inform these big investment decisions as a Public Service. It's like we're seeing in the news, all of these... We're behind on some of our defense investments, and we have to invest as a country. Then when you're going in through that, how much money of our taxpayers is that going to cost? Then the methods and the tools and resources and expertise that are there to support it. So anyways, I just completely loved it. Thank you so much. I was going to ask maybe one question that we had from the group before I go into the closing remarks. So we spoke, I think you're all masters of your trades,
you've demonstrated that super-eloquently in the conversation. What is one thing we could do as a community to engage those stakeholders, like the ADM or the DG of the program that's like, "No, no, go away with your template. No, it's too technical". What would you tell our community as a tip and trick to get them to play with you in this important space? I don't know, John, at DND, what do you think?
John Kealey: This is a tough question because "costing is always the problem", is what I constantly hear from DG colleagues.
Jocelyn Charron: You're always blamed?
John Kealey: We're always blamed because we're responsible for communicating the cost increases, and there are always cost increases. It is very challenging, I think, showing the added value. They really want just "get out of the way and give us the dollar value". But sometimes we're able to explain to them, "Look, this thing is unaffordable. Even with no contingency, it just makes no sense." On the big files in particular, we have very close engagement with the project teams, and they see the added value. It's more on those transactional files, the couple of hundred million dollar files, where they just want us to get out of the way. Showing the added value that we have is something that we continue to spend time and energy with. But I'm sure Val and Mike have more to add.
Jocelyn Charron: Okay, I'll go to Val. 30 seconds, Val. What do you think? Top trick.
[01:23:55 Jocelyn Charron, Valerie Seeto, Michael Hanlon and John Kealey appear in video chat panels]
Valerie Seeto: So agree. Costing can be a barrier, but it is a great piece of information to help inform decision makers. And I think we sometimes identify issues, concerns that may not be visible to others. I think that that's an important piece. I also love how there's a lot of GC organizations that now have not just a CFO, but other important Executives at the table making decisions on the cost estimate. So, the more we can have people involved and the more it can be a team effort, the better results I think we'll have. Thanks.
Jocelyn Charron: Thank you for that. Mike, do you want to chime in? One thing for you?
Michael Hanlon: The biggest thing is making sure that when you're communicating, first of all, you do it in a consistent fashion, so you're not having to explain how you're communicating. They should never see a template. Oh, God, don't ever give them a template. And again, the question is, what should they care about? Everything on that page should be things that they will care about. The stuff that the rest of us find interesting from a costing perspective, leave it out. Where are your risks? How are we mitigating them? In big terms, what is in this cost estimate? What are we spending our money on? Those three points should be 90% of the conversation. Everything else, you try to cut out.
Jocelyn Charron: Cut out the technical. Stick to the "so what" for them. Yeah. And bring in your technical expertise to make sure that those elements are well in hand. Thank you. I think that ends the conversation very nicely.
[00:34:59 Jocelyn Charron appears split screen with information slide, as described]
Jocelyn Charron: I think you've shared a lot about the tools, the resources, the methods, the frames that are provided by the Treasury Board through the policies or the guidelines, and then even how departments will bring in their own layers of expertise to meet the nature of operations in their respective Department. For those of you that are interested, there's actually a Community of Practice on costing. Then you see here on the page, a GC exchange that we're sharing information here. I think as a Public Service, as a Controllership Community, we can do so much better about sharing to mature our costing expertise and our best practices across departments. I'm going to encourage you to rob and steal from each other, borrow and take what you took from this conversation through your DND and CRA, and as well as what is published on that GC Exchange page. If you wanted to take part in the newsletter as well, you have a QR code here.
If you want to stay informed and even mature this subject, I think AI analytics, big data, that can so play in this space as well. Those are really for you to consider. So we're closing our Learning Essentials series on Financial Management through this last session. I just wanted to thank you for being with us. I wanted to thank John, Sam, Mike, and Valerie for being with me this morning.
[01:27:06 Jocelyn Charron appears full screen]
Jocelyn Charron: As well, thanking my team.
[01:27:10 Text appears on screen: Jocelyn Charron, Executive Director, Treasury Board of Canada Secretariat]
Jocelyn Charron: This was a first for the OCD to do these Learning Essential series. Every single time, we reached over a thousand participants. We would raise between 1 000 to 2 000, which I think is amazing. As well, honestly, I have to say we've done with the Canada School, and they put on a beautiful production. They make it easy on us, and I think it's very professional in how they do it. So completely feel supported by the Canada School of Public Service in terms of that support to offer you these conversations.
Your comments, obviously, we welcome. We've adjusted even the series based on your feedback. So, if you wanted to share feedback with us, that would be wonderful as well, and we take it heart. We are actually happy to announce that we are going to be having a Controllership Essential Series for the fiscal year 26-27. So bringing together more conversations that are broader, the integration parts in controllership, the money, the procurement, the projects, the investments, and even the audit as well. That is going to be our focus in the next session. But in the meantime, I hope you learned something this morning, and it was really a pleasure to be here with you. A big thank you for your great participation and also to our speakers, and we look forward to seeing you soon in our new series. Have a good day everyone.
[01:28:40 Animated CSPS logo appears. Text on screen: canada.ca/school-ecole.]
[01:28:44 The Canada wordmark appears.]
