On June 23rd, I had the privilege to lead a discussion and presentation entitled, “What Cybersecurity Should Cost? Updated Life Cycle Cost Estimating Factors for Cybersecurity in Business IT System.” Below are the questions asked during the presentation from our attendees.
Q: Among the agencies for which you have data, what portion are DoD vs. non-DoD?
A: Very limited DOD; none in the Development and only 1 in the sustainment. The DOD has only been phasing in the RMF program for 3 or 4 years and has not accumulated much data. Also, the OMB Dashboard contains OSD-level programs for Medical and Personnel. Individual AF, Army, and Navy programs are not included.
Q: Wouldn’t it be better to identify / remove outliers based on Cyber costs as a % of total spent by phase?
A: I don’t know. That would be a good follow-on project. When I sorted the values by cyber category, the outliers did not seem to be as important. Some categories were just higher than others. So I chose not to remove them.”
Q: “Was there any discussion or consideration that not all categories exist for each agency because they may be all “bucketing” their costs differently?
A: I think that is an excellent observation. We did not talk to any of the programs, because there were so many, so I do not have any insight into how they may have bucketed. Also, the categories were derived from the data and were not part of the OMB database, so it is definitely possible programs grouped many tasks into a single activity.”
Q: Given the DoD “falls out” of the data – which factors would you recommend using for a DoD cost estimate?
A: Great question. I can offer my opinion, based on my view of the research. I think because the DOD does not have as many eyars of experience with RMF, the factors derived from other government agencies would be an excellent start. I did do some research into AF programs in 2015 – 2017 and had some anecdotal data for some space programs and C4I programs. I concluded in 2-17 based on that data that cybersecurity added about $3M to $5M to a program or about 8% to 15%. But the data was like I showed you from the dashboard … a consistent level of effort for cyber engineering labor, with spikes for HW and SW every now and then. SO I think this OMB data is useful.”
Q: Could you clarify what is included in a business IT system? Does it only calculate cyber-related hardware and software (firewall, authentication) or all IT equipment?
A: An IT system includes many end items of HW and SW, but the factors presented here are only for SW or HW that was identified as supporting cybersecurity in the activity description.
Q: Does it only consider the user’s/customer’s system or the developer’s/contractor’s system as well?
A: I did not have that kind of fidelity in the data to tell. Certainly, I think it is fair to assume the O&S factors are the production system. The development factors I just do not know, but would assume they are for the developers’ system that then transitions to a production system.”
Q: How do you handle systems with high value to national security, ie zero failure acceptable systems vs systems with some risk acceptable.
A: I did not have the data to answer that question, although it would be an excellent follow-on program. I was never able to collect any technical data, so I do not know if the expenditures here resulted in a required operational availability, nor do I know if they provided adequate protection for the system. I wish I did. That would be a fun study if you could get the data. I suspect it is all classified.”
Q: Great analysis and presentation! How can we get a copy of the presentation and database?
A: The slides (and recording) will be available after 24 June on PRICE’s Resource Library – https://www.pricesystems.com/resource-library/