I’ve written elsewhere about the value of grounding your metrics program in semantics. This article goes well beyond that. We are just beginning to see a pattern of evolution that seems to be enabled, perhaps even encouraged, by the collapsing of silos into a data-centric knowledge graph. It is still early days, but the patterns are beginning to emerge, from our client work as well as our internal project to make Semantic Arts a showcase for what is possible. The serendipity moment occurred when I saw an outside presentation on a metric we were actively evolving (Accounts Receivable aging).
The plan of the paper is first to reexamine why we have metrics in our enterprises. From there, we lay out an evolutionary progression for metrics. Finally, we wrap up with a detailed case study based on aging accounts receivable.
Why We Have Metrics in Our Enterprises
Most enterprises have a rabid fascination with measuring things, often driven directly or indirectly from this quote.
“If you can’t measure it, you can’t manage it.”
—misattributed to W. Edwards Deming
As John Hunter points out, this isn’t what Deming said or meant. Deming did understand the value of data, for sure, but he also understood there was much more to managing than merely measuring. But it’s this attitude we’re saddled with, and as a result, most enterprises have more metrics than employees.
We have metrics in our enterprise to attempt to manage them better. If you peel it back a bit, you find that “management” is really about influencing behavior. Some metrics aggregate a lot of behaviors. “Net income,” one of the most widely used metrics, covers a range of behaviors from customer loyalty to raw material substitution. The OKR (Objectives and Key Results) movement is all about using key results (metrics) to change behavior (toward the goal of the objective).
In this paper, we’re going to look at traditional enterprise metrics and consider how the lens of behavior change can affect the way the metric and its usage evolves.
An Evolutionary Progression for Metrics
In this section, we outline a progression we’ve seen in several settings and provide a few simple examples. This is to set up the framework we will use for the detailed case study.
The progression is:
- Measure and superficial clean up
- Formal semantics
- Calibration
- Prescriptive action
- Root cause
- Better metrics
Measure and Superficial Clean Up
Most metrics start with just wanting to measure something you’re already doing. Often, you start with data you think you have. Maybe you want to measure customer acquisition cost or sales per square foot, or number of meaningful interactions per salesperson per day.
Inevitably, the act of measuring shines a light on the fact that you either don’t have the data you thought you did, or it’s mostly crap. There are often several months and several cycles of getting to the bottom of why you have poor data quality (at least for this purpose, prior to starting the metrics project you thought the quality was just fine).
Formal Semantics
At some point, the data is at least clean enough as to not be embarrassing to share. At this point, people often start asking harder questions about what the data really means and where it came from. This is especially true in large BI (Business Intelligence) organizations where there may be hundreds, or in some cases, thousands of analysts working in parallel on business metrics and dashboards. Inevitably, this leads to managers in different divisions wanting to compare metrics and realizing they are comparing apples to oranges.
This is where formal semantics and the so called “headless BI” can help a lot. By taking the definition of the metric out of the graphics tool and putting it in a shared place, you end up with shareable and comparable metrics. They still have a way to go before they are optimized, but at least they can be compared.
Calibration
Up to this point, the data is merely “clean” and comparable. It seems odd, but often it takes a while before people start asking if it is right, or if it makes sense, or if it even measures what we hoped to measure.
One of our recent metrics was future cash predictions. You might think your accounting system would be good at this, but most are not. Most of the important inputs to this prediction are elsewhere. We gathered up everything we thought we knew about our next six months of activity (into a spreadsheet naturally) and began reporting on this number.
It was at that point we entered into what we called the “calibration phase.” When you have a million dollars in the bank, your projections could be off by many hundreds of thousands of dollars, and you can still make a $400K payroll. But we recognized that we wouldn’t always be this flush, cash flow seems to come and go. At that point, we made the observation that an altimeter can be off by hundreds of feet when you are at cruising altitude, and no one will notice, but if it’s off by hundreds of feet at best, you’ll have a very hard landing.
Some of the calibration activity we worked on at this point were:
- Realistic-ness of predictions—One of our early projections contained wildly alarming predictions. We could have just accepted that at face value (and either extended the line of credit or started massive layoffs), but instead we chose to question the veracity of the measure. (Turns out, there were many systemic problems with assumptions and logic.)
- Looking for false positives and false negatives—In this case, a false negative is something that should have shown up in the measure, but didn’t. A false positive is a signal without an underlying cause. The former requires you look outside the system. Look for things that exist in the world, that should throw a signal and then look for the signal. If it’s not there, you have a false negative. The false positive is a bit easier, in that you don’t have to go out of the system to find them, but you may have to go outside the system to find out what caused the false alarm. It usually takes a lot of cycles to get false positives and false negatives down to a manageable level.
- Checking our predictions—Most prediction systems are point in time. This is your prediction of cash levels every week for the next three months. In three months, you will have a new prediction for the next three months. What you won’t have, unless you plan for it, is feedback on whether your prediction was any good. You must keep your predictions somewhere for this to be effective. The best time to check your prediction is when the final reality arrives. When week 12 finally arrives, you will know exactly how much cash you have on hand. What will improve your prediction is to look at what was your cash prediction for week twelve in each of weeks 1–11.
- Calibrate at 0—Inventory cycle counters know something we can all learn from: Things are easier to count and easier to reconcile when they are at 0. When the inventory system tells you that you have 2 items on hand, but you go to the bin and there are none, you know you have a problem. This is a false positive from above; the system thinks you have inventory that you don’t have. Note, if the system thinks you have 30 on hand, but you really only have 28, you are far less likely to notice. Conversely, every time the system signals on hand has gone to 0, it’s a good time to count. If you find any you have a systemic problem. Note the relationship to our altimeter analogy.
This whole calibration phase is one of the most important in the life cycle of the metric and often takes the longest. We often find this sends us back to the semantic definition (we thought we were clear on the inputs to the metric, but often reality tells us otherwise).
The power of calibration is what it sets you up for. Until you have a highly calibrated metric, you are reluctant and slow to take action. Traditional metrics are mostly consumed by analysts. When the metric is calibrated, we are ready to turn them over to front line actors.
Prescriptive Action
This is where we use the metric to drive immediate action. If your metric system is throwing false alarms all the time you really shouldn’t deploy notifications and especially not automated actions. Users of healthcare systems are all too familiar with “alert fatigue.” EMRs (Electronic Medical Record systems) typically throw off so many false positives that providers learn to ignore them (which is not the intended behavior).
If the signal is strong, with few false positives and few false negatives, it is a good candidate for notification and action.
It is at this moment that we begin to realize we may have the wrong user. Up to this point, the user was often some sort of analyst. Analysts are tolerant of bad data because they are doing research and figuring out what is really going on. We are finding, when we change the target user for our notifications, sometimes we need to change the metric again. As we will describe in the case study, sometimes at this point, we realize there is a different behavior we want to modify, and it may change again some details of the metric.
Root Cause
Switching to action mode gives us a new dilemma. We want to use this signal to take action, but what action? Often, the action is different depending on the root cause of the signal. But there is usually nothing in the signal to suggest the root cause.
This is where abduction comes in. Not as in alien abduction, but as in the third in the triad of induction, deduction, and abduction. Deductive reasoning is where we start with well accepted rules and apply instances to them to reach new conclusions. Semantic inference is deductive. Induction is where rules come from a large set of instances. Most science is inductive, research is gathered, and conclusions are drawn. The intent with inductive reasoning is to have a sufficient number of instances that the conclusions are statistically valid.
Abductive reasoning comes into play when we don’t have a large set of instance data to drive a statistically valid conclusion, but we need to proceed anyway. Abductive reasoning is generally taking a cognitive leap and proposing a likely cause of the observation (in our case the signal). A very good abductive hypothesis is testable.
As we will see in our case study, we want to take our signal and then apply what is usually a small amount of contextual information to propose a testable hypothesis for the cause of the signal. If it is easily testable, then we should test it. By working this feedback loop our abductive muscles get stronger.
Better Metrics
Finally, as strong signals and good abductive postulates start to take over, we find that often we’ve moved on and there is a higher-level metric that makes more sense to track. At one point, we were going to call these “meta metrics” because they seem to operate at a higher level, but we realized that this was going to get easily conflated with meta data, so let’s just call them “better metrics.”
Classically, cash flow management was a way that small companies managed their finances. As in our above example, cash management never goes away, but most firms find that managing the firm on an accrual basis is more actionable for day-to-day management, and cash planning is a secondary concern.
That was a generalization of an evolution we are seeing in many scenarios. We understand it is a bit abstract. You will need to work through the details in your own examples. The rest of this article works through the six phases in a common case study that may be familiar to many readers: aging accounts receivable.
A Detailed Case Study of the Evolution of a Metric
Some of the illustrations here are from a presentation prepared for accountants in architectural firms, called “Three simple systems to reduce chaos and overwhelm” by Mike Linn of Linn Accountants. Rather than make up something contrived, I opted to use these examples, which appear to be anonymized versions of financials that Mike has observed in architectural firms.
The realm is aging accounts receivable. You have a receivable when you grant your customer credit, which is another way of saying you let them pay later. Typically, you do the work and then send an invoice. At that moment you have a “receivable,” and the collective noun for them is “accounts receivable.” The amount of time from the sending of the invoice to the receipt of the payment is the “age” of the receivable (in days). The older a receivable is, the less likely you are to ever collect it. If you do not collect on an invoice, you must “write it off,” which means you did the work and didn’t get paid. This is a bad outcome. Every accounting system creates a report, much like the ones you are about to see called the “aged accounts receivable report.”
Let’s watch this common metric evolve through our six-step program.
Measure and Superficial Clean-Up
In Mike’s training class, this was the “before” AR aging.
Editor’s note: Mike Linn’s charts aren’t reproduced here, so the dollar figures quoted below come from his examples. The table is a simplified example of our own, with the same kinds of problems.
| Project | Current | 1–30 | 31–60 | 61–90 | 91 and over | Total |
|---|---|---|---|---|---|---|
| Project 1 | $6,000 | $4,000 | $1,500 | $11,500 | ||
| Project 2 | -$1,000 | -$1,000 | ||||
| Project 3 | $5,000 | $500 | $5,500 | |||
| Project 4 | -$7,500 | $7,500 | $0 | |||
| Project 9 | -$8,000 | -$8,000 | ||||
| Total | -$4,500 | $0 | $4,500 | $0 | $8,000 | $8,000 |
This is almost universal. For centuries, accountants have been bucketing unpaid invoices into these five categories based on age.
It is a bit of a mess, and not very helpful for anything. How can projects 2 and 9 have negative receivables? In most scenarios, this isn’t even possible, but here they are. More than likely the cash receipt process allowed someone to book a payment to the wrong project. Also look at project 4: There are negative receivables in the current period and an equivalent amount that is 90 days old. That is almost certainly a payment that wasn’t applied to the right invoice.
The real problem here is the data is of such low quality, you wouldn’t share it with anyone and there is little action you would take on it.
In the second “future” version, Mike has helped the firm clean up its misapplied payments and now has a clean looking aging.
Now we’re ready for the first level of behavior driven from data. There are typically two species of action that this drives:
- Calculating bad debt allowances—one of the main uses for the aged accounts receivable report is to predict and allow for bad debt. It is common to assign a percent to each column in this report. If we applied a common rule of thumb, we’d say 1% of current AR is going to end up uncollectable, 4% of the 30–60, 10% of the 60–90, and 30% of the 90+. For this example, this would give a bad debt allowance of a bit over $2,500, which is about 7% of their total outstanding. At this point, the metric is merely being used to predict a bad outcome.
- Collection—no one wants the bad outcome. Hopefully, the company is doing something to collect on these invoices. You want to collect on all of them, but you usually don’t call in the big guns until they get quite late. More on this later.
Formal Semantics
What starts happening, especially if you send your project managers out to your customers to collect on their “bad debt,” you start learning some things about your systems.
The first almost comical thing is figuring out what date you are using and what it means. Is it based on the invoice date, which is really: “How many days old is the invoice?” Most systems drive this report from the invoice due date. Take a closer look at your invoicing processes, and you’ll likely find a practice that many small businesses adopt, which is defaulting the due date to the invoice date. There is usually a note: “Invoices are due on presentation” or something like that, and so they don’t want to give the client another 30 days.
But this really messes with your metric. If the two dates are the same, you cannot distinguish days in AR from days past due. The deeper you get into this, the more you realize that you want to manage the latter.
Your project managers that you send out to do the collection will find out soon enough that, for instance, you have a master agreement with your customer with terms of 60 days from invoice date. If that is the case, you really should have a due date 60 days from the invoice date for those customers. They are not late until 61 days after the invoice date.
But what of the customers that don’t have a master agreement? You may still want to say the invoice is due on presentation, or perhaps in 30 days. Either way, you might have some experience with them over time and learn that client A typically pays in 20 days and client B in 45. You don’t necessarily want to make that your “due date”—perhaps you have another date that doesn’t show on the invoice called the “expected date.” This is going to be very useful for taking action and for accurate cash planning.
Calibration
Most of our payments now come from direct bank to bank ACH (Automated Clearing House) transfers. Some send a notification, others don’t. We’ve learned that, before declaring an invoice past due, we should check with the bank to see if it is there. That is one level of our calibration. We also keep an actual payment-received date, so that we can continually refine our estimated due date.
In 25 years, we have only had one account that we had trouble collecting on, so we are not calibrating our bad debt, we are doing everything we can to not have any.
Prescriptive Action
When an invoice crosses the past-due point, we know we have a problem. We are moving from an analytic timeframe, which is very much after the fact, to a real-time timeframe. Historically, invoicing and remittance processing were batch processes that we got around to when the more urgent work had died down. Running the aged accounts receivable had to wait until all the bookkeeping was done. And it had to wait on someone to study the report and consider whether it should be forwarded to anyone for action. Usually, a perfunctory email was sent.
Now, we are moving to a notification system. We will probably quit producing the aged AR report.
We have also noticed (from our root cause analysis below) that the invoice that we typically have the most trouble with and the most trouble calibrating is our first with a new client. The cynical part of me wants to believe that all the excessive systems validations, handoffs, authorizations, etc., were set up intentionally to thwart an invoice submitter, but then I’m reminded of Hanlon’s razor.
“Never attribute to malice that which is adequately explained by stupidity.”
—Hanlon’s razor
And here I am not referring to individual stupidity, but the collective stupidity of those who built the systems and procedures. We have learned that the first invoice is a learning process for both companies. We spend a lot of time understanding the process (often now we are required to submit our invoices through a third-party system) and the constraints (in one case our bank number had to be in a very exact position on the pdf of our invoice—most of our other clients now do not accept PDFs).
Having been notified, it is time to act. The action you take will probably be one of:
- Harsh warning or pestering
- Working out payment terms
- Threatening to cut off credit or stop work
- Actually cutting off credit or stopping work
- Turning invoices over to external collection agency
But deciding which to do is at least in part inspired by the root cause analysis.
Root Cause
At this point, we are no longer doubting our metric. If the metric says they are late, they are late. Now we move on to “why.” From a number of examples, we have concluded that these are the primary reasons that clients (at least, our clients) are late on a payment:
- Financial stress—the customer may not have enough money to meet all their obligations. This was our only source of very late payment in our history: A venture-funded client did not get the next round of funding they were expecting and were essentially insolvent but didn’t want to admit it.
- Dissatisfaction with service—sometimes clients withhold payment because they are not happy with service or want to extract some additional concession.
- Problems with their systems—we had a client ask us to invoice them in December to hit that year’s budget but then not pay us until May. Not for either of the above reasons—they were trying as hard as they could—but their internal system was very difficult to navigate.
- Problems with our systems—sometimes the problem is on our side. One that we discovered lately, for some projects we take the full project amount, divide it by the number of months and bill that amount each month. The only problem with this is when we find a client payment (through our bank ACH) we assume it is for the oldest outstanding invoice. On at least two occasions this wasn’t the case, and we were not able to find the problem until the end of the project. One of the payments was late for almost the entire duration of the project without being detected, and might have been missed entirely if we didn’t reconcile them at project close. This is far too late to be uncovering something like that, so in our new system, every invoice has a different dollar amount.
- Scheduling by nudging—some people are just squeaky wheel types and postpone doing administrative tasks until they are prompted. We find nudging a week before the due date to make sure “everything is set up right” is a good intervention for these.
- Their own metrics—some companies have their own metrics that factor into their payment priorities. Maybe they delay as long as possible, unless there are financial incentives. We worked with one company whose metric was they tried to keep their Accounts Payable higher than their Accounts Receivable and would drag out payments sometimes for that reason. We’ve heard of other companies that rank order their payables by incentives and penalties they will incur.
That array of possible causes suggests that different interventions (and different preventative measures) are going to be more or less effective depending on the actual root cause.
This is where making a testable hypothesis comes in. Based on some secondary information (maybe a credit report, maybe some other behavioral clues) you may choose a response and then see if the reaction correlates to the diagnosis. For instance, if you believed that your customer was prioritizing payment for those that had incentives, you could try putting a small penalty or a small gain (remember when there was a term called “2/10 net 30,” which meant you would get a 2% discount if you paid in 10 days, but in any event it was due in 30 days?). I don’t think anyone would offer those terms now, but even a half a percent might be enough to incent this type of customer.
Better Metrics
By the time the evolution comes this far, the metric has changed considerably. As you recall, it started life as an aging by buckets to predict bad debt for the bad debt allowance. As such, it was an aggregate report that was prepared monthly. It eventually became a notification of those customers who had, on the previous day, become late.
After a while it becomes evident that the first goal, trying to minimize bad debt, could easily be achieved by not issuing credit at all, but at what cost? We’ve worked with restaurateurs who do not accept credit cards because of the fees. We are almost certain the additional spending that comes with credit cards probably more than makes up for the fees. One suspects it has more to do with reducing tax reported income, but they are not likely to profess that as a motive. In a similar way, companies could eliminate bad debt expense by refusing to offer credit, but in most industries the effect on revenue would be stark.
Now that, for us, bad debt is all but a thing of the past, the new metrics that we are more interested in are:
- Estimated cash date—we are now much more focused on predicting the date we will receive cash for each invoice. At Semantic Arts we are going further, and predicting this at the time of contract, rather than the time of invoice, this gives us a much longer cash planning horizon. This metric isn’t an end user metric, but factors into the weekly future cash plan.
- Accuracy of predicted cash date—a secondary metric we are just starting to track is how accurately did we predict when the cash would arrive. This essentially gives confidence bands around our cash projection.
- Bad debt as a percent of sales—We’re not trying to provide an allowance for bad debt expense, nor a prediction based on age, but we recognize that bad debt may occur at some point. When it does, we’re more interested in it as a percent of sales rather than outstanding receivables. Bad debt is an overall risk of extending credit and should be viewed as such.
Summary
Companies rarely reflect on why they have implemented metrics in their enterprises; they just keep loading up BI tools and dashboards with ever more variation on things that are measurable and hope to get better results.
A few moments of reflection and we realize that much of the effort is intended to change behavior. As the metric becomes more accurate and more relevant to the locus of attention on the behavior change shifts. This shift is greatly enabled the more unified the data landscape of the firm is, and therefore the broader scope for behavior change.
This column outlined a progression that we think could have broad applicability and could aid practitioners in crafting metric interventions that are focused on ultimate causal behaviors and are therefore more impactful.
First published on TDAN.com in two parts, in September and December 2025.