QuickBooks And The Akcigit All-Stars: A New Small Business Index

A brand new Small Business Index, released by Intuit QuickBooks and an academic team led by Professor Ufuk Akcigit, is the latest entrant in the small business data sweepstakes. And it is impressive. But first …

To The Methodology!

If, when a new empirical analysis or data product is released, you immediately turn to the methodology section, you are not alone. Or, you are irredeemably nerdy. Or both.

And that’s good! The world of small business advocacy is simultaneously bereft of hard data and awash in surveys. Many of those surveys have been periodically covered here, and they offer some measure of insight into how small businesses are doing. A common experience among consumers of these surveys is to read through the exciting headlines and breathless takeaways and then have the methodological fine print cast some doubt on the purported findings.

This is a challenge facing all surveys, not only those concerning small businesses. And each small business survey has its strengths and weaknesses. The diversity of sample frames across them is enormously valuable given the wide range of small businesses in this country. Small business advocates routinely complain about “size standards” at the Small Business Administration and the borderline absurdity of saying that a firm with 450 employees is “small.” Diversity in survey samples allows them to account for this as well as other differences. A four-year-old restaurant with 20 employees may have little in common with a 40-year-old construction company with 20 employees.

Another strength of existing small business surveys is their consistency. A one-off survey of a few hundred small businesses won’t tell you much. A recurring survey of small businesses—either the same ones or close approximations—will be much more valuable, even if the sample size is small or restricted in terms of geographic or sectoral representation.

Administrative Data Versus Surveys

What small business analysis really lacks is a robust dataset on the companies themselves. One limitation of surveys is reliance on the time, memory, patience, and willingness of the respondent. It’s hard to know if a small business owner rushed through her responses or took the time to really consider them, even with multiple choice or yes/no questions.

By contrast, administrative data is based on the activities of the companies: payroll, revenue, etc. This data does exist—all small businesses, after all, file tax returns. But, for understandable reasons, the IRS doesn’t make that information publicly available. Over the last two decades, the Census Bureau has made enormous improvements in creating new datasets—Business Dynamics Statistics, Business Formation Statistics—based on IRS data. These provide a good sense, in the aggregate, of how American companies of different sizes and ages are doing. (The Statistics of U.S. Businesses dataset, also from Census, has great information, sliced by firm size, going back many years.) It’s only if you’re a sworn Research Data Center (RDC) user that you can actually analyze individual company-level data and link that to specific individuals and households.

Enter QuickBooks and the Akcigit All-Stars

Last week, Intuit QuickBooks released a new Small Business Index. The company has released data snapshots before—but this one is different. How? Let’s start with the methodology, of course. Here’s a few observations from the first two paragraphs of the Methodology page:

• “The Index uses purpose-built economic models to normalize anonymized QuickBooks data against official government statistics to reflect the general population of small businesses.”
• It is “calibrated against official statistics.”

Promising, but not quite a paragon of layperson clarity. So, what does all that mean? Basically, the research team takes QuickBooks data from a lot of small businesses (333,000 in the United States), all with fewer than 10 employees, compares the data (“calibrates”) to official government statistics, and—Voilà!—comes up with a pretty close approximation to the entire population of small businesses. (A full methodology paper has yet to be released.) Most encouragingly: “Unlike other small business indexes, it does not rely exclusively on survey data.”

As the report notes, the Index “is not a reflection of Intuit’s business.” That is, it’s not a tool that tells you how QuickBooks or Intuit customers are doing. The researchers make such a big deal out of “calibration” with official government statistics because they’re trying to create what doesn’t exist: a close to real-time gauge of how small businesses are faring in terms of employment levels and job growth.

It’s hard to overstate just how difficult this task is. There are something like 4.7 million small businesses in the United States with fewer than 10 employees. It’s nearly impossible to individually measure the performance of each. The research team has gone a long way toward cracking how to estimate it.

So … What Does It Say?

Enough methodological minutiae. What does the Index tell us?

Here’s something that jumps out immediately: monthly employment growth by small businesses with fewer than 10 employees has been negative for seven consecutive months, through February 2023.

Employment growth among these small businesses varies geographically: positive (in February) in five of eight regions and negative in the other three. It also varies considerably by sector, with strong growth in February in Utilities and shrinkage in the Information sector. (Further exploration of the results and findings will be done in future columns.)

The research team also includes Canada and the United Kingdom in the Index. Canada displays similar geographic variation and sectoral variation to the United States. In the United Kingdom, however (where the variable is not exactly the same, job vacancies versus employment growth, but close enough), every part of the country saw a decline in job vacancies among small businesses in February. In only one sector was there growth.

The Index is impressive: the people involved, the analytical task, the data achievement. The dataset goes back to 2015 and will, according to the site, be released monthly. This should be an essential tool for everyone who cares about small business performance and how to improve the public policy environment.