The second to last paragraph of this useful book helped me understand my ambivalent feelings about it. After an admirable call for a commitment to scientific rigour in assessing what practices and policies work in business and government, the author writes “In this sense – and please excuse me for plugging my profession – data scientists are the world’s greatest untapped resource in both for-profit and non-profit environments. Through partnerships between businesses and academics, scientists and policy-makers, we can build practices that align the pursuit of both progress and profits in ways that benefit us all.” (p. 233)

My ambivalence was driven by being confused as to how to classify this book. In one classification, the book can be read as a practical guide for those aiming to work with data in business and government. In another classification, the book explains insights from economics on how business and government should structure themselves in an increasingly digital economy. Given the comment highlighted, I lean towards the first way of thinking. There is nothing wrong with this and the author, John List, is well positioned to write such a book.

John List is a Professor of Economics at the University of Chicago. He has a long and distinguished career focusing on experimental economics. He has done extensive research on various topics including education policy and charitable giving, and has held positions in the US government. He has also worked at digitally-focused companies such as Uber and Lyft. His background ensures extensive theoretical knowledge and many practical examples that provide a solid book outlining how companies should use data to make businesses that can grow and also how governments should make policies that are effective.

A key message from the book is that a business or social policy will only be successful if it can be scaled. Scaling is the ability “to achieve a desired outcome when you take an idea from a small group - of customers, students, or citizens - for example - to a much larger one.” (p.5)  Scaling, writes List “underlies all social and technological progress, since the innovations that change the world are those that reach the largest number of people”(p.6). The voltage effect refers to the need not to lose power (have a drop in voltage) as a business or policy scales.

List identifies five vital signs that suggest an idea can scale. These are avoiding false positives in surveys of first customers, knowing what your audience wants, identifying whether a business or policy depends on the skill of a few people, identifying spillovers, and avoiding cost traps. Weakness in any one can lead to a failure to scale. For my taste, the section on spillovers underplays the importance of networks. This would probably have been given more time in an approach that focused more on the economics of the digital economy.

Assuming each of the five vital signs is passed, four secrets to high-voltage scaling are discussed. These are largely from the behavioural literature. The secrets cover ensuring incentive structures work well for both customers and employees, thinking about marginal rather than average costs and revenues, knowing when to quit, and having a culture in the organisation that remains supportive of employees and customers as the company grows. This last lesson is most welcome as it is often overlooked.

Those who have worked in a behavioural insights team or tried to evaluate a potential new product by filling out the detail of a business model canvas, will find themselves nodding in agreement throughout the book. List makes his case clearly, combining summaries of academic literature and anecdotes from actual business ventures and policy making.

List stresses that many ideas will fail to scale. He notes that “according to Straight Talk of Evidence, a venture created to monitor the validity of research across disciplines ranging from, software development to medicine to education and beyond, between 50 and 90 percent of programmes will lose voltage at scale.” (p.12). He is also clear that a failure to scale does not mean that a business will be unsuccessful. He cites his own family’s trucking business. It provides a good living for those involved but does not have the characteristics of business that can scale.

There is also an emphasis - to the point of advocacy, as can be seen from the quote given at the beginning of this review - on the need for companies and government to use data and experimental techniques to test whether their businesses and policies can scale. When it comes to policy, he argues that “researchers must understand that the evidence-based policy mindset of two decades ago is woefully outdated. Today we need to create policy-based evidence. The opportunity cost of failing to do so is simply too high.” (p. 233) I interpret this as meaning that policies should be tested on small groups first to show evidence that they can scale. This means adopting an experimental approach and gathering data.

It is here that my interpretation of the book as a practical resource faces difficulties. I fear the book underestimates the difficulty, both within business and within government, of following this advice. Prof John Portes has written of his experience (https://bylinetimes.com/2021/04/09/race-report-sewell-commission-couldnt-find-something-it-wasnt-looking-for/ ) when he was Chief Economist at the UK Department of Work and Pensions about 16 years ago in gaining approval for a field experiment testing for racial discrimination. After refusal from two ministers because of the sensitivity of the topic, a third approved it observing “I’m going to regret this, aren’t I”. Hopefully attitudes have changed. There is some evidence they have. The UK Financial Conduct Authority has a policy of using field trials (see https://www.fca.org.uk/publication/corporate/how-when-we-use-field-trials.pdf ) and the Behavioural Insights Team (https://www.bi.team/ ) continues to provide evidence-based advice and field experiments across government and elsewhere. Still, it is not clear that an experimental and data driven approach is at the core of much policy making. Arguably, more change is needed.

When it comes to the private sector, there can be a number of problems in moving to a data driven approach. Field experiments take time, skill, and money to run and analyse properly. Few businesses have the patience. Managers need quick wins to justify the expenditure and do not like the uncertainty of potentially negative results. Further, some businesses rely on legacy computing, database systems and app designs that make it difficult to get appropriate data from experimentation.

My ambivalence should not be taken as a solid recommendation for this book. There is nothing wrong with adopting a data science approach to business and policy. In fact, it is increasingly clear that no economist worth their salt can work effectively without a better than average understanding of data science. Marrying the two disciplines is vital. Prof List does this well.