X Reasons for policy professionals to get into programming in 202X

TLDR: In 2019 I wrote a post with seven reasons policy professionals should learn to code. Now that AI can write reasonable code, I’ve had a number of people ask me if it’s still worth learning to program. Based on the evidence I’ve seen, AI appears to be a an amplifier, rather than a substitute for human intelligence in the policy space: meaning the people that get the most out of AI are likely to be those that know how to instruct AI to tackle a problem and tell good output from bad. So my answer is yes, learning to code still makes sense. Not necessarily so you can write the code yourself, but so you can make AI a force multiplier for your work.

Note: I’m intending to make updates and refinements to this post over time: both to correct the typos that I haven’t picked up and to incorporate feedback from other coders in the policy community. Feel free to reach out if you have suggestions.

How AI was used: I wrote the first draft of this post. AI was used (minimally) to refine how some ideas were communicated.

Background

In 2019 I wrote a listicle outlining why people working on public policy should learn how to program.

Putting aside that R python was probably the answer to which language to learn, the point of the post was to share some of the practical benefits I’d seen as an applied economist and policy advisor from learning to code.

Despite the listicle being one of my lower effort posts, it seemed to strike a chord: my inbox was quickly flooded with messages thanking me for articulating the benefits others in the public policy community had seen from leveraging code in their work. To my surprise, the article was also used as a reference for several university programming courses, resulting in my inbox still receiving messages from people starting their coding journey.

But a lot has changed since I first wrote the article: the UK voted to leave the European Union; the World Health Organization declared a global pandemic; and 25 movies in the Marvel Cinematic universe were released.

Oh! And a quaint lil’ startup called OpenAI launched something called ChatGPT.

And while there’s certainly a lot that could be said about the unfolding Marvel Universe’s implication for learning to code, this will have to be a subject of a future post. As today, I’d like to attempt to answer a question I’ve been increasingly asked since Artificial Intelligence (AI) tools and/or Large Language Models (LLMs) have made producing code trivial.

It’s sunk costs all the way down

At the outset, I’m clearly not an impartial observer. I was motivated enough to teach myself R, to write the original post and create a dedicated MOOC on using R for policy analysis. I also enjoy coding, so I have a vested interest in answering yes to the question. But, in my defence, these reasons also mean that I’ve thought about it a lot. I’m also a frequent user of AI in my work and find it has greatly expanded the scope and scale of what I can achieve.

This post is therefore my attempt to provide an answer to the question Is it still worth learning to code in 202X? My answer is, yes, but for different reasons to the 2019 listicle.

Doing smart stuff quickly

What got me excited about learning to program when I first started, was its ability to expand the scope and scale of analysis I could do and questions I could answer. In the world of public policy, there’s an almost unlimited number of interesting questions that could be asked, but rarely the time and data to answer them. Sometimes this is simply because the data didn’t exist, which makes learning to code not particularly helpful. However, often there are cases where data does exist, just not in a format that can be readily analysed with the commercial tools made available to you:

• Want to import thousands of individual excel files and merge them into a single dataframe?
• Have a repetitive data cleaning task that would take hours to do using a spreadsheet?
• Want to apply a statistical technique that hasn’t been added to your stats software?

Programming languages makes solving these problems trivial, but with the right prompt, so can AI.

Probability-based slop machines

But let’s be clear: at their core, LLMs are probabilistic slop machines. I don’t mean slop in the pejorative sense, but as a palpable description of what LLMs do: slap together intelligent looking responses based on next-token prediction.[1] I say intelligent looking[2] as LLMs have been trained to produce outputs that look intelligent, rather than to exhibit intelligence in a human sense:

“…AI systems have demonstrated their capability to solve cognitive ability test problems, primarily through guided training (e.g., Zhuo & Kankanhalli, 2020) or programmed approaches to transform problems into algorithmically solvable formats (e.g., Schmidhuber, 2004). While remarkable, it is debatable whether these accomplishments signify intelligence, given that the capabilities of most current AI systems are limited to specific programming and/or training data, without the necessary demonstration of novel problem-solving ability characteristic of human intelligence (Davidson & Downing, 2000; Raaheim & Brun, 1985). Consequently, many AI systems might be more aptly recognised as having the capacity to exhibit artificial achievement or artificial expertise…”

Gignac, G.E. and Szodorai, E.T., 2024. Defining intelligence: Bridging the gap between human and artificial perspectives. Intelligence, 104, p.101832 (link).

But sometimes a little bit of AI slop can be helpful. When producing code, I’ll often get AI to produce the first draft before making tweaks to the approach and style.[3] I use AI to help with writing and research. For this post I asked AI for good search terms to find research exploring the cognitive and economic benefits of learning to code, so I could check my assumptions and think clearly about what has changed since I first wrote the post. I then asked AI for ideas to better communicate key ideas presented in this post.  

“The majority of human production has always been slop. Mediocrity is not a bug of technology; it is the baseline of culture. The canon of art we revere today – those few thousands works in museums and textbooks – is the surviving tip of an immense iceberg of forgotten, derivative, or simply boring creations…”

D’Isa, Francesco (1 December 2025). “The Idea of ‘AI Slop’ Is Slop”. The Philosophical Salon (link)

If you’re wondering why I’m waxing lyrical about intelligence and the mechanics of LLMs, it’s because I think they help decide two things: which problems AI is likely to be good at solving and whether (or when) it’s worth learning to code. After all, if LLMs can already code better and faster than you, why bother learning how to do it yourself? Particularly if the type of intelligence AI outputs can mimic have sufficient crossover with our own to make it practically indistinguishable in the realm of applied economics and public policy.

Widening the scope, scale and quality(?) of analysis

Looking back at my original post, almost all the cited benefits come from programming languages changing the scope, scale and quality of what you can produce as an applied policy analyst. And if I had to put a number on it: of the seven reasons cited in the listicle, six can be achieved without having to write the code yourself.[4] Have some messy data that needs to be cleaned and reshaped before analysis? Share a sample with AI and let it write the code to take care of the data cleaning for you. Have research question you want answered? LLMs can suggest alternative statistical approaches you might not have thought of and write the code to implement them. Need to produce a set of publication ready plots? AI can write the code to do this too in a format that tells the story of your choice.

It would seem to be game over for learning to code and my 2019 post.

Of course, you already know that I don’t believe this but let me tell you why…

The jagged frontier of AI-assisted public policy analysis

Firstly, whether AI is intelligent or merely produces outputs that make it appear to be, early evidence suggests its ability to substitute for human intelligence varies across tasks. Researchers that tested the productivity benefits of AI termed this as a jagged frontier, that the users had to carefully navigate to benefit from using AI in their work.[5] It’s also not clear whether improving model performance is likely to expand the frontier equally across tasks, with many of the benchmark scores not necessarily generalizing outside test questions and being fragile to irrelevant and/or incorrect context.[6] Finally, at the time of writing this post, the types of tasks that frontier AI models had the hardest time completing successfully, looked strikingly similar to the problems encountered in public policy: messy, interdependent and heavily reliant on context.[7]

Secondly, while there’s solid evidence suggesting that AI can lead to productivity gains for knowledge workers, this isn’t automatic and appears to lean on the taste, judgement and experience of the user. [8] This was demonstrated by a field experiment in Kenya,[9] where a little over 600 entrepreneurs were randomly provided access to an AI-powered business assistant. The performance of those with the assistant was then tracked over time and compared to those without the AI assistant. The result: top performers gained around 15 percent from having access to AI, while weaker ones lost around 10. Entrepreneurs that benefited the most from AI appeared to be better at identifying and acting on contextually appropriate advice, while poor performers tended to adopt generic strategies proposed by the tool.

A much smaller experimental study closer to the world of coding pointed to a similar idea,  with both the 2025 (n=16) and 2026 (n=57) replication suggesting the productivity returns to using AI are ambiguous and vary based on the experience of the user.[10] Both versions of the study took the same basic form: experienced developers were randomly assigned to work with AI or not on their own repositories and the performance benefits were compared between groups. In the 2025 study, the authors found the performance benefits were on average negative: experienced developers took longer to complete the same tasks and overestimated the time saved.  In the 2026 replication,[11] the authors found performance increased on average, but not for all developers.[12] Although the authors note their estimates are likely biased downwards due to difficulties in recruiting developers willing not to use AI, they paint the same basic picture: the benefits of using AI are not unambiguously positive and depend on the skills and experience of the user.[13]  

“…The right analogy for AI is not humans, but an alien intelligence with a distinct set of capabilities and limitations. Just because it exceeds human ability at one task doesn’t mean it can do all related work at human level. Although AIs and humans can perform some similar tasks, the underlying “cognitive” processes are fundamentally different…”

Mollick, E. (2024, May 12). Superhuman? What does it mean for AI to be better than a human? And how can we tell? One Useful Thing (link).

There’s no accounting for taste

To be clear, my point here isn’t to suggest AI tools aren’t helpful, just that the size of the benefit depends on the judgement, expertise and taste of the user. AI certainly can suggest a variety of statistical techniques for answering your research question, but are they the right ones?[14] Sure, AI can quickly produce code for twenty reasonable looking plots, but do you understand the precise changes needed to accurately present your results and apply your corporate style guidelines and colours? Certainly, AI might simplify data cleaning, but does the approach make the right assumptions given the context, your research question and how the data was collected? And, while it might be true that research built on AI-generated code is more reproducible than a spreadsheet with a tangled collection of formulas, it may just be encoding hidden errors, the wrong methodology and creating a new source of technical debt you and your team will have to pay in the future.

In my original post I was excited at programming expanding the scale, scope and quality of analysis I could do. AI can help with the first two, but its influence on the latter seems to depend on the user. AI may be able to slap together a solid draft that gets you 90 percent of the way, which for some tasks is all you need, but deciding what good looks like and instructing AI how to help relies on building sufficient judgement, taste and expertise. 

Vibing fast and slow

The need to develop judgement, expertise and professional taste is also not unique to coding. But like musicians with sheet music, a mathematician with an equation and an economist with a set of national account statistics, anyone working with code is likely to perform better with some understanding of how it works, even with the help of AI. In the words of Andrew Ng:

“…people who understand the language of software through their knowledge of coding can tell an LLM or an AI-enabled IDE what they want much more precisely, and get much better results.”

And once again, there’s research that backs this up. For instance, pre-AI a meta-analysis of the effects of learning to code points to benefits to creativity and metacognition i.e. the processes underlying the monitoring, adaptation, evaluation, and planning of thinking and behaviour,[15] both which are plausibly linked to the ability to properly prompt and manage AI to perform tasks. A small observational study[16]  (n=100) seems to support this idea, with their analysis finding a positive association between vibe-coding efficacy and computer science (CS) domain expertise. Although the exact questions used to evaluate CS expertise aren’t explicitly outlined, their cited source points to them covering the same thinking and skills that learning to program helps to build,[17] with the authors concluding that effectively vibe-coding likely draws on problem decomposition and algorithmic thinking skills.

The book Messy Jobs also makes this point by noting that by cheapening the cost of cognition, AI simultaneously raises the value of judgement and taste.[18] The consequence of this for vibe-coding is that both instructing AI how to solve a problem and being able to judge the quality of outputs relies on knowing how code works, otherwise you might just be doing stupid things faster, in the words of Hadley Wickham:

“…LLMs and AI are, I think, a massive amplifier. You can certainly use them to amplify the worst aspects of yourself, but you can also use them to amplify the best aspects…”

Wickham, H. (2026, July 25). y code when ai? A talk on AI and coding. Tidy Design. https://tidydesign.substack.com/p/y-code-when-ai

Summing up

In my 2019 listicle, I argued that learning to code was worth it so you can answer a wider variety of questions more quickly and at a higher standard to traditional point-and-click software. Seven years later and AI tools can produce code for a wide variety of tasks more quickly than an experienced developer. But that doesn’t make what it produces good.

None of this means that I think you should avoid using AI before having mastered how to code, just that to be a force-multiplier it helps to know what good code looks like. Treat it as a brainy intern: motivated, well read, and in need of supervision. Give it clear instructions and it might come back with something useful. Don’t and its code might set the office microwave on fire. Learning to code can help you tell the difference between the two.

That matters because you’re accountable for the result as the human given the the instructions. Use AI to decide how to prioritize which grants to cancel and you might cause unnecessary harm,[19] trust its recommendations without validating them and you might just erroneously reinforce cultural stereotypes[20], and have it author a report for you, and it might just cite hallucinated sources.[21] If used well, AI can help you reach your destination quicker and expand your professional frontier. If used poorly, it might just having you producing more errors at scale.

I don’t know how long this holds. The studies here are small and not focused directly on the tasks we do in public policy. My best guess is that the jagged frontier will continue to expand, but unevenly, which will continue to place a premium on professional expertise and taste for tasks AI continues to struggle with.

So: should you still learn to code in 202X? Yes, but not for the reason I gave in 2019. Learn to code so you can tell good code from bad and steer the tool in the right direction.

---

[1]The push-back I’ve seen to this description of how LLMs work doesn’t change the core point. LLMs are trained to probabilistically predict the next token to produce intelligent-looking outputs at scale.

[2] LLMs are trained on the finished product of human intelligence, such as published research, books and social media posts, but not the thinking that went into producing them. Newer models that ‘think’ before responding to a prompt are meant as a partial workaround to this, but don’t change the basic point that LLMs are trained to mimic articulated intelligence, rather than the process of producing. 

[3] A practice that might have the opposite result if you’re not careful, see: Goldsmith-Pinkham, P. (2026, September 10). How do we know if writing is AI? Some initial analyses of AI’s impact on economics writing. A Causal Affair (link) 

[4] The one that may no longer be true is the trend towards open-source statistics. Certainly, I’ve been experimenting with ways to use AI to better extract government data (see example), but it’s entirely possible LLMs will make the internet a more difficult resource to navigate as text can be generated at larger scales.

[5] Dell’Acqua, F., McFowland III, E., Mollick, E., Lifshitz, H., Kellogg, K.C., Rajendran, S., Krayer, L., Candelon, F. and Lakhani, K.R., 2026. Navigating the jagged technological frontier: Field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. Organization Science, 37(2), pp.403-423.

[6] Akimitsu, P., 2026. Wrong and More Confident: A Field Experiment on Large Language Models Taking a Graduate Economics Exam. arXiv preprint arXiv:2607.23424.

[7] METR. (2026, May). Task-completion time horizons of frontier AI models. https://metr.org/time-horizons/

[8] Ethan Mollick uses the term Centaurs and Cyborgs todescribe the two most common approaches used by  workers that were able to navigate this frontier. In both cases, using AI effectively requires intelligently and selectively integrating the tools into their work without ‘falling asleep at the wheel’ (link).

[9] Otis, N.G., Clarke, R., Delecourt, S., Holtz, D. and Koning, R., 2023. The Uneven Impact of Generative AI on Entrepreneurial Performance: Evidence from a Field Experiment in Kenya (No. 24-042). Harvard Business School Working Paper.

[10] Becker, J., Rush, N., Barnes, B., & Rein, D. (2025, July 10). Measuring the impact of early-2025 AI on experienced open-source developer productivity. METR. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/

[11] Becker, J., Rush, N., Cunningham, T., Rein, D., & Mahamud, K. (2026, February 24). We are changing our developer productivity experiment design. METR. https://metr.org/blog/2026-02-24-uplift-update/

[12] The authors note it had become increasingly in their 2026 follow-up study to recruit developers willing to work without AI, which likely resulted in estimates of AI-assisted speedup being biased downward.

[13] Although the authors recommend their results are interpreted with caution, the 2025 study not only found that developers using AI were slower, but that more experienced developers tended to overestimate the time they would save from using AI.

[14]Like humans, LLMs exhibit systematic biases in the way they work and evaluate problems. This might come in subtle forms that influence outputs subtly, such as assuming doctors are more likely to be men than women. Or explicit, such as producing outputs that reflect how prominent a solution is in its training set, rather than necessarily being appropriate for the task at hand (link).  

[15] Scherer, R., Siddiq, F. and Sánchez Viveros, B., 2019. The cognitive benefits of learning computer programming: A meta-analysis of transfer effects. Journal of Educational Psychology, 111(5), p.764.

[16] Thorgeirsson, S., Weidmann, T.B. and Su, Z., 2026, April. Computer science achievement and writing skills predict vibe coding proficiency. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (pp. 1-17).

[17] Specifically, code tracing, code completion, conditionals, loops and logical operators, see: Parker, M.C., Solomon, A., Pritchett, B., Illingworth, D.A., Marguilieux, L.E. and Guzdial, M., 2018, August. Socioeconomic status and computer science achievement: Spatial ability as a mediating variable in a novel model of understanding. In Proceedings of the 2018 ACM Conference on International Computing Education Research (pp. 97-105).

[18] Garicano, L, Li, J & Wu, Y 2026, Messy Jobs: The Work That AI Cannot Reach, Upriver Press

[19] Alibašić, H., 2026. Introduction: The Imperative of Hybrid Intelligence in Digital Governance. In Hybrid Intelligence for Effective Digital Governance: AI in Administration (pp. 3-48). Cham: Springer Nature Switzerland.

[20]Saar Alon-Barkat, Madalina Busuioc, Human–AI Interactions in Public Sector Decision Making: “Automation Bias” and “Selective Adherence” to Algorithmic Advice, Journal of Public Administration Research and Theory, Volume 33, Issue 1, January 2023, Pages 153–169, https://doi.org/10.1093/jopart/muac007 

[21] Karp, P. (2025, November 6). AI-tainted Deloitte report was worse than previously thought. Australian Financial Review. https://www.afr.com/politics/ai-tainted-deloitte-report-was-worse-than-previously-thought-20251106-p5n863

The post X Reasons for policy professionals to get into programming in 202X appeared first on Giles.