12  AI Tools

You have been writing R for two modules now, and you have probably already asked an AI something about it. This module is about doing that well.

I said in the Introduction that you are encouraged to use these tools to learn, and that they are not permitted on tests. That policy has not changed. What follows is the practical version: what the tools are doing when they answer you, where they fail, and how to work in a way that catches the failures.

There is one question underneath all of it, and it is worth having before you read anything else. When you are deciding whether to hand a piece of work to an AI, the question is not can it do this? The answer to that is usually yes. The question is will I know if it did it wrong?

Everything else in this module follows from that. A task you can check quickly is a safe task to hand over. A task where a wrong answer looks exactly like a right one is not, no matter how capable the tool is.

Learning Objectives

By the end of this chapter, you will be able to:

• Decide whether a task is safe to hand to an AI, by asking whether you could tell if it went wrong
• Explain why an AI can be fluent and wrong at the same time
• Distinguish chatbots, in-editor assistants, and agents, and say what each asks of you
• Recognize the failure modes that show up most often in agricultural data work
• Say what data should not be pasted into these tools
• State the course policy on AI and the reasoning behind it

12.1 What These Tools Are Doing

A large language model predicts the next piece of text, given everything before it. It was trained by doing this over an enormous amount of writing, including a great deal of R code, until it got good at continuing text in a way that resembles what a person would have written.

That is the whole mechanism, and one consequence of it matters more than any other: the fluency of an answer and the accuracy of an answer come from the same process. The model is not consulting a store of facts and then writing them up. It is producing text that reads like a correct answer. Most of the time text that reads like a correct answer is a correct answer, which is why these tools are useful. But when it is wrong, it is wrong in the same confident register as when it is right. Nothing in the tone tells you which one you are looking at.

We are not well equipped for this. When a person answers a question hesitantly we discount it, and when they answer briskly and in detail we trust it more. That instinct is useless here.

12.2 Three Generations of Coding Tool

The way people use AI to write code has changed twice in the last few years, and the three stages still coexist. Knowing which one you are using matters, because they demand different amounts of attention from you.

The chatbot. You open a browser, describe your problem, and copy the answer back into your editor. This is how most people first used ChatGPT, and it is still how a lot of work gets done. The tool has no access to your files, so everything it knows about your project is what you thought to tell it. That is the main weakness — and, from a learning point of view, a quiet strength, because describing the problem clearly is most of the work of solving it.

The assistant in the editor. Copilot and its equivalents sit inside your editor, read the file you have open, and suggest the next lines as you type. You are still writing the code; the tool is finishing your sentences and answering questions about the code in front of it. The friction of copying and pasting disappears, which makes it much faster and makes it much easier to accept something without reading it. This is where the course sits, and the next chapter sets it up.

The agent. Given a task, the agent writes code, runs it, reads the error, edits, and runs it again — repeating until it works or gives up. Claude Code and the Copilot coding agent work this way. You hand over a task and come back to a finished change rather than a suggestion. This is how a great deal of professional software is now written, and it is genuinely useful, but it puts you in a different job. You are no longer writing code; you are reviewing it. You will meet this in industry. We stay with the first two here, because reviewing code well is a skill built on having written a lot of it, and you are still building that.

12.3 The Line That Matters

The distinction that matters is not how advanced the tool is. It is how much you understand of what ends up in your file. Simon Willison, a programmer who writes carefully about these tools, puts it as a rule for his own work:

My golden rule for production-quality AI-assisted programming is that I won’t commit any code to my repository if I couldn’t explain exactly what it does to somebody else.

By that standard a model can write every line and the work is still yours, provided you read and understood it. Accept code you did not read because it seemed to work, and you own a script you cannot debug. Willison calls the second thing vibe coding, and he is not against it — for a throwaway experiment it is fine. It is not fine for anything whose answer you plan to rely on.

This is the same test as the one at the top of the chapter, applied after the fact rather than before it. Could you tell whether this is right? If the answer is no, you have not finished.

12.4 How This Shows Up

Five failures account for most of the trouble in data work.

Invented functions and packages. You will be told to use a function that does not exist, in a package that does, or occasionally a package that does not exist either. The name will be plausible — read_excel_sheet(), summarise_by() — because plausible names are exactly what the model is good at generating. You find these immediately, because R throws an error.

Assumed column names. Ask for code to summarise your yield data and you will get something referring to yield, crop, and year, whether or not your file uses those names. The model cannot see your data. It is guessing from convention.

Out-of-date advice. Packages change. A model trained before a change will confidently give you the old argument name or the deprecated function, and the version installed on your machine will disagree.

Confidently wrong numbers. If you paste in data and ask for a mean, you may get a number that is close to right and is not right. Some chatbots now run code behind the scenes to do arithmetic and get it right; some produce the number the same way they produce everything else. You cannot tell from the answer which one happened. Ask for the code that computes the mean, and run it yourself.

American defaults. The training data is dominated by US agriculture, so the model’s plausible guess is a plausible American guess. It will reach for corn and soybeans, quote county-level figures where you want rural municipalities, cite USDA where Statistics Canada is the source, and describe crop insurance that works nothing like SCIC. Units are the sharpest version of this: canola is quoted in bushels per acre here, pounds per acre in some datasets, and kilograms per hectare in others, and a model asked for “typical canola yield” may answer in any of them without saying which. Our own dataset labels the crop Canola/Rapeseed, which is the sort of local convention no model will guess.

That last one deserves more caution than the first three. An invented function name stops your script. A yield converted with the wrong constant runs perfectly and hands you an answer that is off by a factor of fifty.

The first three announce themselves — the code fails, and you fix it. The fourth does not, and neither does the fifth failure, which is the one to actually worry about.

12.5 The Failure That Matters

Code that breaks is a nuisance. Code that runs and answers a different question than the one you asked is a problem, because nothing tells you it happened.

Suppose you ask for code to join your yield data to a table of prices. You get a left_join, it runs, and you carry on. What you did not notice is that four hundred rows had no matching price and are now NA, and your average is computed over the rows that survived. The script ran cleanly. The number it produced is wrong. You will not find out unless you look.

Nearly every serious mistake I have seen in student work — and in my own — has this shape. Not a crash, but a quiet mismatch between the question asked and the question answered. AI makes it easier to produce a lot of code quickly, which means it makes this easier to do at volume.

Nothing about this failure is exotic, and the habits that catch it are dull: read the code, count the rows, test on something small, check one number by hand. Section 13.6 sets them out properly, once you have a tool in front of you.

12.6 Where AI Earns Its Place

Being sceptical about these tools does not mean avoiding them. They are genuinely good at several things you will do constantly in this course.

• Explaining error messages. R’s errors are often opaque. Pasting one in and asking what it means is usually faster than searching, and this is the single best use of AI in this course.
• Naming the function you want. You know what you want to do and cannot remember what it is called. Describe it and you will get the name.
• Boilerplate. Date formatting, regular expressions, long case_when() chains — tedious to write, easy to check once written.
• A first draft of a script. Something to react to is easier than a blank file, as long as you treat it as a draft rather than an answer.
• Explaining a concept a second way. If my explanation of standard deviation does not land, asking for another one costs nothing.

Notice what these have in common. In each case you can tell quickly whether the answer is any good, either because R will tell you or because you know enough to judge. That is the question from the start of the chapter, doing its work.

12.7 Where It Does Not

• Deciding what analysis to do. The model will happily produce a regression when what the question needed was a cross-tabulation. It has no view on whether the analysis makes sense, because it does not know what you are trying to find out.
• Anything about your specific data. It has not seen your file. Everything it says about your columns is inference from their names.
• Judging whether a result is plausible. A yield of 4,000 bushels per acre will not strike it as odd. It should strike you as odd.
• Finding sources. Ask for references and you will get citations formatted perfectly, with plausible authors, in real journals, that do not exist. Check every one. This will matter when you write up the course project.
• Learning the material for you. This one matters most on test day.

12.8 What Not to Paste In

Everything you send to one of these tools leaves your computer. That includes what you type into a chatbot, and it includes the file you have open when an in-editor assistant is running, because the assistant sends that file to a server to work out what to suggest.

For this course it does not matter. The data we use is public — variety trials, StatCan tables, figures I have made up — and you can paste any of it anywhere.

It will matter the first time you work with someone’s real numbers. On a summer job, in a co-op placement, or helping with the books at home, you will have data that is not yours to share: a producer’s field records, a farm’s financial statements, anything covered by a confidentiality agreement or a privacy policy. A consumer chatbot is not the place for it. Depending on the service and the plan, what you paste may be retained, may be reviewed by a human, and may be used to train future versions.

The habit worth forming now, while the stakes are zero:

• Public or synthetic data — paste freely.
• Someone else’s real data — do not paste it into a chatbot. Ask about the shape of the problem instead: describe the columns and their types without the values. You will get the same code.
• Check what your editor has open. If Copilot is running and a spreadsheet of producer contacts is in the open file, that file is being sent whether or not you asked a question.

If you are unsure whether something is shareable, assume it is not, and ask whoever gave it to you.

12.9 The Course Policy

You may use AI freely for the module practice and for the course project. You may not use it on tests, where you will be working on your own.

The reason is not suspicion. It is that the tests are the one point where we find out whether you can read a data file, choose a summary, and tell whether the answer is sensible. If the tool does that for you, neither of us learns anything from the result.

There is one condition on assignment use that is worth stating plainly: you must be able to explain every line you submit. If you cannot, it is not your work, and you have not learned what the exercise was for. This is the same standard Willison sets for himself, and it is not an academic invention.

One thing that does not follow: this policy is for AREC 261 only. Other courses set their own rules, and some of them prohibit AI entirely, including for work you might think of as drafting. Read each syllabus. Do not assume that because one professor allows it, the next one does.

12.10 What the Rest of This Module Does

The next chapter sets up GitHub Copilot, which is the tool we will use in this course, and covers how to work with it inside Positron. The chapter after that puts it to work on graphing — a good place to start, for reasons that will become obvious once you try it.

TipOther resources — how language models work

• 3Blue1Brown, But what is a GPT? — a visual explanation of the prediction mechanism. Longer and more technical than you need, but the first ten minutes are the clearest version of this I know.
• Simon Willison, Not all AI-assisted programming is vibe coding — where the “did you understand it?” line comes from, and worth reading in full. His blog is a working programmer writing carefully about what these tools can and cannot do, and a good antidote to both the hype and the dismissal.
• Swarmia, Five levels of AI coding agent autonomy — one attempt at formalising the progression above, from inline suggestions to unattended fleets of agents, with the argument that higher is not automatically better.