I'm teaching "Data Science" to computer science majors next spring, and I'm overwhelmed by the prospect of what to teach them in the age of AI. It seems like Fable already way outperforms anything I could historically expect my students to do at the end of the semester, and perhaps even outperforms me. I can describe a vague analysis on the data, and it will handle every step: transformation, preliminary analysis, formulating more precise questions, creating visualizations, running statistical tests, interpreting results, thinking about stakeholders... And in some of those steps, it finds things that I might not have considered.
I wish I had a bigger N so I could rely on that experience a bit more, but it sure does feel like I'm not really contributing anything in terms of "Data Science expertise" in my own research right now.
Plus it worries me that people will no longer know how to do anything basic and we will at some point in the future be slaves to AI if we aren't careful.
If it wasn't for grinding all of the hard problems and hands-on projects and various programming tasks I had to do by hand over the years, I wouldn't be able to keep up with the agent, and wouldn't be able to get good results out of it, because I wouldn't know what it was doing and wouldn't be able to steer it appropriately. My schoolwork 15 years ago was more than formative, it remains critically foundational for my job work today.
Based on the other comment it sounds like maybe I need to give Fable another chance, and then I suppose I won't have to steer anything at all. But at the end of the day I am not paid to be a meat proxy, I am paid to deliver things for the business that employs me, and the things I deliver are my responsibility, not Anthropic's.
So I say, embrace AI as a learning tool, let students use it to bypass programming on assignments where the programming is of secondary importance, require oral defense/presentation for projects, make homework worth a much smaller fraction of final grade, and make all of your tests in-class and hand-written.
The reality is that the entire nature of education has to change to account for cheating becoming essentially free and virtually impossible to detect. Grinding through a problem set in the library is still tremendously important, but it's going to be harder to motivate students to do it without just cheating. Some students are conscientious and will do it anyway. Some won't. The hard part is going to be crafting curricula and exams that separate the two kinds of students. Fortunately you share that problem with just about every other professor in the world right now, so at least you shouldn't feel responsible for solving it all yourself.
Relevant anecdote: I used Wolfram Alpha to cheat on my calculus homework in freshman year, and then I got my butt kicked in the exam and a C final grade that never quite averaged its way out of my GPA. That felt horrible and it was a strong motivating force to get my shit together, but that's only because I'm a fucking nerd and I really cared about learning in school and making a career out of what I learned. On the other hand, I really was struggling with the material, which is why I tried cheating in the first place. Maybe if I had today's AI I could just ask it to help me learn better.
> Maybe if I had today's AI I could just ask it to help me learn better.
You could! I use it all the time to generate pre-calculus and physics quiz sets for my son to work if he needs extra practice on particular concepts. And that's just scratching the surface.
Getting a kid motivated to do that _for themselves_ rather than just getting the answer and moving on to scrolling short-form video? That's another matter.
And especially where the fuzzy edges and blind spots are—the limitations of the methods, and the aspects of problems that are essential to the question but hard to quantify or test or reproduce.
I feel like a lot of people are going to put the same faith in their new quantitative superpowers as psychologists did in their 95% confidence intervals [0], and managers did in their “scientific management”/time-and-motion studies, and American political pollsters did in their 2016 presidential prognostications, and Rudder’s OKTrends blog (delightfully) did in calculating the answers to romance [1]…
From my experience working with data scientists, it outperforms humans on an extreme level. I’ve seen so much bad data science, and here I have a model that can do all of that from one prompt. It is getting really good on finding caveats in data too.
Is it reasonable to take up some of the class with statistical basics and intuition? E.g. you just referred to a bigger N; I don't know what level your students are but do they have a basic intuition for notions like "variance of estimator"? Or is that another class? I think my point is that even if they can nowadays press a button and have the most amazing reports and visualizations produced, that doesn't give anyone any actual understanding of the theoretical side of data science.
I'm so glad I learnt it before LLMs. I can't imaging trying to get good at Pandas/Polars coding nowdays when so much of my analysis is "Fable, please do this for me, k thx bye"
Wait, wouldn't this actually be fairly simple to set up? I'll admit my Git knowledge is a little shaky, but couldn't you set up a Hook that runs when you commit a Docx/Pptx/etc. file to unzip it in memory first, and then another Hook when you checkout to zip it back into the original structure? I guess conflicts are the major issue... asking users to navigate the XML/binary structure stored inside could be a mess. A GUI could help, but that would invoke different issues.
Corridors of Time is still sublime. After I heard that song I had to figure out the notes. Turns out there's this random tool that turns game music rips into midi, so I was able to figure out the chords in that song. I still play it from time to time.
Yasunori Mitsuda's music has had a lasting impression on me. From Chrono to Xenogears and beyond, I had some strong feels associated with his music. Thanks for sharing that link :D
Online exams are a pointless waste of time and invasion of privacy, designed to make money without measuring anything. It encourages good people to cheat. If you don't have in-person proctored exams, you are wasting your time.
Nothing and no one forces you to cheat, it's a choice; an admittedly common choice, but then so are a lot of immoral acts like lying and stealing. I'm not saying that online tests are a good idea because we have to live with the sort of people who cheat, but lets not pretend that this turns "good people bad".
Good people don't need excuses to be good, and don't respond to excuses to be bad... if you want to simplify to that extent.
I wish I had a bigger N so I could rely on that experience a bit more, but it sure does feel like I'm not really contributing anything in terms of "Data Science expertise" in my own research right now.
reply