You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: TODO.txt
+1-111Lines changed: 1 addition & 111 deletions
Original file line number
Diff line number
Diff line change
@@ -1,93 +1,4 @@
1
-
## Harshil and Var
2
-
3
-
Here are some more thoughts on these two tutorials.
4
-
5
-
0) Both these tutorials are very different than a normal tutorial, which is usually centered around creating a couple of cool graphics. In a normal tutorial, we don't really care about the code. We just care about the final result. In fact, the main reason we even check the code is to give an excuse for doing a knowledge drop.
6
-
7
-
In tidymodels and marginaleffects, things are very different because we are teaching *concepts*. We want students to understand what a data generating mechanism is and how to use one. We are teaching ideas rather than just practicing making things. So, we have a lot more examples of questions which just require typing one or two commands. It is not clear if AI is even good at that! Probably easier to just tell students what to type. Indeed, these might be real "throwback" tutorials, ones in which the only time we have the students use AI is when they make a graphic toward the end.
8
-
9
-
In fact, there is an argument that these tutorials should make much more use of exercise code chunks. (Check out the r4ds.tutorials package as an example. Just install the package and run a tutorial for 10 or so questions.) But, at this point, we are probably too far gone in our current approach of everything in the QMD . . .
10
-
11
-
So, for both tutorials, most of the instructions will be "In the last code chunk, replace the current code with" or "Add this line of code to the pipe in the last code chunk" or whatever. Followed by a render and a look at what is spat out. We then share our code and also spit out the results, followed by the knowledge drop. AI plays no role until we make a plot at the end, which we only do with marginaleffects.
12
-
13
-
1) You may assume that students are always assigned both of them together, first tidymodels and, second, marginaleffects.
14
-
15
-
2) There should be four models created under tidymodels, one each linear, logistic, multinomial and ordered.
16
-
17
-
3) Var should use those same four models in marginaleffects, mentioning their origin in tidymodels. That is, they are defined in the set up and Var will have one question for each which is like: copy/paste this code into your QMD in a new code chunk. The question then becomes show_file(chunk = "Last"). Then, we show our version and what it spits out. Then the knowledge drop for each of those questions is longer than normal, reminding the student of what the model does in terms of outcome variable and so on.
18
-
19
-
4) At that point, Var's students have a QMD with four code chunks, each defining a model. Model names should be something like mod_linear, mod_logistic and so on. That makes it easier for students to know what kind of model it is.
20
-
21
-
See below for more discussion.
22
-
23
-
24
-
### Var
25
-
26
-
`marginaleffects`: https://marginaleffects.com/
27
-
28
-
Read chapters 2 and 3 to get background.
29
-
Avoid chapters 4 and 7
30
-
Key chapters are 5 and 6 (Don;t worry about tests) Don't worry about slopes.
31
-
Check out chapter 15
32
-
33
-
Difference between marginal and conditional predictions is critical! And confusing!
34
-
Difference between `by` and `condition` is critical.
35
-
36
-
Maybe basing the whole thing on 15.1 and 15.2 makes sense.
37
-
38
-
And counter-factual stuff is interesting too!
39
-
40
-
I think that Var will have several sections, each of which uses a marginaleffects function and applying it to each of the four models.
41
-
42
-
* Simple predictions()
43
-
44
-
This runs predictions with `data` set to a single observation, created by hand. This shows the simplest thing that predictions() can do. Does this for all four. Then it runs predictions() with `data` set to an object with two rows, where at least one of the rows matches an observation from the data, presumably one of the observations near the top. The purpose of predictions() is to take a DGM and then use it to predict what would happen to a unit with the specified values for the right hand side variables.
45
-
46
-
Models other than linear are tricky, as we discussed, because they have more than one outcome, so predictions() run on a single data unit will produce two rows for logistic and multiple rows for multinomial and ordered. And that is OK. The purpose of this section is to understand the simplest use cases of predictions().
47
-
48
-
Along the way, we show the class() of the result from predictions() and also what happens when you pipe the result of predictions to as_tibble().
49
-
50
-
* predictions()
51
-
52
-
Now that we understand how predictions() work, we can run it on larger numbers of inputs. There are two cases to consider. The default, which feeds back in all the data in the input data set, is the most important. The second is when we use data_grid (?) or similar functions to make a large number of units which follow some pattern which we are interested in. Maybe, first pass, we don't provide any examples of this case, but we do mention it in a knowledge drop.
53
-
54
-
* plot_predictions()
55
-
56
-
Now that we understand what predictions() produces, we can understand what plot_predictions() is all getting at. But boy is it confusing, not least when it comes to more complex models and/or more complex data inputs than the default. And don;t forget to highlight the differences between `by` and `conditions` as well as the difference between marginal and conditional. I am still hazy on those myself!
57
-
58
-
And that about covers it! With each of these sections covering 10+ questions (because we need to cover all four model types each time) --- and those questions not being that easy --- we really don't have room for more, at least in this tutorials. Eventually, we might have turn this into marginaleffects-1 and add a marginaleffects-2 which would cover the comparison() family of functions.
59
-
60
-
61
-
### Harshil
62
-
63
-
`tidymodels`: https://www.tmwr.org/
64
-
65
-
Maybe this is better source? https://www.tidymodels.org/start/
66
-
67
-
Link to both.
68
-
69
-
Goal is to understand the overall tidymodels framework. How does it work? What does it mean?
70
-
71
-
useful to review chapters 1 through 3
72
-
73
-
Students need to understand linear_reg(), logistic, multinomial and ordered.
74
-
75
-
Then discuss set_engine()
76
-
77
-
fit() and what that means. And then we have the model object.
78
-
79
-
parsnip package is important. Discuss mode. See https://parsnip.tidymodels.org/
80
-
81
-
Discuss tidy() for looking at coefficients.
82
-
83
-
Once you have a fitted model, always show the latex math representation of that model. This is the DGM!
84
-
85
-
So, we might have four sections, one for each model type. Each might feature 4 or so different versions of the model, with the difference being a change in which right hand side variables are included. (Each time, you show the mathematics of that model, including the estimated coeffficients.) This is one of the great difficulties of data science! There are many possible DGMs for a given problem. We never know which one is true! We just use our best judgment. (And, in general, we choose the one in which the coefficients of the right hand side variables have confidence intervals which exclude zero.)
86
-
87
-
Each of the four sections has knowledge drops which recapitulate the themes in my 1,000 word essay, with each successive round being more sophisticated.
88
-
89
-
90
-
### More misc.tutorials
1
+
## Var
91
2
92
3
Our next project is to write a few more misc tutorials. We need to give students more practice in just making things. Advice:
93
4
@@ -104,27 +15,6 @@ Our next project is to write a few more misc tutorials. We need to give students
104
15
* The tutorial names will be the basic topic, like `baseball`.
105
16
106
17
107
-
### Progress
108
-
109
-
* baseball — DONE (first project-tier tutorial; its project type is a
110
-
multi-page Quarto website). Site is index + home-runs + sluggers, using
111
-
the Lahman R data package directly (no extdata/manifest). Future
112
-
project-tier tutorials may instead build Shiny apps, Quarto dashboards,
113
-
etc. — same tier, different project type. Anomaly: total HR appears to collapse in 1981/1994/
114
-
2020, but those are shortened seasons (strike/COVID) — normalizing to HR per
115
-
game reveals the real eras (dead-ball, 1920 jump, WWII dip, steroid era,
116
-
2019 juiced ball). Sluggers page ties 9 of the top-15 career HR leaders to
117
-
the steroid-era spike. See CLAUDE.md "Website tutorials" for the recorded
118
-
overrides (website artifact, AI-prose-as-content, topic-driven not storage-
119
-
driven, package-data exemption from extdata).
120
-
Open follow-ups for baseball:
121
-
- Lahman updates ~yearly; the "1.39 in 2019 / 762 Bonds" numbers in the
122
-
knowledge drops are version-pinned (Lahman 14.0.0, data through 2025).
123
-
Re-check the KD figures if the package is bumped.
124
-
- Consider a Statcast/baseballr appendix to give caching a genuinely
125
-
expensive chunk (currently the cache arc is on a fast aggregation,
126
-
justified as habit-building like .gitignore).
127
-
128
18
### Possible tutorial topics
129
19
130
20
I am flexible on the topics you might select. Anything which at least some college students might be interested in is good. Especially interested in topics which students might use for final projects.
0 commit comments