Beginning a simple stats gathering app in Django

I just pushed a commit to my worksheet generator website containing a new app: something I’m calling “tobystats.” (Guess what my first name is.) The idea is simple: to try and collect the kind of stats that I’d like to have as I think about intentionally growing the site to be useful to more than just myself. (I mentioned the idea already.)

What stats would I like to collect?

In all honesty, this is probably a poorly-conceived project. I mean, I don’t know what I want to know. I do know that I’d like to start working on my landing page and create something of a conversion funnel to attract new users.

To that end, I’d like to know how many people who visit my site either 1) log in (they’re returning users and wouldn’t have clicked through, anway) or 2) click through to a “learn more” or “try it free” page.

Eventually, the same app should also be used for something akin to event logging. I’d like to know how many total worksheets the site has made. (It would be fun to celebrate a thousand worksheets made, or whatever) as well as counting other events. This is not something I’ve added, yet, but will.

How am I doing this?

The goal is to be able to add a single line to the views of pages that I’d like to track. Right now, that line is:

from tobystats import tobystats

tsid = tobystats.process_request(request, 'LandingPage')

And then the tsid string that’s returned is passed through to the template. (tsid stands for “tobystats id”), where it’s appended in the query string in links, like this:

<a href="example.com/link/blah.html?tsid={{ tsid }}">Example link!</a>

So, the idea is simple: when the process_request() function is called, it checks if there’s a tsid already associated with the URL. If there isn’t, we assume this is the first page that’s been loaded and create a new SiteVisit object (SiteVisit is the model I’m using to store the basics of a single visitor–eventually I’ll use some kind of IP geo-lookup to get the city and country of origin and store it there as well) and a PageView is created for that specific page, storing only the name of the page visited (this seemed to make more sense to me thatn URLs) and the date and time. Lastly, the tsid (a random five-character string) is generated and associated with the SiteVisit and returned to the view that called it.

If there’s already a tsid in the query string, it’s even simpler: look up the SiteVisit with the corresponding tsid and create a new PageView object associated with it. Job done!

I still have to make a dashboard

So far, all of the stats reporting available to me is the number of site visits and the number of those which were successful logins (the idea is that we’re going to only worry about what happens to visitors who don’t login–they’re the ones at whom the messaging is directed).

There’s a lot more to look at: how many visitors are identified by django-user-agents as bots (turns out that, within moments of uploading my stats, I was seeing referer spam), how many of the non-bots clicked through to the “learn more” page. If each step in the signup/initiation process is logged, it would be interesting to see how any people get fatigued by it and at what point. (Also, with geo-lookup, I could skip a step in the signup process).

It’s just a start

I’m sure there is more I can do–see the comment above about using geo-lookup for IP addresses. However, it was important to me to push something to the production site as soon as possible to just get a baseline for current usage. After all, if I don’t measure now, how will I know how many more visits are coming?

Also–and this may be the point of the whole thing–it’s nice to have a coding project to tinker with as I have time.

Making my own stats

I live in Germany. I can’t click on a single website without getting a “we use cookies to enhance your experience” message and being required to consent before I can see whatever content I actually want to see.

Cookies, am I right? Nobody likes them.

For a while, I’ve been saying in conversation that it has to be possible to collect the data you need without using cookies (though, I guess I’ll continue to use them to keep users logged in). So, while I’ve played around with Google Analytics (and believe I still have it on the worksheet generator page), I never nerded out on it.

My idea

It’s not super-scalable or a product I can sell, but I’d like to make my own system for trying to get some insight into user behavior. The idea is something like this:

  1. A user hits my webpage: If the querystring doesn’t already have a unique identifier in it, the system generates a unique ID (I’m thinking six digits like “a8d0zw”) and records things like the time and IP address in a database entry.
  2. This unique ID is passed through to the Django template: It’s inserted into all the links, so that the next page knows which “session” this pageview belongs to. This could be used to record how much time is spent on each page, and, ultimately, how the user moved through the site.
  3. Individual links can also have identifiers: Here, I’m thinking about the fact that I plan to use two different CTA links on the re-written landing page. It wouldn’t be a lot of work to see if people typically scroll to the bottom before clicking, or click at the top.
  4. Logging-in would have to be accounted for: The login mechanism would have to include this, so that page hits that resulted in logins could be separated from page hits that were generated by things like paid advertising.
  5. Finally, I’d have to write code to make sense of everything: It would be on me to make my own dashboard that would parse the information out of all of this data.

Next steps

It doesn’t seem overwhelming, does it? I figure this is a project I can do to practice, and to keep at coding while I’m also working on things that aren’t code intensive. I get that it’s not going to be the kind of thing that I’ll be able to send others, but I like the idea of making my own stuff, and of disconnecting from Google Analytics.

So, I guess the next thing to do is to write out the literate programming plan for what I want this to do, and then implement it.

I’ll keep you posted.

Rewriting the landing page

The Marketing Examples newsletter hits my inbox irregularly, but it’s always been valuable. I’ve re-read the Guide to Landing Pages That Convert several times.

Wanting to get out of a winter/Corona funk, I’ve decided to invest some energy in sprucing up the Worksheet Generator. I still think it’s my best chance to convert something I’ve made into a second income. And, reading the examples he provides, I think “I can do that.”

Sitting down to actually do it, it’s much harder.

I use too many words. Still, I’ve tried to cut things down. The current landing page is at dynamic-efl.com, you can have a look a that, if you’re curious. Once you’ve seen that, here’s what I have:

The unformatted landing page

Above the fold:

Title: Provide your learners with tailor-made vocab review and save time.

Subtitle: Review worksheets so specific, they’re single-use. Made with only moments of effort. (Learners will think you spent hours.)

Instead of social proof: Vocabulary isn’t easy for learners, and no amount of work you do will make it easier. You can give some structure and a clear review task to do.

Visual: Video showing Dynamic-EFL in action (not a walkthrough, just moments of it in use) (see below)

CTA: Button: Try it for for 3 months, no charge Subtitle: (not even a charge card)

Below the fold:

Make it concrete: Manage a vocabulary list for each class and generate structured review worksheets and one-off review activities.

Classroom vocabulary rotates through different activities, coming back again in the next worksheet to refresh learners’ memories. Use the same vocab list to make crosswords, word searches, memory, and more.

Frequently asked questions:

Do I really want to give my learners another resource printed off from the internet?

No. That’s why I don’t brand the worksheets. In fact, the goal is to make this look like you did all the work. Use your own branding or change the text on one worksheet–or on every worksheet you make. 

Do I really have time to do the extra work of making worksheets for each class?

I don’t know. I can say that, not counting writing down the classroom vocabulary as we go in class, I invest about two minutes per class. The secret? Most of the work has already been done, and the rest is easily done by a computer.

I have my own way of explaining things. Should I worry that yours will confuse my learners?

Probably. I think you can use 99% of what I’ve written as it is–especially if you’re teaching Germans (they’re who I write for)–and I’ve included tools to let you tweak things as much or as little as you think you need to.

I don’t really have time to learn something new. How difficult is this to learn?

Not very. I’m a teacher and built this for teachers. If you can type vocabulary and click on “make a worksheet,” you already have the skills you need.

Second call to action:

Let me start making your review worksheets and I think you’ll see the value of what I do. I’m so convinced that I’m ready to do it for free.

Button: Try it free for three months

Subtitle: (no credit card required)

Founder’s Note: I’m a teacher, first, not a coder. But, when students told me they “couldn’t learn vocab” or were “too old,” I knew that my standard bit about how to learn vocabulary didn’t help, it just made me feel better by making it their problem.

I teach adults: none of them needed another problem. They needed a teacher who would step up and find a way to help

I began to make structured review activities. They were a hit. Students saw their own success and thought I was a big part of it. It felt good to be a part of their success story.

But it also ate time. Lots of time.

So I thought of automating it with a software product. And, when I realized what it would cost to hire a coder to make it for me, I taught myself to code. (We all have dry spells in freelance work, right?)

Very quickly, it went from a time-suck to a time-saver. I had time to help my students in other ways (see my collection of stories and other worksheets), but I still think of this as my greatest teaching idea, yet.

Automating it doesn’t only mean that I can produce great worksheets in minutes. It means that anyone can.

Video script:

The idea is simple. As I teach, I write down the words that I’m introducing–and also the vocabulary that comes up as learners talk about their own stories.

After class, I type the words here into the classroom’s vocabulary list. That’s done in seconds.

Then, I double-check that the vocabulary is being understood correctly–a table can be furniture, but it can also be a tool to format a document–and I’m done.

For a worksheet, I click and skim through the worksheet that the system wants to make for me.

When I got that much done, I was proud of myself. The magic is in the next worksheet, though, when vocabulary from the previous worksheet comes back.

That’s the gist of what I do in a nutshell, but if you want to keep watching, I’ll keep talking.

From the same vocabulary list, I can make a memory game… a crossword puzzle… a word search… or a vocabulary puzzle…

When vocabulary enters the classroom list through one student’s funny story, I can make that part of the definition of the word… or I can make that part of the gapfill sentence. Learners see that one, small personal touch and think that I made the whole thing.

I mean, I did make the whole thing–these are translations, definitions, and gapfills that I’ve created over years of developing this–but your learners will think that this was all your work. You can reenforce that impression by putting your branding on the worksheets, or by setting up your own default definitions and gapfills.

(I timed a read-though of this about just about two minutes)

Next steps

You’d be forgiven for thinking that the next thing for me to do is to just copy and paste that in some format on the landing page and see what happens. But I have a grander vision.

I’d like to take my time and format everything new with my newfound CSS skills from freecodecamp.org. And, while I’m doing that, I’m thinking about how to do my own stats.

Feeling Inspired by a Changelog

In what is material for another post, I recently decided to take another class in Django. And, in lesson two of Django for Everybody, I thought “I should start a project to practice on.”

That lead to me thinking it wouldn’t be all that much work to get a barebones version of Governors and Generals going. How hard could it be, right?

Turns out, it’s not super-hard–the literate programming preparation I did really helped–but it’s hard to know where to stop.

Enter the eldest son. He’s been looking over my shoulder as I do these things and I try to always have time to explain what’s going on. Some of his suggestions are our of my league (“there should be a 3D version to download”) but on suggestion changed the way I’ve been working: He said I should put a version number at the bottom.

So, I did. It was simply a matter of altering the footer template. And then I thought: “this could be a link to a changelog page that lists the stuff added. Then, I could look back to see the work I did.”

That was done pretty quickly.

Today is my third day of working on the project and I’ve found it to be quieting to finish the day’s portion of work by writing short, bullet-point entries about what I did that day. It’s nice to think “yeah, I got some stuff done.”

What’s more, starting at version 0.0.1 I’ve decided that version 0.0.2 will be reached when I’ve made something to put on PythonAnywhere and ask people to have a look at it. Initially, it’ll just be the framework on which the game mechanics will be built. Using that, I’ll start putting game mechanics together, though I expect there will only be intermittent days of intense coding after that. (At least until I start learning new stuff and updating the front end.)

Literate Programming

To be clear, I don’t think I’ve written a line of code yet this year, at least, not that was for a legitimate project and not for FreeCodeCamp.org.

Still, I work with people who code and, when I was thinking about the Governors and Generals thing, I asked about how to plan a program. One of my colleagues–fresh out of college–talked about using UML to model stuff. Then, one of the more experienced programmers heard us talking about it and told me to Google Literate Programming.

Short aside on Google suggestions

There was a time when I thought people telling me to Google something were just saying “you’re not worth my time.”

Getting older, I realize they’re saying “you seem to be teaching yourself this stuff, but here’s a concept you may not know to learn about. Have a look.” And I appreciate that.

Now, as a rule, when I have the idea something is possible and I don’t know what to search for (I want to update a website dynamically in a certain way) I tend to ask “do you know what I need to Google to figure this out?” It respects the other person’s time and makes it clear that I’m not looking for them to do the work for me.

So what is it?

The idea is that, before you write a line of code, you write out your program in English. The guy who pointed me towards it said that, eventually, nothing should be a mystery: you just have to translate English into code.

That means that you write “here, the cards in the user’s hand are selected and passed through to the Django template to be displayed,” because that’s something I’m confident I can do in straightforward code without having to figure out how it’s done.

The way he sold it to me was that you construct the entire program in plain English before writing anything, so you know what you need to get done and then you just translate it into code.

I’m giving it a try with Governors and Generals

I like the idea. But then, I can be sold on “literate” anything. Because I’m tired of pandemic stagnation, I’ve decided to kick myself in the butt and get an MVP going.

Now, sitting down and pounding out what needs to get done… I’m realizing that there’s a sense to it and I’m excited to see where it takes me. I would like to get back to being a person who makes things.

I’m gaining diabetes chops

I feel weird being “proud” of being good at diabetes. I mean, I’m not super proud to have it in the family–I’m not ashamed, either, it’s just a thing for me to deal with,like gray hair–but it’s a special combination of being both a monotonous, frustrating grind and super scary.

It’s been a while since I’ve reflected on all of this in writing, so I thought I’d try to make a list of the things that I’ve learned or am trying to do.

  • Bad sugar is not (always) his fault. I come from a family that expects kids to take responsibility for themselves and my first instinct is to be angry at bad sugar. That’s not super helpful and it’s not always his fault. I don’t always estimate sugar correctly, and I would be the kind of person who forgot a shot when sweets are passed around at school. Being angry doesn’t help. At worst, it probably makes him not want to talk about his sugar.
  • I try to connect his blood sugar to immediate things. This is hard for me because he’s an eleven year-old boy and not super aware of his feelings, anyway. Asking him to sense if he feels different with his sugar high or low–when it’s not an emergency–hasn’t been super helpful. However, I do point out that he had a rough day–and that’s the kind of thing he knows–and I say his sugar was high all day. If I’m doing this correctly, I’m not rubbing his nose in it, but just saying “one of the strategies you have to avoid bad days is managing your sugar.”
  • I try to avoid telling him to take shots. Partly, this is because I want him to learn to get shots on his own. What I’ve been doing for the last weeks is noticing that his sugar is high and saying “hey, this is a great time to do a workout video.” I try to not make it a punishment, but instead to say “then you won’t have to eat any carbs to do a video later, and we want to stay active during the lockdown.” My goal is not to present the workout as a punishment, but to have him notice that my feeling a need to jump into his sugar management is less fun than him managing it himself. (And, to be fair, I also run and do the videos that I ask the kids to do, and he would be doing videos anyway.)
  • I’m getting older and wiser. This doesn’t really belong on the list. I don’t do anything, but I’m gaining experience. I can give a shot before going to bed and don’t feel the need to set an alarm to make sure it wasn’t too much. I understand diabetes–at least, his diabetes–enough to take an active role in the “why didn’t his sugar do what we expected?” conversations, and that makes me feel more competent as a dad.

Christmas is a rough time to have diabetes and, to a lesser extent, rough for the people trying to set up a nice Christmas without sending the kids on a diabetes roller coaster.

But, winding the year down, I feel better about my role in the whole thing.

Blood sugar and behavior

I might not be the smartest person you’ll ever meet, but, given enough time, I can pick up on a pattern. The problem is, when our eleven year-old, diabetic son is moody and aggressive, it’s easy to think he’s heading towards puberty… because he is.

However, sometimes, he’s our little boy again and wants to hold my hand on walks. And I really enjoy those times.

High blood sugar days

It’s become clear that, with high blood sugar, our son is in the middle of his rebellious teenage years: he’s cruel to his siblings, he uses a tone with his mother that shouldn’t be tolerated, and he’s accused me of not loving him. Recently, he even asked what boarding school cost. (When I asked why he wanted to live at a place that he hated, he just rolled his eyes.)

We’ve had several of those days, and, sometimes, several in a row. Especially when he’s already in a mood, he’s not motivated to get his sugar down.

Is autonomy the problem?

Sometimes, if his sugar isn’t super high, we’ll let him deal with it. His mom and I can both see his current blood sugar on our phones, so we have an idea. But, beyond not letting him eat if it’s not where it should be, we let him do his own thing.

The thing is, that tended towards a downward spiral: he’s primed to be angry at us and then we don’t let him eat. Then, when his sugar is down far enough to eat, eating sends it back up before it comes back down again.

By the time it’s down, either the anger has become the day’s defining characteristic or he’s in bed.

Sigh.

Some other ideas

As I keep saying to him, I want him to be the kind of adult who can move away from home and be independent, so it seems ridiculous that I should start monitoring his blood sugar for him. After all, what will he learn from that?

But, as I’ve written, all of my incentives are to keep an eye on it, in the name of harmony at home.

So, I’ve adopted a pair of strategies. First, I point out the correlation to him. Often. I’ll tell him that I had a good day with him, and that he seemed to have a great day, and I’ll point out that his sugar was a factor. Having a good day seems like more of a motivation than some abstract long-term damage.

And, when we’re having a rough day, I’ll point that out. I try to do this in a way that says I’m not blaming him for it being high, but… I’d be the last to say I’m perfect at dealing with him in that state.

The other strategy I’ve only been able to get out once or twice has been fitness instead of insulin. Especially in the lockdown, I’ve been the family’s activity ambassador, putting on mandatory exercise videos or getting out for walks. The family has even started a couch-to-5k program that’s showing results.

With him, when I notice his sugar is high, I’ve said that it’s a good time for he and I to get out for a running workout, since he won’t have to actively load sugar. Or, another time, I just said “you’re sugar is pretty high. This is a good chance to get the rowing machine out and row it down to six.” (We measure sugar in mmol/l.)

Needless to say, most people aren’t living in a state of wanting to be surprised by a workout and it’s definitely negative reinforcement, but I try to not make it into a punishment. “It’ll give us a chance to check that off our list,” I say, or, “you told me you want to do a backflip. Remember the guy in the video who said the first thing to do is to get your body stronger? We can do that now.”

So, I try to point out that high blood sugar is an opportunity to do some unplanned fitness (and, to be fair, I spend the day doing random sets of pushups or wall stands and am always ready to join a kid in a workout video) and hope that he’ll decide to become aggressive about monitoring his own sugar, if only in the interest of fending off my fitness.

It’s an ongoing experiment

Whenever I think I’m in a parenting groove that works, some kid decides to grow in a way that throws me for a loop. I’d love to hear about how other parents are managing the tension between seeing our own reasons to keep blood sugar levels in line and wanting the kids to develop their own motivations.

And, most of all, finding ways to avoid making it seem like he’s being punished for having diabetes.

Look at me only paying €55.39 per workout!

I suppose that there’s something positive to be said of a guy who managed to get a €1,439.84 workout down to a mere fifty-five Euros.

Twenty-six rowing workouts later

That’s right, I’ve recorded twenty-six rowing workouts. I sometimes forget to write down that I’ve rowed and then I’m not sure if I recorded it or not, so that number might not be accurate.

So, that’s the first thing that’s changed: I don’t record workouts with tally-marks. I write down the date of each workout and then, if I’m not sure whether or not I wrote down yesterday’s workout, I can see if yesterday’s date is there.

A developing routine

As rowing has become less and less something I “get” to do (there was a time when I looked forward to the next day’s workout and resented the rest day in between) and instead became another thing that I have to make time for, it’s probably inevitable that a certain degree of routine has crept in.

My workouts tend to be one of the following:

  • Steady rowing. This is probably my “laziest” workout. There are days when I don’t feel up to pushing myself, but know I’ll be disappointed if I don’t get a workout in. So, I get the machine out and I try to maintain a stroke rate of 22 for thirty minutes.
  • A complete dark horse workout. There are a couple of workouts that Dark Horse Rowing has made available on YouTube. These always kick my butt and leave me feeling like I did something worthwhile. Generally, I download these videos if I plan to do the workout again, because the ads are less than fun.

I haven’t yet branched out into other rowing channels on YouTube. I like the way Shane Farmer (spelling?) of Dark Horse talks to me, I enjoy his mannerisms, and I haven’t gotten tired of anything yet.

Balancing out rowing

I don’t know if you know this, but Germany is going through another “light” lockdown and most of my work is from home. My step counts are way down, and, if I don’t run, there are a lot of days when I would never leave the apartment.

I’ve begun to get the sense that rowing and running aren’t enough. So, I’ve started doing other exercise videos (including this one, that my daughter picked to do when she was home for a quarantine–it routinely kicks my butt) and pushups and wall stands.

My son has got it into his head that he’d like to do backflips, so we might start training cartwheels. We’ll see.

Fantasy Pilgrimage: The first 329.5 km

Making a fantasy pilgrimage has been a… well, a fantasy of mine for a while. I’ve been saving my workout data on my phone since August, and now seems like an appropriate time to make my first entry.

From August through October, I’ve traced 329.5 km. I guess that’s not bad? I’m content with my level of activity, that’s for sure.

Tracked out on a map, that’s enough to get my to Berlin, Potsday, and then almost to Halberstadt, if I had been running and biking entirely in a straight line and didn’t stop to see anything in any of those places.

Just seeing it on a map is impressive to me.

With time, I’ll try and do some virtual sightseeing around the route and share here what I’ve seen.