Adding Mailchimp actions to a Django model

As I get ready to try a second time to attract users, I decided to use MailChimp mailing lists to keep in touch with these users, as well as setting up some ‘tutorial’ emails at regular intervals. (I’m imagining something like “You’ve been using for a week now, did you know you can do this to get more out of it?”)

To do this, I’d have to have code in Django that automatically signed members up. That means that this is my first attempt to work with an API, even by way of a library.

I decided to go with the mailchimp3 library and installed it in my virtualenv according to the instructions and set up some variables in my file.

# Mailchimp stuff
# Authentication

# The lists
MAILCHIMP_LISTS = {'welcome': 'listID',
                   'germany': 'listID',
                   'beta-testers': 'listID'}<span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>

The idea here is that I’ll be able to iterate over the keys in the lists, if I ever need to check that the user is in all the lists (for example, removing a user should include removing him from all lists).

To that end, I added a function at the beginning of my file. (This probably isn’t good form).

def mailchimp_get():
    """ Returns a logged-in mailchimp client """
    return MailChimp(settings.MAILCHIMP_LOGIN, settings.MAILCHIMP_KEY)<span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>

It seemed like the easiest way to only write the login once.

Finally, I had to create a few methods in the Model in question (in my case, this is my teacher model, as my users are called ‘Teachers’ in this bit.

def mailchimp_add_teacher_to_list(self, listname):
    """ Takes a listname, checks that the teacher is not a member of the list, then adds it. """
    client = mailchimp_get()
    listid = settings.MAILCHIMP_LISTS[listname]
    if not self.mailchimp_teacher_in_list(listname):
        data = {
            'status': 'subscribed',
            'merge_fields': {
                'FNAME': self.user.first_name
        client.lists.members.create(listid, data)

def mailchimp_lists_get(self):
    """ TEMPORARY - Returns a list of all mailchimp lists """
    client = mailchimp_get()
    return client.lists.all(get_all=True, fields=",")

def mailchimp_teacher_in_list(self, listname):
    """ Returns a bool indicating if this teacher's email address is in the list """
    email =
    client = mailchimp_get()
    results = client.lists.members.all(settings.MAILCHIMP_LISTS[listname], get_all=True, fields="members.email_address")
    members = results['members']
    addresses = []
    for m in members:

    return email in addresses

I think the names should be pretty self-explanatory. The mailchimp_lists_get() method was just an experiment based on following the instructions in the documents.

Lastly, I created a view and a URL just to test these functions by printing to the console and looking at the (currently empty) lists in mailchimp.


It was my first time sitting down and thinking critically about what I wanted and how to do it and I have to say that mailchimp and the library basically made it painless. I have other project ideas that will require more API integration, and it’s made me hopeful.

Can’t say I’m not learning

I continue to be frustrated by whatever it is that is interfering with my rented Ubuntu server serving up my worksheet-generating django site. I’ve even come so far that I’ve written up my troubles in a Stackoverflow question.

While I’m decompressing, I thought I’d summarize some of what I’ve been learning. A lot of what I did felt like trying things wildly (Is this the right path? Maybe up a level?) and restarting services. Then, I took an answer to this question on Stackoverflow to heart:

You should try to invest a bit of time in understanding all the components involved. For example you have merged the .ini uWSGI file with nginx.conf that is completely wrong. I can suggest you to start from here:

Try to understand every step (expecially the part about using official sources instead of distro packages). Start deploying without nginx (only uWSGI), and only after you are sure the thing is clear, you can proxy it behind nginx.

I went through the steps in the quickstart tutorial one at a time, writing a short Python program to do ‘hello world,’ and moving forward from there. It was helpful, and I have learned.

For example, I guess I never really thought about it before, but I think that uwsgi is the part of the whole thing that actually runs my code. I’ve certainly seen something like this often enough:

web browser <=> web server <=> wsgi <=> django code

And, to be honest, I never really thought about what ran my code. When you run the development server, it feels clear.

It looks like, whatever I’m doing wrong, I have the uwsgi service incorrectly configured, such that it can’t load my code. That’s obviously a problem. I just don’t know how to fix it (and I’m not even sure that I know it, this is all a question of me reading error logs, googling, reading answers to questions that are only similar to mine, and trying to extrapolate).

Still, in addition to beginning to wrap my head around wsgi, here are some things I’ve learned:

  • A few new console commands in linux, including ‘env,’ ‘useradd,’ and ‘nano’
  • Where some error files are stored ( /var/log/nginx/error.log — I can type that from memory)
  • How to use ‘sudo’ and ‘su’
  • Starting, stopping, and restarting services

There are some things I’d like to know more about, but my brain feels too melted to really absorb new information. They include:

  • How to uwsgi and nginx communicate via sockets? What does that mean? What is a socket file?
  • I only did the bare minimum of ‘hardening’ my server. How vulnerable is it? What are the chances that it will be attacked deliberately, or by someone just attacking every website in the hopes that one is vulnerable?
  • I’ve realized that all of the http://localhost:8000/…/ in my Django templates will need to be updated to use the domain name I bought. How do grown-up developers deal with this? How will I test things here ‘in development’ before sending it to ‘production.’ (I feel like a poser using those words).

So, yeah, I’m frustrated, but not so much that I’ve opened the red wine. (See my last post.) If I had a bottle open, however, I’d raise a glass to learning.

How XML Works

So, this will obviously not be an explanation of how XML works. Instead, this will be a short explanation of “what I think about when I think about XML.” And, I want to begin by saying that I am happy that this page here exists, as that’s where I went to double-check my vocabulary and, in the process, learned quite a bit.

XML basically is a standard by which your computer (or a Python library, to be specific) knows how to put information into text format to save to a file. If you’ve ever done much by way of HTML encoding, the basic formats will look familiar.

How to write XML.

Remember in the last bit where I said that I think of XML data as a pile of folders, some containing other folders? Well, basically, each folder is an Element. It has this format:


They both are elements named ‘ELEMENT’ (I don’t know why I do so much in all caps in my own XML, I guess it’s so that I know I’m writing for a computer and not a person.)

Of the two, the first is a self-closing tag. (That’s just the way of saying that no second tag is required to ‘close it’, like in the second example.

In the second example, the first tag ‘opens’ the element, the second ‘closes’ it.

The reason for opening and closing the element is pretty simple. Remember my example of a bundle of nested folders? Well, if we have an element meant to represent a user, it might also contain things like grammar this person has worked on, or a list of vocabulary that they’re using.

To include another element in an element, you open the element, create the other elements, and then close the element again. Like this:

<USER NAME=”Toby the Amazing”>

<VOCAB_LIST WORDS=”XML, pizza, whining” />


<GRAMMAR NAME=”Simple Past” />

<GRAMMAR NAME=”Simple Present” />



(I’m sorry for the formatting, but I’m not willing to do all the weird stuff required to make it look nice with WordPress.)

So, if you look at what I have above, there is a user, and inside the user element (the first and last lines open and close the element) there is an element containing a vocabulary list, as well as another element for practicied grammar which contains more elements, each for a specific grammar that has been practiced.

If you look at the different elements in my example, within the tags, or within the opening tags, you’ll see where I wrote NAME=”Toby” or WORDS=”XML…”. These are attributes in the element.

Going back to my paper metaphor, I think of opening and closing tags as the folder that contains something, and self-closing tags as individual pieces of paper in that folder. So far, so good. The thing is, the attributes are what you’ve written on each folder/paper. So, in the folder you might have a hundred copies of a form that gives information on a grammar practiced (I could not name a hundred forms of grammar, that’s a bad example.)

But, just as it makes no sense to have a folder on which you have “Name: Toby” and “Name: Liam” written (which name is it?!) you can’t have two attributes with the same name in XML. I don’t know what happens when you manually write XML that way, but in the python library I’m going to write about next, it would just change the name.

Remember this: If you want many of something — grammar forms, words, users — what you want are elements. If you want to define your elements with unique properties, name, height, level of charm, then you want attributes.

Don’t worry, you won’t have to write XML

I went through all of that, and you’ll probably never write XML in your life (though I do write the initial tags in the XML files that I use, mostly because I’m too lazy to do it with code.)

In the next little bit that I write, I’m going to explain how I do XML in Python, and you’ll need to know the words element and attribute, but you’ll never have to worry about how to write it. After all, the goal is to have your computer do the work, not you.

Still, one of the great things about XML is that you can manually edit it, and I think it’s good to know how it’s put together.

This is something I’m writing mostly for my nephew in support of the Typing Tutor challenge that I’ve issued to him. I imagine all of this information has been explained better elsewhere on the Internet.

So why are you reading this?

Why I like XML

So, I haven’t been programming long enough — and certainly not well enough — to be allowed an opinion on what other people should do. But, I do think I can share why I recommend using XML to other new programmers.

It’s fair to point out that I’m talking about programming in Python with the Element Tree library.

It behaves like you’d think.

I didn’t fall in love with programming until I discovered object-oriented programming. It’s just easier for me to abstract out the idea of the different parts of a program as ‘objects’ that interact. (I can write a bit on object-oriented programming, if you’d like, too!)

A lot of my programs are just collections of objects. I have a dictionary object that manages so many word objects, for example, in my ESL worksheet application. And, when I need to save all this as a file, it’s nice to have a format that makes sense.

The way that it makes sense to organize all this data, to me, in the physical world, would be a great big folder or binder with lots of other folders and binders in it. But also, with thousands of loose pages.

There would be a binder that is labeled “Dictionary” and that binder would probably have a name and, because I record dates unnecessarily, the date when it was created. Inside, each word would be a folder that contained information on the word, its German translation, space, maybe for other languages, as well as things like what part of speech it is and if there are other words that have the same text. (A real example from my teaching: I teach in one company where ‘forks’ normally mean the parts of your bicycle that hold the front wheel, but I also teach a lot of people who also think that a fork is something to eat with. Obviously, it’s going to make a difference in the worksheets.)

And these word entries should have a unique label of some kind (so the different kinds of ‘forks’ can be referenced later) and, of the other binder full of sentences contains a good example sentence for this word, I might want to store the unique label of that sentence.

That’s all the data I have in this binder, right? (In your typing trainer, the binder might be for a single user — with records on the letters they’re learning, another with a list of keys they often miss…) And what’s great about XML, is that it gives me a way to save data in this format, if I only think about it the right way.

Even more than just saving it in this way, the Element Tree library gives me objects (they’re called Elements and SubElements, when I write about using Element Tree I’ll write more about it.) that behave this way. That means I can assign an entire Dictionary to a single variable and pass it to a function or a method.

In fact, a lot of my programming is basically writing objects that take a single XML object and ‘bring it to life’ by adding methods to work with the data in it.

You can read it.

Here’s the other thing about XML: I can use it to store data and then, when I realize I’ve made a mistake (there’s a typo in a word, or students say a definition I wrote is unclear), I don’t need to write up a bunch of code to access that data and change it — though I should and eventually will — but instead I can open the file in any editor and edit the file.

I don’t think it’s possible for me to overstate how thankful I am that I can read and directly edit the information I have stored in XML. Even more than being convenient, it’s reassuring to me that I can fix my mistakes without too much effort.

This is the first in a small series I’m doing on XML for my nephew who is also a new coder. He recently wrote me that he’d rather just set up a text file for the Typing Challenge I set him. Of course, he can do what he wants, but I think it’s the prerogative of an uncle to send long, rambling dissertations on his opinions.

Colors in Reportlab

Looking for an exhaustive list of the colors in ReportLab, I was surprised that there wasn’t a list of them — with examples — in the ReportLab docs. Then, when I didn’t quickly find one with Google, I thought I would turn this into a test of my coding chops.

So, this might seem simple to you, but I needed quite a few tries to get it going. Which means, of course, that I’m pretty proud of it.

Here’s the code I wrote:

from reportlab.lib.pagesizes import A4
from reportlab.lib.enums import TA_LEFT
from reportlab.platypus import SimpleDocTemplate, Paragraph
from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle
from reportlab.lib.colors import getAllNamedColors

doc = []
document = SimpleDocTemplate('colors.pdf', pagesize = A4, rightMargin = 72, leftMargin = 72, topMargin = 72,
                                     bottomMargin = 34)
styles = getSampleStyleSheet()

for color in getAllNamedColors():
    styles.add(ParagraphStyle(name=color, alignment=TA_LEFT, fontSize=12, leading=22, fontName='Helvetica',
    doc.append(Paragraph('Test color '+color, styles[color]))

And here is the result of the code: colors.pdf

A quick and dirty grammar check in NLTK

So, while working on my app to generate worksheets (update: I’m using it! Like, in practice!) I’ve wanted a way to identify the grammar in a sentence. A quick Google search on doing this in NLTK came up mostly with diagramming sentences and looked more complicated than I am ready to tackle. (Though I will learn!)

So, with a bit of poking around, I found a method that I liked. It works with NLTK’s pos_tag function. And that, in turn, works like this:

>>> import nltk
>>> t = "This is a test."
>>> tokens = nltk.word_tokenize(t)
>>> nltk.pos_tag(tokens)
[('This', 'DT'), ('is', 'VBZ'), ('a', 'DT'), ('test', 'NN'), ('.', '.')]

NLTK actually includes a help interface to help with the ‘DT’ and ‘VBZ’ tags. It didn’t take me long to put together a few ‘patterns’ that basically represented the grammars I was looking for:

self.grammar_lookup = {
            'SIMPLE_PAST': ['VBD', "VBD-did/RB-n't/VB", "VBD-did/VB", "VBD-did/RB-not/VB"],
            'SIMPLE_PRESENT': ['VBZ', 'VBP', "VBP-do/RB-n't/VB", "VBP-do/RB-not/VB", "VBZ-does/RB-n't/VB",
            'SIMPLE_FUTURE': ['MD-will/VB', "MD-wo/RB/VB", "MD-will/RB/VB"],
            'COMPARISON_AS': ["RB-not/RB-as/JJ/IN-as", "RB-n't/RB-as/JJ/IN-as", 'RB-as/JJ/IN-as',
                              "RB-not/RB-as/RB/IN-as", "RB-n't/RB-as/RB/IN-as", 'RB-as/RB/IN-as'],
            'COMPARISON_THAN': ["JJR/IN-than", "RBR/IN-than"]

As an explanatory note, the ones with a hyphen in them represent the part of speech tag and the specific word. I did that because I want to reduce the chance of false positives.

From there, it was a fairly straightforward process to start iterating over sentences and looking for these patterns. I’m not especially proud of the code that does this and I think that any serious coder could do it more elegantly, so I’m not going to post that. Still, I’m proud of myself for finding a solution to my problem that works…

…for now. (As you can see, I’ve only tested it on five grammars. I’ll add more as I need them.)

Numbered Lists (sort of) in Reportlab

So, for the first time, I think I may have stumbled upon something I wanted to do in code for which Google searches didn’t give me a quick and easy answer. (When it’s not quick, it generally because I’m slow to understand.)

So, since I worked out a solution for myself and am more than usually proud of it, here’s what I have.

The challenge:
I’m automatically generating worksheets in PDF format. For them, I’d like to have exercises in a nice numbered list. You know the format, it’s pretty easy to do in most software.

  1. This is an exercise.
  2. So is this.
  3. And here is a blank to fill ____________.

However, even though Reportlab includes numbering in the bullets, I spent most of yesterday evening trying to tweak different indentation settings to get it to do what I want. And it didn’t. (Again, maybe I’m slow to understand.)

The problem was that either the text didn’t wrap right (it should wrap to be beneath the beginning of the text, not to beneath the number) or I couldn’t get the spacing between the number and the content to look right.

My solution:
I wound up using a table for this. It’s not what I wanted and I feel like the way I have it is a bit of a hack, with different methods checking the length of the list that feeds the table, so that I can add lines via a method an arbitrary number of times. Nonetheless, I’m pretty happy with how the result looks.

from reportlab.lib.pagesizes import A4
from reportlab.lib.enums import TA_LEFT
from reportlab.platypus import SimpleDocTemplate, Paragraph, Table, TableStyle
from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle, ListStyle
from reportlab.lib.units import cm

# We need some text to use. I happen to like this one
lines=['Sing, Goddess, sing of the rage of Achilles, son of Peleus--',
       'that murderous anger which condemned Achaeans',
       'to countless agonies and threw many warrior souls',
       'deep into Hades, leaving their dead bodies',
       'carrion food for dogs and birds--',
       'all in fulfilment of the will of Zeus.']

style=ParagraphStyle(name='Normal', alignment=TA_LEFT, fontSize=12, leading=18, fontName='Helvetica')
doc=SimpleDocTemplate('iliad.pdf', pageSize=A4)

# Add a title, straightforward enough.
document.append(Paragraph('&lt;b&gt;The Iliad&lt;/b&gt; by Homer', style))

# Now, add numbered lines
for line in lines:
    epic.append([Paragraph('&lt;b&gt;'+str(len(epic)+1)+'.&lt;/b&gt;', style),
                 Paragraph(line, style)])

t=Table(epic, colWidths=(1.2*cm, None))
    ('VALIGN', (0,0), (-1,-1), 'TOP')


That gives you this result: iliad.pdf.

About it
Obviously, it’s a table which is made up of a list of lists. The various ‘cells’ (entries in one list) need to be wrapped in Paragraph objects in order to get the wrapping and formatting right. (It seems the <b>bold</b> tags are processed by the Paragraph object, if you’re curious.)

As for the TableStyle, that sets the vertical alignment to ‘TOP’ as it defaults to middle, which looks odd when the line it’s numbering wraps.

Lastly, it was interesting — and helpful — to me that the table cells could be numbered with the negative notation that is familiar from things like lists and strings. That is to say that (0,0) is the top left and (-1, -1) is the bottom right, because it counts from the end of the list. That was nice. (I was ready to start doing janky things with len(list) to get it right.)

Maybe I overestimate the problem I solved…
I would be surprised if I’m the only one who has done this, but it is the best I could come up with. I’d be happy to know if I overlooked some obvious solution / reference. How have you (or others) solved the same problem?

The Right Way isn’t Easy


I don’t know how often I’ve come across this comic. But, stumbling about the internet teaching myself to code in what is probably the worst was possible (figuring out how to fix one problem at a time) I’ve come across it several times. I believe it’s from here.

Not long ago, I was writing to my nephew about how I wrote a text-based game for him. In it, I was honest about the fact that I did things the ‘wrong’ way, and that I was certain the ‘right’ way was easier.

Now I know: it’s certainly different, but it might not be easier. At least, not at first.

Recently, I’ve been inspired to re-visit a project to automate a part of my work (whenever I find myself thinking “this is monkey-work!” I begin developing an algorithm by which it could be done — I’ve never successfully done it) I thought that, since I was starting over from scratch, this would be the right time to do start learning to do it right.

You know from the title where this is heading: it’s not easy.

I’ve generated so many pages of text, describing how I want using the program to be, how the XML it uses should be structured. I’ve followed the advice of a programmer I know and, where possible, did the GUI first and the ‘guts’ second.

And it’s exhausted me. So far, I’m not super far into the project, and I have a much better idea of what I’m doing and where it’s going… But it’s not easy.

I made myself hold off on any coding until I had a pretty solid skeleton sketched out in prose. Then, once I started coding, what I think I should have done is only coded as much as felt needed to keep fleshing the skeleton in, not the other way around. It’s pretty typical for me to — as I work on code to add tags to XML elements — realize that I’ll need to organize things in a different way but not to update the planning documents.

It’s reached the point now where, instead of having all the planning done and being able to ‘mindlessly’ code as another friend described his approach to coding, I’ll need to find the energy and concetration necessary to get the planning documents back in harmony with what’s actually happening in the code.

Of course, this is my first rodeo, so to speak, and I wanted to begin trying this in oder to get the learning curve behind me. Nonetheless, the learning curve does not feel good.