Applications Coding Tutorials

The missing piece in my Python scripts

My quest to automate parts of my workflows usually involves writing Python scripts to streamline some of my tasks as a Happiness Engineer.

One output has confused me for far longer than it should have, and I was shown the light today (in the form of a pretty simple solution).

My dilemma

Many of my scripts generate template documents. These include meeting agendas, hangout notes, and team reports. These documents include a couple components that I’ve largely automated using Python, such as –

  • Calculating dates and times in reports (sometimes including timezone conversions);
  • Taking inputs I provide in response to command line prompts, running calculations, and adding those to fields in my reports; and so on.

Many of my documents have a static list of team members that I ping much like Twitter mentions when I publish them as posts on internal blogs (we use internal blogs quite a lot, as you can imagine). Those lists each need to be updated manually every time there’s a change in the team composition, in each of the documents the list appears in.

Although the team composition doesn’t change frequently, I often find myself re-using a script for a team report, or hangout agenda, for example, in another group I work with. This means another set of documents where I need to manually create a list of people, and maintain that.

An example could be something like this:

Pinging: @john-doe, @maryjacobs, @davesmith, @janestuart, @tomwright, @steverobinson

The solution

Currently, the first part of generating this sort of list of people is creating a .csv file that looks something like this:


I was planning on using Python Classes to do this, but quickly realised that I don’t understand Classes nearly well enough to using this feature for this aspect of my documents.

I realised that there’s actually a simpler solution using .csv files and the csv module available in Python instead, after watching Corey Schafer‘s tutorial on this, here:

As an aside, Schafer’s tutorials are wonderful!

I was able to borrow from Schafer’s solutions to write a script that produced a list that looks something like this:

Pinging: ['@john-doe', '@maryjacobs', '@davesmith', '@janestuart', '@tomwright', '@steverobinson']

My script looked like this:

import csv

with open('people.csv', 'r') as csv_file:
    csv_reader = csv.DictReader(csv_file)

    members = []

    for line in csv_reader:
        wpuser = f"@{line['wp_user']}"

print(f'Pinging: {members}')

I couldn’t work out how to just generate a list of usernames separated by spaces. I ran into a similar issue with other scripts where I similarly loop over a list of items to produce some sort of list-generated output.

In other words, I couldn’t figure out how to output the list of usernames without the []' characters you see in my initial output.

So, I asked for help on reddit where JohnnyJordaan pointed me to this StackOverflow post that includes a couple solutions to a similar question. I had searched for a solution to my challenge, but didn’t come across this answer (or anything like it).

Clearly, I need to work on my Googling skills when it comes to finding solutions to my coding challenges.

Anyway, the solution that JohnnyJordaan suggested is pretty simple. Instead of using something like print(f'Pinging: {members}'), I could rather join the items in the list with a statement like this:

print(f'Pinging: {", ".join(members)}')

As I suspected, the solution is pretty simple. It just eluded me, completely. So thank you to JohnnyJordaan for the pointer!

I also like the * that came up in one of the StackOverflow solutions, but it doesn’t work with the f strings that I tend to use. Still, there are other ways to use them. Trey Hunner has an interesting post about these operators here (if you’re interested).

Business and work Mindsets

What? Me? Procrastinate?

Oh boy, this explains so much about me and why I probably tend to procrastinate so much …

I’ve become better at noticing when I procrastinate and developing tactics to help me manage my procrastination habit better. Much of what I do tends to focus on a highly structured and granular productivity system, specifically my imperfect implementation of Getting Things Done.

GTD has become my lifeline to a much more productive workflow and I guard it from other people’s attempts to disrupt it fairly actively. As it is, I often find I need to somehow harmonise my GTD implementation with whatever my colleagues are using so it can become a little complex. That said, it remains the bedrock of how I (mostly) get stuff done.

I can’t say I have conquered procrastination but I think the trick is to reign it in so it is more manageable. I try be more patient with myself when I do drift off. There is no point being hard on yourself, it just reinforces the habit.

Anyway, where was I … ?

Image credit: ejaugsburg (via Pixabay)

Applications Semantic Web

IFTTT v Pinboard

Never mind Batman v Superman. Now it’s IFTTT v Pinboard and I’m caught in the cross-fire.

I use Pinboard daily in some form or another. I also use the great “connector” service, IFTTT, daily to automate a host of little tasks like adding Instapaper highlights to a text file in Dropbox and many more.

In particular, I use a number of IFTTT recipes that include Pinboard in various little workflows that make my life easier and now it seems that is going to come to an end in just a week or two. I received this alarming email from IFTTT:

I rushed over to the Pinboard blog to see what Pinboard’s creator, Maciej Cegłowski, has to say about the matter. It turns out, he has quite a bit to say in his blog post titled “My Heroic and Lazy Stand Against IFTTT”. He cited two reasons for this little impasse:

Because many of you rely on IFTTT, and because this email makes it sound like I’m the asshole, I feel I should explain myself.

In a nutshell:

  1. IFTTT wants me to do their job for them for free
  2. They have really squirrely terms of service

It’s entirely IFTTT’s decision to drop support for Pinboard (along with a bunch of other sites). They are the ones who are going to flip the switch on working code on April 4, and they could just as easily flip the switch back on (or even write an IFTTT recipe that does it for them). Weigh their claims about Pinboard being a beloved service accordingly.

I understand his concerns and I agree with him that he shouldn’t have to reinvent the wheel at his cost to satisfy IFTTT’s requirements to remain connected to the service. Surely IFTTT could have come up with a more developer-friendly way to migrate to their new platform and help developers make the transition at a lower cost?

I would also have reservations about the contract they want developers to agree to as part of their transition to the new platform. Requiring developers to agree to the sorts of terms Cegłowski quote seems pretty unreasonable given what the clauses would seem to be saying.

At the same time, I’m caught between these two providers I rely on for various tasks. I don’t like the approach IFTTT seems to be taking but I love the service they provide. It literally makes my life easier in so many ways. I have also been using Pinboard for a while now as my personal bookmarking service and I even pay for the archival service. It is a simple and effective service.

This IFTTT v Pinboard impasse between the two companies just hurts people like me who either have to switch to some IFTTT competitor to address the soon-to-be introduced gaps in my workflows or just abandon those workflows altogether. The effect of that is to diminish the value of both services for me, just enough to leave me with that sick, disappointed feeling you get in times like these.

I hope this situation is resolved in some form or another but, if it isn’t … well, that just sucks.


IFTTT v Pinboard Redux – Contracts and Condescension

Mindsets Politics and government Travel and places

A good experience at Home Affairs is possible

Update (2014-10-13): I received some feedback on how to expedite applications you submit to Home Affairs so read through to the end of the post for those tips.

We’ve had to apply for birth certificates and other documents from the Department of Home Affairs for a little travel we have coming up so I’ve spent many hours at Home Affairs offices lately. I started off at the Edenvale Home Affairs office because I thought that it would be more efficient than other offices we’ve used in the past but that turned out to be wishful thinking. A big reason for the hours you spend at this and similarly run Home Affairs offices is this.

The problem

Home Affairs runs on paper almost exclusively. There are forms to complete and submit in person and long delays before you receive what you request. I had to apply for unabridged birth certificates for our kids (the ones we received after they were born were abridged) and it takes 6 to 8 weeks (hypothetically) for the certificates to be delivered to the Home Affairs office you applied from.

It boggles my mind that I have to apply in person at a Home Affairs office for these sorts of documents and that it takes so long for what is basically a printout on fancy stationary. Why couldn’t each Home Affairs office have a printer installed with the requisite paper to print these certificates in a secure environment? The answer is probably because decentralising this would only lead to dramatically more fraud than is currently possible through the centralised model.

What would make more sense is to enable citizens to log into a secure site using a username, password and secured by an identity number and something like a one-time PIN to order these sorts of documents. Collections can still take place in person using an ID book or card to authenticate the requester but this would probably make huge inroads into cutting back on the ridiculous queues just to order those printouts.

For now

In the meantime, we discovered that not all Home Affairs offices are created equal. We received advice to head to the Centurion Home Affairs office to order documents because it apparently takes far less time to receive the documents you order from there. That sounds a bit extreme but it turns out this Home Affairs office is across the road from the Centurion Gautrain Station so the commute is actually pretty quick.

Another tip I received is to call the Home Affairs call centre and ask them to expedite applications where you need the documents fairly urgently. I’m going to do that tomorrow and that will hopefully speed the process up.

Update (2014-10-13): I just called the Home Affairs call centre in an effort to speed up the process and I was told that it isn’t that simple. Here is how you expedite document requests:

  1. Ideally, prepare a motivational letter (1 for each document applied for and written by the particular applicant) stating why you need the documents urgently and asking for the document requests to be expedited (make it convincing).
  2. You can submit motivational letters if you have already already submitted the applications but this probably isn’t ideal. If you do this, you should probably include copies of your receipts so they can link the requests back to the motivational letters.
  3. Hand deliver the motivational letters to the Home Affairs office you submitted your applications to. No faxes or emails.
  4. When you hand in your letters, take an extra copy of the letter with the attachments and have the person at the counter stamp the copy of the letter so you have some proof that they received it.
  5. Wait a week and then call the call centre on 0800 60 11 90 to check on the progress of your motivation (in other words, whether they feel it is sufficiently compelling and what the outcome is).
  6. Keep your reference numbers for your calls (they didn’t advise me to do this but do it anyway).

Pretty productive morning at the office

I still have a couple hours of work ahead of me but much of that is a new, more developed project management workflow for my businesses.