Sudo Zest

I spent a long time today struggling with a bug in my application. I was using django-timezone-field and whenever I tried to assign ‘UTC’ to the field and save it, I received an `IntegrityError: myapp_mymodel.timezone may not be NULL` exception.

After a bit too long debugging, I realized that the timezone field uses pytz to handle the time zone info. That’s great because pytz is pretty comprehensive, but unfortunately, pytz has a crazy design decision not to make UTC inherit from the same base class as every other time zone. Therefore, django-timezone-field’s validation was causing ‘UTC’ to become NULL because it couldn’t be converted to a BaseTzInfo class.

I now have a pull request open for django-timezone-field, but that doesn’t fix the underlying issue with pytz, so just in case you’re thinking about validating your timezones by checking that they inherit from BaseTzInfo — don’t.

I recently found Blake Master’s writeup of Peter Thiel’s startup course at Stanford, and have been reading it on my cell phone every chance I get. Honestly, it’s like an incubator in a blog – really amazing content, and there’s a new one every week. Highly recommended.

Yesterday, my Acheev.it
cofounder Nicky
forwarded me a question on Quora, “What makes a good engineering culture?”

This answer from Edmond Lau is astounding. The place he describes would truly be an idyllic work environment as an engineer. I can see pieces of his description in my last few companies, but no where has been perfect yet.

Of course, at Acheev.it, I have a chance to build up the engineering culture from scratch. I can’t wait for the day when we are growing so I see what culture we can build.

For now, though, I’m a single developer. In light of that, Edmond’s answer changes a little bit. Some parts are more relevant, and some don’t apply yet. Here’s my reorganized list of good engineering culture for one-man startup tech teams:

1. Optimize for iteration speed.

   This is the same as Edmond’s general list. Before we have
   traction, it’s even more important – the name of the game is
   trying new experiments as often as possible until we find one
   that resonates with our audience.

2. Push relentlessly toward automation.

   Again, this is in the same place as on Edmond’s list.
   Automation is the only real way to have more hours in my day
   for free; it’s like magic.

3. Build a culture of learning and continuous improvement.

   Culture starts with one person, and our tech stack will never
   be as flexible as it is in the beginning. Therefore, I try take some
   time to keep making it better (always keeping in mind #1 – if technical
   learning is preventing us from iterating on the product, we’re
   doing it too much)

4. Build the right software abstractions.

   Since I’m learning and iterating as fast as possible, the
   right abstractions will let you try more variants of your ideas
   faster. And when it’s time to scale, this will help out there as well.

5. Invest in automated testing.

   For me, the value of automated testing is most powerful after
   I’ve figured out some core abstractions that you’ll reuse over
   and over. Often, however, when I’m just starting out with a
   new idea, the architecture changes too fluidly so mainting tests
   slows down the discovery process. When I go back to refactor
   something the first time because I now have a solid feel for how
   it should work, that’s when I know I should write the first tests.

6. Maintain a respectful work environment.

   Nicky doesn’t write code, but we still work together a lot -
   on designs, strategy, etc. We don’t always agree, but we listen
   and come to an agreement in the end.

7. Hire the best.

   Naturally, this doesn’t matter until we start hiring people,
   so it’s a sort of demarcation in this list from the one-person
   team above and the multi-person team below.

8. Allot 20% time.

   When it’s just one person and a product, all my time is sort
   of like 20% time – the project is unproven and I’m trying new
   techniques to make it work. However, I think we’ll start real
   20% time immediately when we start hiring people.

9. Develop a focus on high code quality with code reviews.

   This is another thing that will start with our next engineer.
   I really like code reviews for helping everyone improve together,
   so it’s something I miss right now.

10. Build shared ownership of code.

    I think that with less than 5 employees, this naturally
    happens; it’s hard to have so much complexity at that point that
    you can’t share it. Past that point, though, we’ll need to be
    thinking about this consciously. I’ve heard some good techniques
    like rotating pair programming, etc. That would be interesting to
    try, especially as it sort of builds in real-time code reviews.

I forgot to take my camera out this weekend, so here’s another picture from our trip to Guilin a few weeks ago. This one was taken out of an open-air cart as we zoomed down the road after a raft ride. I took about 30 photos, which was enough to get just one good one between the trees, electrical wires, and people on scooters.

I often want to know what directory one of my bash scripts is in so I can refer to other scripts with relative paths. When you execute a script on the command line, this is relatively straight-forward:

#!/bin/bash
SCRIPTPATH=`dirname $0`

In this example, $0 is the currently executing command (e.g. ‘~/bin/test.sh’) and dirname gives you the name of the directory that contains that file.

Unfortunately, this doesn’t work when you want to source a script instead of executing it. Sourced scripts run as if they are in your current environment, not in their own, so the $0 variable is actually the name of the program that started your shell. For example, run echo $0 on the command line:

mpdaugherty@grit ~ $ echo $0
-bash

Luckily, there is one variable that works in both cases – $BASH_SOURCE. $BASH_SOURCE is an array which lists the files that contain each of the function in the stack in the current file. It corresponds to $FUNCNAME, which is an array with the names of functions on the stack (and oddly is only populated when execution is currently in a function). That’s a little confusing, so here’s an example:

#!/bin/bash

function f1() {
  echo 'In f1'
  echo ''
  echo ${BASH_SOURCE[0]} + ${FUNCNAME[0]}
  echo ${BASH_SOURCE[1]} + ${FUNCNAME[1]}
}

echo 'in bash_source_test.sh'
echo ''
echo ${BASH_SOURCE[0]} + ${FUNCNAME[0]}
echo ''

When I execute it from the command line here’s what I get:

mpdaugherty@grit ~ $ ./bash_source_test.sh
in bash_source_test.sh

./bash_source_test.sh +

In f1

./bash_source_test.sh + f1
./bash_source_test.sh + main

When I source the file instead, here is what I see:

mpdaugherty@grit ~ $ source bash_source_test.sh
in bash_source_test.sh

bash_source_test.sh +

In f1

bash_source_test.sh + f1
bash_source_test.sh + source

This is great! We can now get the current filename regards of whether the file is being sourced or executed. From there, we can get the current directory. Here is the final script:

#!/bin/bash

# Temporarily save the old values so we can restore them after execution
SOURCE_TEMP=$SOURCE
DIR_TEMP=$DIR

SOURCE="${BASH_SOURCE[0]}"
# Go through all symlinks to find the ultimate location of the source file
while [ -h "$SOURCE" ] ; do SOURCE="$(readlink "$SOURCE")"; done
# Get an absolute path to the directory that contains this file
DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"

###
# YOUR CODE HERE
###

# Restore old values
SOURCE=$SOURCE_TEMP
DIR=$DIR_TEMP

This week I read a fascinating article from Dustin Curtis about a downside of vector-based graphics. Because the straight lines in them don’t necessarily match to a pixel border, they’re anti-aliased, just like the curves, which makes them look noticeably less clear.

This is pretty easy to fix in Photoshop – just zoom in and delete the half-pixels next to the straight lines in your image. However, it’s time consuming.

For rectangles and rounded rectangles, you can optionally have them ‘snap to pixels’

But for other shape types, you can’t.

I’m pretty surprised that there’s no built-in option yet that will get you 90% of the way there, however. That would be incredibly useful.

This tiny kitten belongs to Judy’s friend Li Hong and lives at her new teahouse.  I’m still playing with my new 50mm lens and really love the narrow depth-of-field it enables.

For Acheev.it, I’m currently building a Django application that integrates with GMail. We plan to distribute this via the Google Apps Marketplace so it will be very easy for organizations that are already using Google Apps to install.

One of the requirements of the Apps Marketplace is that apps support single-sign on (SSO). Google Apps domains use OpenId to support SSO, but with a few differences that mean standard OpenID packages don’t work directly.

Problems

First, because Google Apps users are authenticated against google.com, their claimed id (e.g. from their custom domain) doesn’t match the authenticating domain (google.com). This means their identity cannot be verified by OpenId frameworks that don’t treat Google Apps specially.

Second, most users of Google Apps don’t know what their OpenId endpoint should be. Therefore a better experience is to calculate it from their domain name. The Google Apps endpoint is always at the URL below:

https://www.google.com/accounts/o8/site-xrds?ns=2&hd={DOMAIN}

This is not only useful for improving the UX for users who come to your site directly, but also for enabling a navigation link within Google Apps that links directly to your application. For that, you need to provide a URL in your appmanifest.xml that includes the Apps domain. That URL should immediately start the authentication process, thus enabling apps users to have the experience of clicking a link and immediately being logged in to your application.

Solution

Because I’m working in Django, I had to address these problems in python. The first one involved changing python-openid to support Google Apps domains and I now have a pull request open with the project. If you’d like to use my fork immediately, though, it’s at https://github.com/mpdaugherty/python-openid-with-google-apps.

For the second issue, I decided to extend django-socialregistration to support Google Apps as a separate authentication method. The socialregistration project already provides the ability to create users and profiles based on information from a 3rd-party authentication system and supports OpenId. Because GoogleApps is just a slightly-different way of processing OpenId, this seemed like a good way to go. My branch with google apps support is at https://github.com/mpdaugherty/django-socialregistration-with-google-apps. There is no pull request yet because it depends on the open pull request to python-openid.

Moreover, because GoogleApps can provide information about a user (first name, last name, email address, language, and country), my addition to django-socialregistration automatically populates those properties on users and profiles as long as the information is in the user’s Google profile.

Example

Now, adding Google Apps SSO support to a Django project is simple. First, include my two repositories in your requirements.txt file and install them with pip:

-e git://github.com/mpdaugherty/python-openid-with-google-apps@master#egg=openid
-e git://github.com/mpdaugherty/django-socialregistration-with-google-apps@master#egg=socialregistration

$ pip install -r requirements.txt

Next, update your settings to include socialregistration and socialregistration.contrib.googleapps in INSTALLED_APPS

INSTALLED_APPS = (
    …
    'socialregistration',
    'socialregistration.contrib.googleapps',
    …
)

Add ‘socialregistration.contrib.openid.auth.OpenIDAuth’ to AUTHENTICATION_BACKENDS

AUTHENTICATION_BACKENDS = (
    …
    'socialregistration.contrib.openid.auth.OpenIDAuth',
)

Finally, add the socialregistration urls to urls.py

urlpatterns = patterns(
    …
    url(r'/social/', include('socialregistration.urls',
                                namespace = 'socialregistration')),
   …
)

After all that setup, in any of your templates, load the googleapps template tags and include a googleapps_form. This will ask the user for a domain and authenticate them against google apps.

{% load googleapps %}
<ol>
   <li>{% googleapps_form %}</li>
</ol>

To enable SSO within a marketplace app, add a navLink extension to your appmanifest.xml that goes directly to the google apps redirect page. Replace ‘yourapp.com’ with your actual application URL, but leave “${DOMAIN_NAME}” exactly as it is here. That will be replaced by Google with the domain name from which the user is logging in.

<!– Show this link in Google's universal navigation for all users –>
<Extension id="navLink" type="link">
  <Name>Your App Name</Name>
  <Url>http://yourapp.com/social/googleapps/redirect/?domain=${DOMAIN_NAME}</Url>
</Extension>

Conclusion

I will keep these repositories updated from their respective upstream repositories as long as the pull requests haven’t been merged in yet. Let me know if you find them useful or if you have any questions/suggestions in the comments below. Thanks!

Amazon EC2′s micro-sized servers are great – inexpensive and quick to set up temporarily for development.  I usually have one or two going at all times.

One of the downsides of their small size, though, is that their default storage space can fill up pretty quickly.  I ran into that this morning with one on which I had installed MongoDB.

To double check that it was a disk-space issue, I use the “df” command to see file-system usage:

ubuntu@ip-10-152-107-63 ~/ $ df
Filesystem    1K-blocks      Used Available  Use% Mounted on
/dev/xvda1    8256952    82576952         0  100% /
udev           295284           4    295280  1%   /dev
tmpfs          121252         352    120900  1%   /run
none             5120           0      5120  0%   /run/lock
none           303124           0    303124  0%   /run/shm

Now that I knew 100% of my hard drive was being used, I wanted to find out what was taking up so much space.  ”du” will give you a list of all files and their sizes in a specified directory, but if you just run it on the root folder, the list will be so long that you can’t find what you’re looking for.  Therefore, I pipe the output to “sort” and do a reverse number sort so the largest files/directories are at the top.  Finally, you can use “head” to look at only the largest 20 files:

ubuntu@ip-10-152-107-63 ~/ $ sudo du / | sort -nr | head -n 20
6434240 /
4854264 /var
2549148 /var/lib
2310172 /var/lib/mongodb
2097168 /var/lib/mongodb/journal
1512296 /var/log
1019880 /usr
403988  /usr/lib
401028  /var/cache
359780  /usr/share
358808  /home
358804  /home/ubuntu
330812  /var/cache/apt
261904  /var/cache/apt/archives
247256  /var/www
193720  /usr/bin
176408  /home/ubuntu/downloads
175112  /var/www/testsite
138420  /var/spool
138384  /var/spool/mqueue-client

From there, I could tell that my MongoDB journal folder was taking up a lot of space, which lead me to learn about the pre-allocated journal files that MongoDB creates by default and are each 1GB in size.  Since this is a test machine, though, I didn’t need so much performance, so I was able to remove one of the unused pre-allocated journals so I could keep working.

The locals refer to the morning fog in the mountains as 云海 (yùnhăi), which means “Cloud Ocean.”  Judy and I visited Longshen on Saturday with our good friends Nick and Alice.