Cerrisian Blog

I just got finished with working on the ranking stuff a bit. Writing out the CRC cards and then the specification based tests seemed to work very well. The specification got fleshed out as I was writing the test, those simple CRC  cards were all that I really needed.

After reading the Django documentation, I also learned that I’ll be putting most of the logic for getting rankings by date into a custom Manager class.

The one issue I ran into is when I had a syntax error in a file under test, spec barfed and I had to run nosetests without any options to track it down. But it wasn’t that big of a deal.

It’s also nice how you can set it to output colors for specification/tests that pass/fail or are skipped. I stubbed out a few tests that I haven’t implemented yet as skipped tests.

By outptutting the list of specifications every run, it also forces me to name my tests  correctly. I’ve refactored the tests and test classes several times to use a more accurate description.

This is an awesome way to develop. I’m definitely adding better formatting to nose spec after I finish this project.

Here’s my first try at doing some CRC cards for submitted links:

Link

Responsibilities:

• Maintain information related to submitted link
  • URL
  • title
  • submitter
  • date submitted
• Calculate score based on votes
• Keep track of whether the link has been flagged inappropriate or broken
• Keep track of whether the link has been approved (for anonymous submissions)

Collaborators

• Vote
• User

Vote

Responsibilities:

• Keep track of vote information (+1, -1, 0)
• Keep track of date vote modified
• keep track of which user voted.

Collaborators

User

Responsibilities:

• Keep track of user information and authentication.
• Keep track of user trust level.

View

Responsibilities:

• Display links in correct order based on vote, session and user.

I finally upgraded pinocchio (extensions to nose, hah!) and –with-spec is working.

Here’s the output I currently get from my tests. Now, this is really slick. It generates a “Just good enough” specification based on the automated test cases I’ve created.

Index functional tests
- The response to a GET should have a 200 status code.

Tests for submitted link model object.
- link attributes

Index view should return correct format.
- Content should not be empty.
- Content should contain a list of the top links from the past week.
- Content should have a login/create account link.
- Should return a status code of 200.
- Content should have a submit-link hyperlink.
- Content should have a tag cloud.
- Content should have links for month, day, year, all time and the words “week” for navigation.
- Anonymous users should not see a settings link.
- Should return HttpResponse object.

I’ve run into two slight niggles with it. First, it doesn’t handle methods that are in camel case, only methods that are separated by underscores. Second, it doesn’t dedent docstrings or format them at all. I’m going to get darcs going and submit patches to fix both these problems.

So, I’ve finally started working on the database and ran into the biggest problem yet with my nose setup. Specifically, django needs a test database setup before tests of the model object.

To do this, I first tried to find a modern django plugin for nose (with no luck). I then found nose fixtures, which allowed me to setup and teardown the test database for my tests package.

Here’s the __init__.py file for it:

from django.test import utils
from django.conf import settings
from django.db import connection

database_name = None

def setup():
    """Setup the various django fixtures."""
    database_name = settings.DATABASE_NAME
    utils.setup_test_environment()
    connection.creation.create_test_db()

def teardown():
    """Tear down the fixtures and database."""
    connection.creation.destroy_test_db(database_name)
    utils.teardown_test_environment()

So, starting with the previous blog post, I’ve turned the verbal description f the test cases into python code. I also implemented the necessary additions to urls.py and views.py to get them to pass.

Here are my “functional tests” These use the django testing framework, which means it’s pretty close to what a real user would experience. This file needs to be updated after I get some more functionality lower down with unit tests.

functional_tests.py

from django.test import TestCase

class IndexFunctionalTests(TestCase):
    urls = 'linkranking.urls'

    def testIndexReturns200(self):
         """The response to a get should have a 200 status code."""
         response = self.client.get('/')
         self.assertEqual(response.status_code, 200)

test_views.py
This file contains all the real test cases, currently it tests templates, urls and views.

from django.http import HttpRequest, HttpResponse

from linkranking import views

class TestIndexViewLayout():
    """Test the index view so that it returns an HttpResponse object that
    contains:
    * Submit Link hyperlink
    * A tag cloud
    * A list of the top Bayesian ranked links from the past week
    * Links labeled day, month, year, all time
    * A login/create account hyperlink
    """

    def setup(self):
        """Test the index view with an empty HTTP request object."""
        self.response = views.index(HttpRequest())

    def testShouldReturnAnHttpResponse(self):
        """View should return HttpResponse object."""
        assert isinstance(self.response, HttpResponse)

    def testShouldHaveStatusCode200(self):
        """View should return a status code of 200."""
        assert self.response.status_code == 200

    def testResponseShouldNotBeEmpty(self):
        """The response should not be empty."""
        assert self.response.content != ''

    def testShouldHaveSubmitHyperlink(self):
        """The response should have a submit hyperlink."""
        assert '<a href="/submit-link">Submit Link</a>' \
                in self.response.content

    def testShouldHaveTagCloud(self):
        """The response should have a tag cloud."""
        assert '<div id="tag-cloud">' in self.response.content

    def testShouldHaveListOfTopLinks(self):
        """The response should contain a list of the top links from the past
        week."""
        assert '<ul id="top-links">' in self.response.content

    def testShouldHaveLoginLink(self):
        """The response should have a login/create account link."""
        assert '<a href="/login">Login/Create Account</a>' \
                in self.response.content

    def testShouldNotHaveSettingsLink(self):
        """Anonymous users should not see a settings link."""
        assert 'settings' not in self.response.content

    def testShouldHaveTimeNavigationLinks(self):
        """The webpage should have links for month, day, year, all time and the
        words "week" for navigatiion."""
        assert '<a href="/today">today</a>' in self.response.content
        assert 'week' in self.response.content
        assert '<a href="/month">month</a>' in self.response.content
        assert '<a href="/year">year</a>' in self.response.content

So, I’ve generated a lot of very simple test cases to make sure that the correct pieces of the HTML output are there. I’m not sure if this is a good strategy (we’ll see in the future). Luckily these tests are high level enough that they shouldn’t change much when the underlining functionality changes.

One thing I’ve been debating is using BeautifulSoup to look up the tags in the output HTML instead of just poking at it to see if it directly contains strings. Using beautiful soup would definitely be more robust, but I don’t know if it’s necessary.

So, as part of this link ranking website project, I’m trying to do TDD In Django, I’ve got a nice development setup with virtualenv with django installed locally in my development folder. My apps are added to sys.path through the PYTHONPATH environment variable and I set the DJANGO_SETTINGS_MODULE variable depending on what I’m developing at the moment. I also used easy_install to get the latest version of nose, my favorite test runner.

So, the first thing I do is write a simple smoke test to make sure that my application is setup correctly:

from django.test import TestCase

class IndexFunctionalTests(TestCase):

    def testIndexReturns200(self):
        response = self.client.get('/')
        self.assertEqual(response.status_code, 200)

This requires the DJANGO_SETTINGS_MODULE environment variable to be set. I copied what django-tagging does, and have a tiny project settings.py specified within the tests folder. I also set up the urls.py file in the linkranking directory and reference it in tests.settings file.

This psuedo project will need to be expanded once I start developing templates. And I’m still running into problem #8358. I really should try this patch to see if it fixes it.

So far using nose with Django is pretty straightforward, if I use the Django TestCase object for all the database and client related stuff.

Here’s the first few functional tests that I’ve based off of the usecases for the new link aggregation website.

Index Page

When an anonymous user visits the main page, the user should see:

• A Submit Link hyperlink.
• A tag cloud.
• A list of the top links from the past week. Each item in the list should have:
  • A hyperlinked title
  • A short description
  • A link to the user profile that submitted the link.
• Links labeled month, day, year, all time. and a ghosted link labeled week in the upper corner of the list of links.
• A login/create account link

When a known user visits the main page, the user should see:

• Everything an anonymous user sees except that the links day, week, month, year, all time should be ghosted based on which date range the user had selected previously
• A settings link.

This is a continuation of this post and this post, showing how the examples gain more specifics as time goes on.

Use Cases

Submitting a Link

Bobby finds a neat link about plate tectonics. he goes to the website “tectonic-plate-o-matics.com” to enter the link to the community. He sees the current links ranked by rating. In a corner, he sees “Submit link”. He clicks it and and is given a form that has the following inputs:

• Link URL
• Link title
• Link Description
• Tags

He then submits the link and sees it show up on the front page. Note: Since Bobby isn’t logged in when he submits the link, it doesn’t show up for other anonymous users. Just the submitter and any users that have the “See anonymous links” set to true. For Bobby, It is also artificially floated to the top of the link list regardless of its current ranking, since Bobby submitted it.

Visiting A Link

Bobby now clicks on one of the links on the front page. He is taken, in a unobtrusive frame, to the link. The frame has a few buttons or inputs on it, including:

• rank +1
• rank -1
• add tags
• report link

When he clicks on any of these, he’s taken to a login page, the login page supports either open id or creating a user account on the web page itself. After he’s logged in, he is returned to the page he was on. It also contains a list of current tags for the link.

It also has a prominent “close X” button which cleanly closes the frame.

Ranking Links

Sally is an active member of the link ranking site. She goes there multiple times a day to check out new links about plate tectonics and moderate them, so she always has her login cookie set. When she goes to the homepage, she is presented the top links for the day (since this is the setting she has selected last time, the default is for one week).

She clicks on the link she finds most interesting, “Daffy Ducks view of plate tectonics”, and visits it. She then uses the upper frame to moderate the link with a score of +1, she hits back and is returned to the homepage, the link she moderated is closer to the top of the list because of her moderation.

Note: There needs to be some sort of trust metric (advogato style?) for reputation of moderators, and strength of their moderations. The interface must also be designed so it isn’t like reddit.com, it’s not a news site, just an index.

Reporting Dead Links

Sally then clicks on another link “Tectonic Plates of the ancient world” which brings her to a 404 page, she then clicks the “report bad link” button on the top frame and selects “page not there anymore” from the drop-down menu. When she clicks submit, a status message is displayed in the frame saying something like “Thank you for your attention to detail!”

Note: Other options could be “inappropriate link”, “link moved”, etc. We also need to handle tagging/incorrect tagging somehow…

Main Page

The main page has three main sections:

• Header, including site logo, title, login, and settings link
• Sidebar consisting of a Tag Cloud and other navigation links
• Links, which is a list of links that have been rated in the past week with the highest Bayesian based ranking. Each link has a title and a short description and a link to the profile of the user who submitted it.
  • This also has links to display the links by rank from the last day, month, year and all time.
  • It also shows which, if any, tag is selected for the links.

Time for more usecases! (continued from this post)

Ranking Links

Sally is an active member of the link ranking site. She goes there multiple times a day to check out new links about plate tectonics and moderate them, so she always has her login cookie set. When she goes to the homepage, she is presented the top links for the day.There is also a tag cloud that has the most popular categories.

She clicks on  the most interesting link, “Daffy Ducks view of plate tectonics”, and visits it. She then uses the upper frame to moderate the link with a score of +1, she hits back and is returned to the homepage, the link she moderated is closer to the top of the list because of her moderation.

Note: There needs to be some sort of trust metric (advagato style?) for moderation of links. The interface must also be designed so it isn’t like reddit.com, it’s not a news site, just an index.

Reporting Dead Links

Sally then clicks on another link “Tectonic Plates of the ancient world” which brings her to a 404 page, shen then clicks the “report bad link” button on the top frame and selects “page not there anymore” from the dropdown menu.

Alrighty, time for the spec writing to start!

drumroll

So, the basic goal is to create a link-repository application where people can submit links on a topic and they can be ranked, verified and deprecated as needed. E.g.

Submitting a Link

Bobby finds a neat link about plate tectonics. he goes to the website “tectonic-plate-o-matics.com” to enter the link to the community. He sees the current links ranked by rating. In a corner, he sees “Submit link”. He clicks and and is given a form that has the following inputs:

• Link URL
• Link title
• Tags

He then submits the link and sees it show up on the front page. Note: Since Bobby isn’t logged in when he submits the link, it doesn’t show up for other anonymous users. Just the submitter and any users that have the “See anonymous links” set to true.

Ranking Links

Bobby now clicks on one of the links on the front page. He is taken, in a unobtrusive frame, to the link. The frame has a few buttons or inputs on it, including:

• rank +1
• rank -1
• add tags
• report link

When he clicks on any of these, he’s taken to a login page, the login page supports either open id or creating a user account on the web page itself. After he’s logged in, he is returned to the page he was on. It also contains a list of current tags for the link.

It also has a prominent “close X” button which cleanly closes the frame.