Corporate Hackathons: The Fine Line Between Engaging and Exploiting

Corporate Hackathons: The Fine Line Between Engaging and Exploiting

Great write-up from a friend on the Campbell Soup Hackathon.

Git Email Notifications on Push

So I’m doing a private collab and hosting a git repo off my server. Got annoying pretty quick with these occasional emails “hey pushed xyz change that could affect abc for you, make sure to pull the latest”

Enter this script, post-receive-email

To get going with it, I did the following, (line breaks for readability here):

wget -O '/user/local/bin/post-receive-email' \
'http://git.kernel.org/?p=git/git.git;a=blob_plain;f=contrib/hooks/'\
'post-receive-email;h=60cbab65d3f8230be3041a13fac2fd9f9b3018d5;hb=HEAD'

chmod a+x /usr/local/bin/post-receive-email

Next, link to this in your repo with ln -s /usr/local/bin/post-receive-email hooks/post-receive and add something like this to config

[hooks]
mailinglist = "jane@email.com, john@email.com"
envelopesender = no-reply@email.com
emailprefix = "[GIT] "

And that will notify the mailing list of any push that occurs. There’s a number of other options worth exploring by reading the post-receive-email bash script.

Sublime Text 2 and multiple cursors

Two days ago I started using sublime text 2 for projects. I’m coming from a long sprint of VIM usage and I have to say that sublime is pretty awesome. In fact, I’d describe sublime as VIM + awesome. Unfortunately sublime isn’t free and not to be a glory taker, but I hope to produce something on par for free in the future.

One thing that really wow’d me into at least trying sublime was the proposition of multiple cursors. With that being said, this wouldn’t be my first foray into the lust for nonsensical features. But, only after a day of usage I can honestly say multiple cursors in a text editor is just a plain win.

Primarily, I’ve used VIM in the past for all but the most heavily-dependent-on-IDE tasks (namely, java). I love VIM. Just the other day I opened a 6.4GB mysql dump to make some minor changes by hand before passing it on to an AWK script for conversion to postgres (yeay 16GB of system memory), but sublime is, well, simply sublime.

Refactoring tools in an IDE typically consist of being scope aware and allowing you to rename method variables and class members. One thing no refactoring tool can touch though is editing multiple values at the same time. As I worked my way on the very first day through sublime on a python project, I found myself wanting to change a value set on five variables. Instead of a default value of `None`, I wanted it to (effectively) read `kwargs.get(”, None)`, and I thought, “ok! let’s try multiple cursors!”.

I moved into position and slammed `ctrl+d` five times and there they were, five cursors ready to alter the default value of five members.

Afterwards, I reflected on the wonder of how practical multiple cursors really were. “Is this just some cheap skate refactoring tool?”. No. It’s much more powerful. I think that’s one of the many reasons I’m a sublime convert now and hope to see this idea spread through free tools in the future.

Thanks for reading, try sublime, hell buy sublime, and feel the preposterous range for which it defines.

xmonad

During my lunch break yesterday, i ran across a couple comments on slashdot like “maybe emacs just works and thats why ppl are using it” and “tiling window managers are awesome”. So in my usual, screw it – lets see, i fired up emacs (not the first time) which ended pretty quick and I did some quick googling on a decent tiling window manager to try.

Enter xmonad, written in haskell no less (something I was going to learn after I wrap up my C# starter project, a calculator with a lexer so I can insert arbitrary words into a math equation, does proper math too, not just “eval(x)”). Anyway, within 20 minutes, I was sold. A tiling window manager is the best thing since sliced bread. Solves my window management issues I’ve been trying to fix since who knows when, things I’ve looked to for answers from desktop effects like scale, expose, overview of multiple desktops with drag and drop of windows, all that pretty flurry graphical stuff that gives you good feelings but then you rarely use.

Key Points: 
  • Focuses on little to no need for the mouse
  • The only window decoration is a 1 px border that changes color if it’s focused, you’d be surprised how much faster programs start without decorations
  • with no decoration, dragging a window involves alt-click, which is all I do anyway so low barrier to entry for me, but floating a window is rarely needed
  • you dont minimize, you tile so no need for decoration buttons
  • alt+shift+c closes any window – this could be seen as annoying but everything centers around alt anyway so it becomes natural quick
  • Navigating between tiled windows is dead simple, I don’t lose track of what’s opened.
  • alt+space swaps to different layout arrangements (3 total by default) which handle any use case I have
  • Moving windows between virtual desktops is easy
  • Use on a multimonitor setup is where things really shine. If my first monitor is desktop 1 and my second monitor is desktop 4, I can easily swap them by selecting desktop 4 on monitor 1. No need to drag something over or anything like that.
  • I can push windows around way to easy

I could probably type in all sorts of blurbs that might not really get the point across. If you’re bothered with window management, give xmonad a try and read up their quick guide to usage. Doesn’t take much time to get use to it.

I’ve honestly been putting off getting another monitor for my setup, i just imagined it being a pain to use but xmonad makes me feel like a master of window management and now I’m eager to get another monitor or two.

AndroidWeb, package for handling concurrent requests with multiple receivers, sqlite caching, and a built-in content provider

Recently, TSG released AndroidWeb, a networking library for android (of which I’m currently maintaining and the original author). This is a great library for handling concurrent networking connections throughout the activity lifecycle, is backed by an sqlite database for caching, and has a ContentProvider allowing share by intent to other apps from the cache.

Check it out, fork it, put in feature requests, and tell me what you think.

https://github.com/dskinner/AndroidWeb

I’ll be doing some follow up posts in the next couple weeks to provide tutorials over specific features as well as providing those as examples in the git source.

Fast Web Development with Damsel for Python

Time feels surmountable when looking forward. In retrospect, I see surmise.

My title to this post feels like a real plug piece, but I guess that’s what happens when you’re looking to get indexed. Recently, I tagged two new versions of dmsl in git. If you’ve never heard of dmsl, please go check out the README on github to see what it’s all about.

https://github.com/dskinner/dmsl

I started off implementing haml in python which gave way to a unique direction for the project. I’ve been using it quite actively in work and am continuously enamored with its simplicity and power. Of course I wrote it and there might be a sense of bias, but this isn’t the first or second or third or fourth attempt I’ve made to write something like this. Each of those projects were short lived in the same way that once they were mostly complete, they just flat out didn’t “have it” for me.

0.3-stable is a wrap up of features and fixes over the course of many months of use and has, in the end, proven reliable. This tag is available on github. There were still some annoyances in the code related to speed and the dependency on lxml though that I badly wanted to fix.

The speed issue was solely in the use of a class inheriting python’s Formatter to handle extensions to string substitutions. These substitutions alone could take three times as much of the time as the parser for the whole document. Granted, on a whole, this was still quite a bit faster then a number of template engines out there, but I felt this could be drastically reduced.

The second issue was the dependency on lxml. Simply put, I wanted that to go.

So recently I addressed these issues and tagged 0.4 in git. I’ve also made this tag available on pypi. If installing from pypi, the only dependency is a build environment and python headers for building the C extensions. If building from github, you will need cython 0.15.1+ installed.

0.4 also has another change with how context variables end up in the templates sandbox. Previously, these items were packed in a kwargs dict available to templates. In 0.4, this is no longer the case and those items are unpacked in the environment for use. Planned though is the ability to revert to the old behaviour as needed for upgrading older installs.

While I feel the changes in 0.4 are drastic, unit tests pass and I’ve upgraded my personal projects successfully to do a little stress testing while also using it in another upcoming public project with success.

So check it out and tell me what you think.

daml 0.1.4 Release, A Python Markup Language for the Web

Since my initial post outlining daml as a haml-for-python, a lot has changed. Now it has become something more and plays a new role in a bigger project I am working on. Let’s look at some of the new syntax changes.

CSS Selector Style Attributes

Declaring attributes now follows css selectors appended to a tag hash. For example,

%html
    %body
        #content.main[attr=val][apples=yes] some content

Inlining Tag Hashes

One can now inline tag hashes. The functionality is still being worked on and is broken beyond a single tier. The intent is to handle use cases such as,

%html
    %body
        %ul#nav
            %li %a[href=/] Home
            %li %a[href=/contact] Contact

        #wrapper #container
            #top
            #content

Implicit Python Declaration

Since my last post, all python code had to be preceded with a colon (:). Now, this is only necessary when embedding a function call in plain text (or calling a filter). For Example,

def greet(s):
    return 'Hello, {0}.'.format(s)

nav = ['www', 'www2', 'www3']

%html
    %body
        %p A greeting for you. :greet('Daniel')
        %ul for x in nav:
            %li {x}

The trade off to the above is explicit line breaks for multiline text as similarly done in python,

%html
    %body This is some text \
        that spans multiple \
        lines.

I am still putting alot of thought into this portion of the syntax and it will most likely see some changes in a number of months. Explicit line breaks can become rather confusing when handling complex mixed contents, but again, tools such as daml aren’t particularly suited for writing page content, but rather establishing layout.

New Filters

I added two new filters as a test, these may be removed in future releases (and migrated to my personal set of daml extensions). They are for declaring css and js files. I think the syntax speaks for itself in function,

%html
    %head
        :css /css/
            main.css
            extra.css
            lib/that_lib.css

        :js /js/lib/
            utils.js
            js.js
            jquery.min.js
            etc.js
    %body
        %h1 I do enjoy filters!

I think such filters could be upgraded to support appending additional files, but again, I dont really have a place for such things that aren’t a necessity to a fully featured template engine.

In Summary

This release requires cython >= 0.13 and lxml. I use this regularly on Windows 7 x64 and linux. The 0.1.4 release can be found on pypi, http://pypi.python.org/pypi/dmsl/0.2 and you can follow development at github, https://github.com/dasacc22/dmsl

Python implementation of HAML

UPDATE Dmsl is an active project on github and I am looking for active developers that can submit any bugs they may come across. I have been using it in production environments for nearly one year as of 12/2011. Refer to the repository README for an in depth look at current development, http://github.com/dasacc22/dmsl/blob/master/README

So it has been a busy past few days. During which I have written up an implementation of HAML in python. The closest thing I saw to this before was GRHML, whose site seems non-functional and I had a hard time finding information on its status. So without further ado, let me introduce DAML, my HAML implementation.

:title = 'Hello World!'

%html
    %head
        %title {title}
    %body
        :include('header.html')

        :greeting = lambda x: '%p Hello, {0}'.format(x)

        :greeting('user')
        :greeting('me')
        :greeting('world')

        :items = ['a', 'b', 'c', 'd']

        :def is_last(i):
        :   return len(items) is i+1 and '(class=last)' or ''

        %h2 Loop Method 2 (python list comprehension)
        %ul
            :['%li{0} {1}'.format(is_last(i), x) for i, x in enumerate(items)]

        :include('footer.html')

Ok, now this isn’t all it does but let me talk about a couple things first. For one, this project, as of today, is only 3 or 4 days old. Its hard for me to recall b/c I’ve poured a lot of time into trying to make this run as fast as possible. Today on the other hand has produced a lot of slow downs and hackish code for implementing features like Django code blocks. For you XSL people (which would be me) that means a way to name and call templates from other templates. So anyway, a lot of this code from today (and yesterday) needs some love. With that said, lets go over some differences between HAML and DAML when it comes to marking up HTML.

You’ll notice right away that declaring tags is straight forward and precisely the same. I’ve never actually used HAML so I dont know its exact syntax, but I spent a lot of time perusing the documentation for HAML. Now the actual text processor in DAML is quite fast for building documents. At one point during development, when i was still testing speeds in comparison to HAML, I had results along the lines of 0.21ms processing time for DAML versus 2.4ms processing time for HAML. This was for plain-jane HTML declaration. Lets look at some of that now with DAML.

%html
    %head
        %title Good Stuff

    %body
        #header Some stuff here
            and indentions of plain text
            will all be part of this div

            .span while this child div whose class="span"
                will be embedded within the above div

            while this text is tailed
            and multilined too

        %p and heres some random content too
            %strong that can be played
            however you like.

        %p one thing worth noting is that new tags
            %strong need
            to be on
            %em new lines
            so keep that in mind.

All of this renders fine of course. But I still have a TODO list for handling stuff unrelated to python expression evaluating. Namely comments, escaping, whitespace control. This is pretty much in line with HAML thus far although I often see blank lines in HAML appended with equals-sign and I cant recall what for. Regardless, the next thing to note is the use of variables. Which loosely follows the string.Formatter (and would fully support it if not for speed issues at the moment, but soon hopefully). Lets go over an example of setting and using variables.

:title = 'Hello World!'
%h1 {title}

Simple, eh? Now here’s the deal. There is a sand-boxed python eval going on. All lines starting with :colons are getting added to an evaluation queue that gets compiled and eval’d. Currently you would tag variables just like using string.Formatter and in the future, this will support all the goodies associated with it. Currently though whats going on is its simply accessing the variable declared (versus string.Formatter being setup with the documents namespace and getting called). Never-the-less, this works fine so far. Lets look at some other things we can do with “:”

:l = []
:for x in range(20):
:    l.append(x)

%div
    :['%p {0}'.format(x+1) for x in l]

Notice first, that you can basically freely declare normal python code blocks. You can also declare functions to make use of as shown with the initial intro document. I would say, ideally, the syntax I would want to go for in the majority of cases is something like (for-loop with plain-text block not yet functional)

#body
    :items = ['a', 'b', 'c', 'd']
    :is_last = lambda i: len(items) is i+1 and '(class=last)' or ''
    %ul
        :for i, x in enumerate(items):
            %li:is_last(i) {x}

    #div.class(attr=val, attr2=val2) oh and here's this too

So this is something I am working towards, but you can totally declare :func(*args) inline right now and it works. Notice, going back to the plain-text that you can freely declare %tag#id.class(attr=val) in almost any order, the only except is you can not start a line with (attr=val) nor can you declare (attr=val)(attr=val) though the latter may be added. Part of my TODO list is to have the ability to span attributes across multiple lines.

Ill touch on one last bit here, I implemented something similar to django blocks (at its most basic level). Its a wee bit limited until i implement multiline filter options as part of the preprocessor I *just* started working on not but 4 hours ago. But let me show what the final syntax would look like

:extends('template.daml')

:block header
    %h1 {title} OVERRIDE!

Now the above doesn’t work until i finish a bit on this preprocessor but to show you how that will work, let me show you how the above does currently work

:extends('template.daml')

:block('header\n    %h1 {title} OVERRIDE!')

Now that does work, and as you can see, my preprocessor is basically going to go through for indented portions of text under colon directive calls and push it into the related function thats in the sandboxed globals. This basically means that you can write all sorts of text filters and everything really really easy b/c then back on your end you may have a need to do “blank” with “blah. So..

my_python_code.py
-------------------------
def my_filter(s):
    s = s.splitlines()
    for x in s:
        pass # do blank with blah
    return s

I’ve basically tried to unify a number of features of HAML here so that anything can be easily overridden or customized and extended. Nothing thats just built-in and untouchable. There’s even more to come surely but I am honestly way burnt out on this.

Let me lastly explain just how FAST this works currently. Now before I went in today/yesterday adding in all these crazy bits and pieces, I grabbed the bench suite from genshi that benchmarks all the most popular python templating engines. I wrote my template to do everything to spec as the others were doing and these were my results.

Mako: 0.38ms
DAML: 0.44ms
Cheetah: 0.66ms
Genshi-Text: 0.96ms

and after that it just trailed off into really big numbers with Genshi coming in at 1.5ms?? and django at 2.4ms?? So this thing has the potential to be FAST. Now since the hackery I’ve pulled in the past number of hours I’ve actually almost doubled my processing time and the above test for DAML now runs at 0.85ms but keep in mind i have done no optimization/refactoring/code-cleanup/obvious-bug-fixes-needed to the codebase i just recently committed. So that portion needs some love and I’d love to get the speed back down to what it was just early yesterday morning.

Now, if your interested in the project, pleeease check it out at github, http://github.com/dasacc22/dmsl and play around with it. I’m going to slow my development down to a crawl comparitively as I’m really burnt out. I wont be implementing any new features and I’ll be cleaning up my code base yet again and trying to write beautiful code to everything ive hacked together today/last-nite.

And please feel free to contact me by email or comments or however regarding this project. I would like to get the code cleaned up so that I can collaborate on different things to really bring this project up to par.

Thanks, and enjoy
http://github.com/dasacc22/dmsl

Line 6 UX2 and Linux

I originally wrote a bad/sad review about the Line 6 UX2 not working under linux as can be seen here:
http://line6.com/community/thread/17663

I was looking to sell it recently when I decided to give it another look over and try out the drivers done for the PodXT devices by someone as can be found here:
http://www.tanzband-scream.at/line6/

The 0.8.1 release failed to compile so i checked out the trunk of the development version
$> svn co https://line6linux.svn.sourceforge.net/svnroot/line6linux
$> cd line6linux/driver/trunk
$> make
$> sudo make install

To my surprise, it worked fairly ok. My line 6 ux2 device lit up and seemed ready to go. One of the first things I noticed was that there was a constant buzzing out of the right channel monitor. This was alleviated by simply plugging my guitar in. Next, I read through the docs to get some general info. I went ahead and fired up alsamixer, increased pcm output and zero’d out the monitor. Note: the docs say to set pcm output to zero as they can be much louder then the guitar monitor. I had no trouble with this when i increased output to 75, volume was normal.

So then I fired up jack and played around with a couple settings. To my surprise I was able to get latency down to 1.5 ms according to ardour (2.87 according to qjackctl)! Either way, this was much lower then I ever managed with my edirol which normally clocked in around 15ms and was littered with xruns that were very audible in the recording. Normally around 22ms I could achieve good sound with no xruns. After a short recording session, I noticed qjackctl had logged numerous xruns but I never heard a thing. Once during the recording, ardour disconnected from jack but this could be totally unrelated.

This is fine news indeed and prompts me to want to hold on to this device. The bad news is I have been unable to get the microphone inputs working. I haven’t had time to look into it fully but hopefully after exploring the line6linux docs some more, I can have some success with getting this working.

So all in all, this is great. What I would really like to see is Line6 spend a few resources on this project or doing something themselves. But that aside, hardware wise (i guess), this little bugger rocks!

Binding (and unbinding) JQuery Event Handlers in a Complex Web App

I’m commonly placed in a tough situation. Here I am, working on a web related project and I would like to implement a lot of complex features via javascript. JQuery is the first library I’ve used and it has allowed me to at least dream of a functional, yet complex, web application. Typically, the first 25% of the javascript related work is a total breeze, thanks to jquery. The next 30-50% of the work becomes totally convulated, thanks to jquery being my crutch, and never diving in and bothering to understand javascript all these years. The final percent is ussually left on a wish-list of too-buggy-to-deploy and wouldn’t-it-be-nice.

Here I am again, endeavoring to build a complex and functional web application and this time I really want to get it right, and low and behold, I’m beginning to run into one of those typical problems.

Binding Events with JQuery

JQuery really makes this easy. I love it. I remember first starting off and writing a little bit of code like this:

$('.button').click(function() {
  $.get('/some/stuff', function() {
    $('.somestuff').append(....);
  });
});

and what not. Then I discovered that beloved .load() function. Really cut down on a lot of legwork I was doing. But there was always a slight problem. I always ended up with conflicting binds. There were various ways I worked around this, among other problems before .live() was around, but it was always a bit of a conundrum. I think this mostly deals with the approach JQuery takes towards usability. The focus is quickly accessing an object and binding the specified event, nothing more. It makes no effort to force any sort of design paradigm, which leads me to wonder, how should I handle this? I figure the first place to look for inspiration are in libraries I already currently use that solve this sort of design pattern.

Pyglet Event Handling for OpenGL/Python

I’ve used pyglet for a while now. It has a few design patterns for handling events that go something like this:

# first, it allows for decorators over a method
@windows.event
def mouse_press(...):
    pass # and do stuff

# and also a window method to push handlers of class instances
class SomeStuff(object):
    def mouse_press(...):
        pass # and go

some_stuff = SomeStuff()
window.push_handlers(some_stuff)

# and likewise, you can pop those handlers
window.pop_handlers(some_stuff)

The first design pattern doesn’t really solve any particular problem of mine and is pretty much already what is going on using jquery events. But the second pattern is something I would have considered impossible for me to accomplish in javascript, I’ve just never understood the language enough to consider it doable. But recently, I’ve been coming to understand the language a lot more and wrote a small utility for making better use of prototypal inheritance in javascript, js.js, and it struck me that this would be pretty easy to implement.

A Design Paradigm for JQuery Event Handling

First, its important to understand a small bit about the js.js utility. It’s somewhat based off of Crockford’s function that prototyped an empty constructor with a passed in object. The difference here is that js.js calls magic methods (instead of being an empty constructor) and allows one to pass in multiple objects and functions to be prototyped.

The design consists of two parts. A jquery plugin in the form of $.events.push() and $.events.pop() and a prototype inheritance pattern, optionally using the js.js function.

This is the plugin code:

$.extend({
    events: {
        recurse: function(args, action) {
            for (var x in args) {
                var obj = args[x];
                for (var k in obj) {
                    if (typeof obj[k].bind != "undefined") {
                        if (action == "push") {
                            $(obj[k].bind).bind(k, obj[k], obj);
                        } else {
                            $(obj[k].bind).unbind(k, obj[k]);
                        }
                    } else if (typeof obj[k].live != "undefined") {
                        if (action == "push") {
                            $(obj[k].live).live(k, obj[k], obj);
                        } else {
                            $(obj[k].live).die(k, obj[k]);
                        }
                    }
                }
            }
        },
        push: function() {
            this.recurse(arguments, "push");
        },
        pop: function(obj) {
            this.recurse(arguments, "pop");
        }
    }
});

The scenario is this. A lot of ajax events are very similar and would work across different loaded pages/forms. Where necessary, one could override an event to add extra functionality:

controls = {
  index: function(params) { $(...).load(...); },
  form: function(params) { $(...).load(...); },
  save: function(data) { $.post(..., this.index); }
}

events = {
  keyup: function() { ... },
  submit: function() { ...; this.save(data); return false; }
}
events.keyup.live = '.main .filter'; // just demonstrating access to live and bind
events.submit.bind = '.main .form';

user = function() {
  this.init = function() {
    $('a[href=#user]').bind('click', this.click, this);
  }
  this.click = function() {
    highlight(...);
    // do some more stuff;
    this.index();
    this.form();
    
    $.events.pop(one, two, three);
    $.events.push(this);
  }
}

var User = prototype(user, events, controls, object);

$(function() {
  var u = new User({url: '/db/User'});
});

This allows for common parts to be factored out when shared by many different objects. Picture five other objects alongside User. Some might override events and call the original, for example, events.submit.call(this, …).

This is a little rudimentary to start, but its a start none-the-less.

Follow

Get every new post delivered to your Inbox.