I first heard about Python pandas from a friend at RenTech or AQR in the early summer of last year. At the time, the project was little more than a documentation page and a few wrapper methods around numpy. Since then, pandas has matured into a very useful and featureful addition to numpy, scipy,
Here's a fun example of how you might use my data on Congressional bill length and complexity. Imagine you want to understand the empirical distribution of Flesch-Kincaid reading level for Congressional bills and how this distribution is related to bill stage. A first step might be to visualize this relationship. Based on this
In case you're too lazy to dig into the full CloudSearch developer guide, here's a quick summary of how to install the CloudSearch API command line tools on Linux: Ensure that you have a JRE/JDK installed. Most distros will ship with something like OpenJDK, which is fine. If you'd like, grab the "Oracle" JDK
Since there are always questions of support for newly released models, I thought I'd put up a post on Natty Narwhal on the new Dell Precision M4600. I got mine last week and have just finished replicating my M4500 setup on it. The only issue I've run into is for my Quadro. NV claims that
Later this month, I'll be giving a keynote at a meeting on Law and Computation at the University of Houston. As part of the talk, I'm putting together an example of how I think machine learning and structured data can help build a better legal search engine. For the talk, I'll be building a subset
I've been playing with Twitter data over the last week, archiving Algerian, Egyptian, Iranian, and Chinese tweets. I thought I'd bring the story a little closer to home this time by archiving tweets from Wisconsin Union protesters on the #wiunion tag. Grab the dataset of 165,593 tweets here, and check out the two figure
Earlier this week, I posted a ~100k tweet dataset on the #25bahman protests in Iran. The corresponding figure of frequencies showed a strong presence on Twitter, with over 500 tweets per 5 minute period at peak. You can download the dataset or check out the figure in that post. I decided to take a quick
My first experience with R was in 2007 as a sophomore in undergrad. As part of a larger project on pricing day-ahead electricity futures, I wanted to cluster locational marginal price (LMP) data from the ISO-NE. Something like k-means is easy to plot and visualize in low-dimensions, but this data was better approached by hierarchical methods.
You may have heard about the recently leaked presentation on combating Wikileaks that was produced by employees of HBGary Federal, Palantir Tech, and Berico Tech. You may have also heard that Anonymous retaliated against HB Gary Federal for threatening to release their identities. I thought it would be interesting to run some analysis of the email networks
I wondered yesterday whether one of the Algeria/Yemen hashtags would dominate. In order to bring some numbers to the discussion, I grabbed all public tweets on #feb12 and #fev12 through this afternoon. You can see the results below: First off, note that all timestamps are UTC. Second, based on the scale, it's pretty clear that
