Let me introduce you to ...

Variable length opinions about arbitrary: things, situations and opinions (meta, right!).

0 notes

Mutt, patches and GitHub

Earlier this week, I saw this thread on r/vim, a few of these apps looked interesting but nothing too amazing, until I saw mutt, it immediately caught my attention, the gist of it: “an e-mail client with vim-like bindings”. I thought to myself: “this can either be fantastic or horrible”, I tried to install it in OSX and had no luck.

Later that day, I sent that link to my friend Santiago, who also though mutt was an interesting idea, so he installed it (in his linux box) and immediately sent me an e-mail from mutt. I was both intrigued and jealous, so I decided to give it another try, sure enough brew (the missing package manager for OS X) provides a formula to install mutt. However when I ran the executable I noticed that I didn’t have a few capabilities that Santiago described having, after more investigation i. e. running mutt -v, I noticed that there are a bunch of compilation flags to be passed to mutt’s configure file, all of which enable/disable: imap, smtp, header-caching, etc.

In no time I was downloading the source and compiling from scratch, it took me a while but the ./configure command ended up looking like this:

CPPFLAGS=-I/opt/local/include/ LDFLAGS=-L/opt/local/lib/ ./configure --with-ssl --enable-pop --enable-imap --enable-smtp --with-sasl --enable-hcache --prefix=/usr/

Note that setting the CPPFLAGS variable is only needed because I used macports to install some of mutt’s dependencies.

Anyways, I was up and running, but it looked too plain and not very usable (see screenshot below) which made me rethink why was I doing this.

Note that I was doing all this around 11:00 pm and I still had a lot of things to do before waking up the next day, I decided to hand over this win to mutt and continued working.

The next day I was using Mail.app and of course I was getting tired of it, it seriously has decremented its usability lately (at least for 6 accounts with a total of 40K e-mails). Hence I decided to load mutt and answer a couple e-mails with it. Later that day I found that Steve Losh had blogged about mutt with this more than appropriate title “The Homely Mutt”. That blog post was super useful and I ended up going from a crappy looking and barely usable mutt to something like this:

Note that the colors are a custom modification of Solarized plus some additions. Like being able to highlight the color of patches that come in e-mails, special thanks go to my friend Adam and Karel Zak’s blog post:

If you are a heavy GitHub user, this is IMHO where mutt shined as being highly customizable, you can set your color scheme to highlight diffs coming in an e-mail, for example look at the following screenshot where the patch in which I made a comment is now colored in green for the additions:

These are the lines in my configuration that make this possible:

# diffs coming from github, thanks to @adamrp, for making this work better

color body green color234 "^(\>\ )*diff \-.*"

color body green color234 "^(\>\ )*index [a-f0-9].*"

color body green color234 "^\(\>\ )*\-\-\- .*"

color body green color234 "^(\>\ )*[\+]{3} .*"

color body brightgreen color234 "^(\>\ )*[\+].*"

color body brightred color234 "^(\>\ )*[\-].*"

color body cyan color234 "^(\>\ )*@@ .*"

I’ve been using my e-mail via mutt for the past two days and I have to say I cannot be happier.

Filed under mutt terminal mail config

0 notes

Statwittstics, a data analysis journey

In this blog post I am going to talk about a program that I wrote, initially motivated and conceived as a consequence of my curiosity to see what my twitting activity had been in the past few months. I was curious to see at what time of the day was I twitting the most, which days was I twitting the most, etc. So I came up with Statwittstics.

In a nutshell what I wanted to do with Statwittstics was to have a dashboard to ask questions about the activity of a Twitter user and do this in a very interactive way. At the time I looked for existing solutions, but none of them seemed to fulfill what I was looking.

Finding a way

I had an idea and I knew what I wanted, but just like with many other things, I didn’t know how to implement any of what I wanted nor I was sure if I should be doing this. Oddly enough I came across a talk from Apple where they go into detail about blocks in Objective-C, a feature that at the time was novel and had a lot of interesting use cases. I watched this talk and was intrigued by the power of blocks, so as I was searching through the catalog of talks in the developer resources at Apple’s Developer Portal, I found a talk about Twitter’s native API for iOS. I watched the talk, what a great talk, and followed along with the examples. As I completed one of these demos the following thought struck my mind:

Hey … I could write that Statwittstics thing in Objective-C, and have it be an iPad app, this in turn might be useful for social media managers who are interested in tracking their “performance” or just for curious users.

I was excited, this seemed an excellent opportunity to:

  • Dive into using blocks.
  • Doing some relatively simple data analysis.
  • Writing an iPad application (I had done this before but never one that was not trivial).
  • Having a pet project that I could devote time whenever I was bored.

I kept playing around with the cool things that the API allowed you to do and as I did my amusement kept growing and growing. I was sure that this was going to be the right way to make Statwisttics come to life.

A familiar sounding name

With technical curiosity now being the main driver that I had at the moment, I started looking for ways to take care of the visualization capabilities (my initial intents were rather simple) that I wanted Statwittstics to be capable of. Some googling around lead me to a few projects that were capable of making pie charts, scatter plots, etc, but none that was an actual plotting framework with a well thought API.

Luckily for me a post in StackOverflow lead me to Core Plot, which at the time was hosted in Google Code, this was a great solution. Core Plot was a mature, active and well supported open source project, needless to say, I was extremely excited.

As I dug around some of the examples, I quickly realized that I had read before about Core Plot, and not only that, I had even tried out some of the demos and was pretty happy with them. This was great, Yoshiki from the past had already approved this framework.

I searched for a couple tutorials to get a better understanding of what was going on in the examples, started playing around with some of them and started creating some of my very own plots.

Core Plot is carefully well designed and feels a lot like the way a plotting framework would be structured if Apple authored one. Regardless of this, programming a plot always requires a lot of code, why you ask? Well because if you want to have control of how you want to display the data, you are going to need to customize a variety of parameters, this seemed boring but I kind of embraced it and wrote a set of wrappers to do what I needed, I called it the PBKit.

Assembling the pieces

Now that I had the plotting backend and some of the basic functionality to retrieve twits from a user I knew I had to start prototyping the user interface. At the moment I came up with a really simple one-screen app that would load the data for a user (the owner of the iPad) and plot it.

To start my prototypes I decided to use OmniGraffle, and a set of stencils designed for rapid prototyping of iOS user interfaces. Using OmniGraffle I started thinking more on the type of things that I could add to the screen and the things that would be handy to have so the few things I came up with were:

Main Screen: a simple screen with a summary of the user bio, a plot of the selected activity and some selectors to choose what to plot:

image

Plot Manipulation Actions: I wanted the user to be able to see what the data that was being plotted was, so I wanted to be able to show a (a) cross-hair kind of thing as the user touched the plot. Alongside with this I also wanted to be able to see the twits that were composing a data point (b):

a

image

b

image

Statistical Stalking Searching: A new screen would show up if you so decided that you didn’t want to look at your own data but that you instead wanted to look at the data from another user, the first thing you would need to do would be to go to the user menu on the right (a) and then search for the username you wanted to gain an insight into their twitting activity:

a

image

b

image

In retrospect this was probably the most important part of this project, as I was not very familiar with the available widgets for the iPad iOS and I didn’t really know how would this end up looking; hence I wanted to avoid programming the way Bret Victor defines it:

Programming is blindly manipulating symbols

The birth of Statwittstics

After a couple weeks I came up with a functional version of Statwittstics, it was working, I was able to use it and I was having a lot of fun using it, needless to say it was buggy and rough around the edges. As I continued working on it and adding more functionality (sharing on twitter, selecting which user you want to log in as), I knew there was something lacking, something quintessential to any project: a logo.

I contacted my good friend and graphic desginer Ana, explained her my idea, of course she told me: “You are crazy that’s a weird thing to do …” but she agreed that in some of her spare time she would work on a logo for Statwittstics, I was thrilled and excited. The result:

image

After a couple weeks I had already finished the initial prototype with all the functionality that I wanted, the program was “stable” and was usable. I was extremely happy with it. See for example the implementation of the main screen at an early stage:

image

And then an updated version a couple weeks later, with a few extra controllers:

image

And of course we had to be able to share these sweet plots on twitter so hence the following:

image

And this is how searching for a user looks like:

image

If you are curious about this and more, I’ve made the code available on GitHub, you can find it here:

http://github.com/ElDeveloper/Statwittstics

I encourage you to try it out, depending on when you try it out, you might find that some settings in Xcode have changed so, please let me know if this is the case so I can update this.

Filed under statwittstics twitter data analysis analytics big data github ios project programming software engineering software api twitter api

3 notes

Mechanical Typing

Over the past few months I’ve had to use my keyboard more than I’ve ever had before, of course I had no issue doing this because I was enjoying every second of the crafting, tinkering, advancing and discovering that my typing usually involves; everything or almost everything I’ve been doing requires a keyboard and/or a mouse.

On busy days all the fun that I would get from typing would lead to pain on my forearms and hands, not the type of pain that you have after exercising a part of your body at the gym but a type of pain that get’s to be hard to ignore, even to do the most simple tasks. This pain became a huge limitation to my creative process.

After talking to my mother and father (both health experts) they suggested me to look for a more ergonomic peripheral or to just plain stop typing or I would really screw up my wrists and or fingers due to severe pain, muscular atrophy, etc. So I started looking for such a thing and the first thing I came across were these really odd shaped keyboards:

image

A weirder one:

image

I even got a chance to try one of these when I was visiting a friend in NYC but I didn’t like it at all, plus the fact that this would make me dependent on that specific shape to feel comfortable which for once was never going to be found in a laptop and not to mention that it’s very unique form would definitely change the way I type make it extremely uncomfortable when not typing in this keyboard (this is a consideration I had to make given the way I work i. e. everywhere/anywhere).

Then I came across the CODE Keyboard (see below) which my parents even offered to buy for me as a present as I was being very indecisive about whether or not I should be buying a keyboard like that, of that price range without even trying it out (the CODE keyboard priced at 150 dollars). Before I knew the keyboard had sold out.

image

As I was lurking WASD keyboard's website (the makers of the CODE keyboard), I found out that they provide a plethora of ways to customize a keyboard, from keycap colors, switch type to the print in each of the keys. So one saturday morning at 7 am, I decided to open Adobe Illustrator and start the design of my custom made keyboard. The end result, what I decided to call the Y1:

image

The main idea of the keyboard is to show the hexadecimal representation of the ASCII characters for all the symbols but for the letters a-z and for the enter key and tab key, which I decided to use ‘\n’ and ‘\t’ respectively. They keyboard has Cherry MX blue switches and needless to say, it’s loud. But it is also an extremely comforting typing experience. I couldn’t believe how pleasing it was to type in a mechanical keyboard, my reaction was such that I had to carry this from home-work and from work-home on a daily basis for a good two weeks, then I got tired of doing this and decided to buy another keyboard.

Initially I wanted to buy the exact same keyboard, but I couldn’t find a way for WASD to get it in a reasonable time (less than 4-5 months). Hence I decided to go with the Das Keyboard for the Mac:

image

So now I have the WASD keyboard at work and the Das keyboard at home. After switching to this new world of clicks and clacks, I have to say: I’m an extremely happy costumer. And lastly if you ask me: “Is the price worth it?" I will respond: It is worth it for me, working on these keyboards gave me back the freedom that I would usually get from typing and added an extra comfort to it.

Filed under das keyboard mechanical keyboard wasd keyboard wasd typing software

0 notes

Contextualized Timeline of Sequencing Cost

Earlier this year I came across a tweet from Thea Whitman where she drew a contextualized version of the increasingly reduced costs in sequencing technologies as reported by the NIH. This plot is widely used in presentations commonly when the presenter is trying to make the point that sequencing is very cheap now as compared to previous years and that this trend has a much more steep slope than Moore’s law.

Whilst this is a useful graph and mainly communicates two really important ideas, it is a figure that a lot of people use and if you are at a meeting where there are several presenters, chances are you will see this plot a few times.

Fast-forward to a couple months later, I found out that Matplotlib is capable of producing XKCD-like plots i. e. the plots will look hand-drawn.

Hence with these two things in my mind, I decided I should be able to recreate the drawing that Thea made to hopefully provide an alternative to NIH’s original plot:

image

I decided to do this using the IPython notebook given that it is an extremely useful tool for prototyping, analysis and processing (among many other things). With the IPython notebook, it’s easy to share code and plots in a unified document; hence I created the following repository with the notebook itself. If you want to take a look at it see this link.

Filed under ngs illumina qiime microbes genomics metagenomics science matplotlib python

1 note

How I tricked Wolfram Alpha with Twitter’s API

This is the story of how I tricked the marketing team of Wolfram Alpha back in 2009, more specifically the social media group that was in charge of a campaign called Holiday Tweet-a-Day. The idea there was that at a random time during the day, these guys would post a tweet, which had to be retweeted as fast as possible and the first person to do this would win something. So I tried a couple times, this was a boring and tedious process and the only time that I got really really close to do this was when I saw the tweet on my phone and immediately tried to re-tweet it, but I couldn’t, because I was not fast enough using my touchscreen.

So I got tired of this specially because it was a really tedious and boring process, but I got to a point where it almost felt like a challenge, luckily for me, earlier that year I had taken Stanford’s CS193P class online, in this course you learn how to use some of the key Frameworks in iOS to create usable mobile apps. Through this course you also learn how to create a bare-bones twitter client, which was able to tweet, read tweets and a few other things. So I decided to write a small program to pull down all the data from the guys at wolfram and re-tweet as soon as it could.

The program is not even 150 lines, since I had no desire to make it clean nor pretty it didn’t take me long to write it (perhaps 30 mins or 20 mins). In short this program would pull down @Wolphram_Alpha's twitter timeline, and check if there was a new tweet that could be re-tweeted, if it found a new tweet it would immediately create an post a new tweet from one of my two Twitter accounts.

So I started the program checked on it for about 5 mins and then left my computer, I think I had dinner with my family then when I went back to my computer, the logs showed that it had posted something already. First thing that crossed my mind was “I probably screwed up and this program is not working correctly”, so I decided to check what happened  and I realized that it had indeed found a new tweet and successfully re-tweeted it. A few minutes later I received a direct message from one of the guys of Wolfram Alpha.

image

They kindly sent me a Wolfram Alpha spikey and a Wolfram Alpha shirt. I was super happy and exceedingly excited but not because of the gifts themselves. The reason why I was so joyful was the fact that I was able to effectively automate a process that some could argue did not need to be automated. At this point I was finishing my second year at engineering school, so this was one of my first few approaches where I was able to see myself crafting a solution that immediately had a personal benefit.

Just in case you are wondering, this is the tweet that made me a winner that day:

And if you are curious about the program, I have uploaded it to GitHub and you can find it here.

Filed under wolfram github ios cs mathematica wolfram mathematica software engineering tricking twitter api

0 notes

Sublime Text

I’ve always said that if I could marry a piece of software, that would probably be Altium Designer (a next generation electronics design program), there’s no easier way to go from an idea to a hardware design than this beautiful integrated development environment. It incorporates multiple aspects of the design and implementation into a single and consistent user interface. Their documentation is extensive, clear and professional.

The program is targeted for professional level applications and is one of the most stable systems that I’ve ever used (note that this program only runs on Windows).

I spent a long time using Altium as an undergrad and I got the chance to design and fabricate a variety of boards, going from an schematic to a real board cannot be easier. I’ve tried some other programs similar to this one, but none of them were nearly close to the mature state in which Altium is.

The reason why I’m talking about Altium is because if that’s the program that I would marry, Sublime text is the program I would cheat on Altium with. Last year I got to a point where I realized that I needed a multipurpose text editor that I could use, I already had a bunch of them (TextMate, VIM, Emacs, TextWrangler, nano, pico, Coda) installed in my computer but none that I could use on a daily basis efficiently. So I embarked on a search for a good text editor that could fulfill my needs, it had to be fast and be very lightweight with support for as many programming languages as possible.

On this journey, I tried a few of them for short periods of time. As a long time Xcode user, I of course tried to force myself to use Xcode for everything that I needed to do, however it is nearly impossible to use Xcode as a multipurpose text editor. If you want to quickly edit a plain text file you’ll fell like you are using a bazooka to hammer a nail. Python support is not that good and really the thing that it is the best for is Objective-C programming (hands down).

The next one was Coda, one of the thing that I loved the most about Coda was it’s FTP integration, it’s close to magic, though I have to say that I do not like at all it’s lack of low-level customizations. I tried Coda 2 and got a similar feeling.

I was lost and wandering around without an editor, then I came across a blog post talking about Sublime Text, my first thought was: “Sublime Text is a stupid name for a text editor" (I still believe this). However the first thing that I was amused by is it’s general purpose auto-completion, the next thing that caught my attention was the huge catalog of plugins and customizations that you can have with the program.

After a lot of testing (some could say usage) I ended up adopting this program and can say that now, getting close to a year and a half, I’m still happy with it.

One of the downsides is that I spent quite some time trying to make it look like Xcode. The first thing was to emulate the syntax highlighting, lucky for me someone had already done this for TextMate and guess what! Sublime Text is compatible with Text Mate’s color schemes. All I had to do was to tinker it a little and I came up with something that I felt more comfortable with.

A couple weeks ago the guys at Sublime Text released Sublime Text 3, which works great and is definitely faster than it’s predecessor. it’s not until you are editing 20K lines files that you appreciate performance.

If you want an advice on how to select a text editor, my recommendation is to try them all, until you are convinced that you are making the right decision. Do not just adopt a text editor because “it’s the one that everyone at work uses”, select a text editor because you feel comfortable with it and you feel it adjusts your workflows the best.

Filed under sublime sublime text 2 sublime text 3 textmate xcode software

0 notes

The DNA Data Deluge - IEEE Spectrum

A co-worker originally sent this article to me last week, I just had the ten minutes to read the whole text and I have to say that, it’s an intelligible source of general knowledge about terms, problems and goals that computational biologists deal with everyday.

Definitely worth a read if you like the topic or if you are not familiar with the field and would like to delve a little more into it and see “what’s all the fuzz about it”.

Filed under dna illumina sequences genomics

0 notes

Launch Services

In my humble opinion, Apple has a very unique and beautiful way to create software, however some of their APIs are not so easy to understand at a first glance, still very robust and reliable. With this in mind, there have been a couple times where I have found myself misunderstanding what the documentation was intended to communicate and therefore it is usually my fault when something is not working how it is supposed to. 

That being said, I had a very unpleasant experience last weekend when I was learning about the Launch Services within App Kit. One of the things that Launch Services is good to, is to have an application interact with Finder or with the Login Items of the system, something pretty cool and useful. Everything was going well, and I was able to programmatically add an item to the Finder Sidebar. So once that was done I was happy, but then I thought that it would be cool to add an icon to that item and oddly enough I found out that the current function I was using to add the item had an option to add an icon, so I proceeded to add the icon, but it didn’t work at all. And I continued to try and try and try, without any luck.

Once my pride was in the floor, I realized that the all-mighty internet should already have an answer for this, so I found the following sources of knowledge:

And I tried some of the suggestions, I tried different ways but I couldn’t get a custom icon to show with the item in the Finder sidebar. I finally found a thread in the Apple developer forums with a user mentioning this has been reported to Apple’s bug tracker Radar

Given this my conclusion was to just let this go, though then I found that Radar bugs are taken into account given the number of reports posted in the tracker, so I Issued a bug, let’s hope for this to be solved sometime soon.

Filed under apple appkit osx finder bug software

0 notes

Macfusion on OS X Lion 10.7 and Mountain Lion 10.8

A few days ago I was about to install Macfusion, which if you don’t know already, is a great project that aided by the FUSE project or rather the macintosh port MacFUSE is capable of providing support for users to mount filesystems over the network. It is all good and both projects are open source but the problem is that as far as I know the software is only supported for 10.5 and 10.6 i. e. Leopard and Snow Leopard. And even worst not even MacFUSE is supported as it has turned into abandonware.

Using Macfusion in OS X 10.7 Lion is rather painful, a few bugs here a few bugs there, but some of them are fixable by the regular user, which now have to rely on OS X FUSE and SSHFS two projects that are in active development.

Doing some quick google searches I was able to solve a few of the issues, still some other issues were still there, so I decided to take a look at the source and I realized that there is a bunch of people who have identified the same issues using github’s network feature and even more awesomely they have fixed them.

So this is a call to everyone and anyone who find this project useful and want to try to bring it back up. I have made a few modifications on this repository, be sure to see the commit history to see a detail of the modifications.

Filed under mac osx lion mountain-lion software open-source source-code macfusion